Hugging Face Papers ↗ · 2026-11-12
论文提出统一多模态模型 UniSpark-9B
研究论文 Hugging Face Papers 原文语言:中文
论文提出 UniSpark-9B,一个统一多模态模型,采用混合注意力同时处理视觉理解与生成。它基于每个分支约 9B 参数的 MoE Transformer 主干,支持视觉问答、文生图、指令编辑和交错图文预测;结果来自作者报告,尚无独立评测。
全文
We present UniSpark-9B, a unified multimodal model that investigates hybrid attention for visual understanding and generation. Built on a Mixture-of-Transformer-Experts backbone with approximately 9B parameters in each branch, UniSpark combines full attention for explicit token interactions with GatedDeltaNet linear attention for compact context propagation. Autoregressive text prediction and visual flow matching jointly support visual question answering, text-to-image generation, instruction-guided editing, and interleaved text-image prediction. In interleaved generation, noisy visual targets are isolated from subsequent predictions, while clean visual representations provide context for later outputs. A blockwise protocol enforces this distinction through attention masks in full-attention layers and recurrent-state retention rules in linear-attention layers. The selected training sources contain 159.7M generation samples, 213.4M understanding samples, and 624.8K interleaved sequences. A four-stage curriculum jointly trains multimodal understanding and visual generation, refining answer accuracy and instruction following while extending image synthesis to 2K. UniSpark scores 2470.3 on MME-S and 0.83 on GenEval. Qualitative examples further illustrate its capabilities in interleaved image generation, video synthesis, and video editing. Together, these results demonstrate that hybrid attention provides an effective backbone for unified multimodal understanding and generation.
View arXiv page View PDF Project page Add to collection
正文以来源语言显示。