Hugging Face 模型 · tencent ↗ · 2026-06-26
Rosetta-inference
Model release Hugging Face 模型 · tencent Source language: Chinese
官方机构模型仓库创建于 2026-06-26。仓库创建时间不等于正式发布日期。任务类型:未提供
Full text
🪨 Rosetta: Composable Native Multimodal Pretraining
Escaping the Forgetting-Synergy Dilemma in Native Multimodal Pretraining
Xiangyue Liu 1 , Zijian Zhang 2 , Miles Yang 2 , Zhao Zhong 2 , Liefeng Bo 2 , Ping Tan 1*
1 HKUST 2 Tencent Hunyuan
Figure 1. (Left) Performance on MMLU (language ability) across composable pretraining stages (LM → +MMU → +T2I). Standard MoE and structurally-isolated MoT suffer catastrophic routing collapse upon integrating continuous generative objectives. Rosetta maintains a stable semantic anchor throughout all stages. (Right) Qualitative image generation results from the Rosetta model.
🏗️ Architecture
Figure 2. Rosetta FFN. Three mechanisms enable non-destructive modality expansion: (1) Unified Attention — globally shared QKV projections preserve dense cross-modal interactions. (2) Composable FFN — modality-specific plug-and-play experts (Text / ViT / VAE) are bridged by a single Global Shared Expert that anchors foundational knowledge. (3) Conflict-Free Optimization (MAOP) — surgically neutralizes destructive gradients with zero memory overhead.
📊 Benchmarks
✍️ Citation