Rosetta-inference

模型发布 Hugging Face 模型 · tencent 原文语言:中文

官方机构模型仓库创建于 2026-06-26。仓库创建时间不等于正式发布日期。任务类型:未提供

全文

🪨 Rosetta: Composable Native Multimodal Pretraining

Escaping the Forgetting-Synergy Dilemma in Native Multimodal Pretraining

Xiangyue Liu 1 , Zijian Zhang 2 , Miles Yang 2 , Zhao Zhong 2 , Liefeng Bo 2 , Ping Tan 1*

1 HKUST    2 Tencent Hunyuan

Figure 1. (Left) Performance on MMLU (language ability) across composable pretraining stages (LM → +MMU → +T2I). Standard MoE and structurally-isolated MoT suffer catastrophic routing collapse upon integrating continuous generative objectives. Rosetta maintains a stable semantic anchor throughout all stages. (Right) Qualitative image generation results from the Rosetta model.

🏗️ Architecture

Figure 2. Rosetta FFN. Three mechanisms enable non-destructive modality expansion: (1) Unified Attention — globally shared QKV projections preserve dense cross-modal interactions. (2) Composable FFN — modality-specific plug-and-play experts (Text / ViT / VAE) are bridged by a single Global Shared Expert that anchors foundational knowledge. (3) Conflict-Free Optimization (MAOP) — surgically neutralizes destructive gradients with zero memory overhead.

📊 Benchmarks

✍️ Citation

正文以来源语言显示。

参考来源

← 返回每日简报