TPHC: Improving Hyper-Connections with Tensor-Parallel Grouped Updates

媒体报道 Hugging Face Papers 原文语言:中文

论文摘要(作者报告,未经本站复测):Hyper-Connections (HC) expand the residual stream of a Transformer into N parallel streams, and manifold-constrained HC (mHC) stabilizes them at scale.

全文

TPHC combines tensor-parallel shards with hyper-connection residual streams to enable more independent stream updates, improving language model loss and zero-shot reasoning with modest training overhead.

正文以来源语言显示。

参考来源

← 返回每日AI报告