Hugging Face Papers ↗ · 2026-11-17
TPHC:通过张量并行分组更新改进 Hyper-Connections
Media reports Hugging Face Papers Source language: Chinese
Paper abstract (author-reported, not re-tested by this site): Hyper-Connections (HC) expand the residual stream of a Transformer into N parallel streams, and manifold-constrained HC (mHC) stabilizes them at scale.
Full text
TPHC combines tensor-parallel shards with hyper-connection residual streams to enable more independent stream updates, improving language model loss and zero-shot reasoning with modest training overhead.