Hugging Face Papers ↗ · 2026-11-17
Chorus: Shared N-gram Embeddings with Layer-wise Adaptation
Media reports Hugging Face Papers Source language: Chinese
Chorus shares a single set of n-gram embedding tables across all layers, enabling each layer to project and rescale the embedding using its hidden state. This approach achieves better quality with much smaller tables and lower optimizer memory.
Full text
Chorus shares one set of n-gram embedding tables across all layers, letting each layer project and rescale the embedding using its hidden state, achieving better quality with much smaller tables and lower optimizer memory.