TeleChat 至 Xing4:在国产芯片和框架上推进前沿智能体模型

媒体报道 Hugging Face Papers 原文语言:中文

Xing 4 系列大语言模型基于华为 Ascend 910C 芯片构建,使用 MindSpore HyperParallel 在 8,192 张卡上预训练至 2,850 亿参数,并由 TeleGym 智能体执行平台支持以实现高效推理和智能体任务。

全文

We present Xing 4, a series of large language models that extends the TeleChat lineage through coordinated advances in model architecture and training infrastructure on Ascend. This report describes the design principles and systems engineering behind the series, spanning architecture selection, large-scale pretraining, and reasoning and agentic post-training. We study how sparse activation, attention design, model depth and width, and speculative decoding balance model capacity with computation, memory access, and execution overhead. Our pretraining infrastructure builds on MindSpore HyperParallel, combining topology-aware hybrid sharding, efficient distributed optimizer execution, and coordinated management of long-sequence workloads and activation memory. CPU affinity and core isolation further reduce host scheduling interference. Together, these mechanisms support the pretraining of a 285B-parameter model on 8,192 Ascend 910C cards, providing a concrete setting for the architectural and systems studies. For post-training, we introduce TeleGym, a unified agent execution platform that connects execution, evaluation, and trajectory construction to policy optimization. Disaggregated asynchronous execution, bounded oversampling, and partial rollout accommodate heterogeneous task durations while preserving trajectory continuity across policy updates. Together, these advances establish an Ascend-oriented training foundation for the continued development of Xing 4 across model scales and increasingly demanding reasoning and agent workloads.

View arXiv page View PDF Project page Add to collection

正文以来源语言显示。

参考来源

← 返回每日AI报告