Hugging Face Papers ↗ · 2026-11-07
Test-Time Scaling for Diffusion Language Models with Frequency-Aware Remasking
Research Hugging Face Papers Source language: Chinese
论文摘要(作者报告,未经本站复测):Diffusion language models (DLMs) generate text through iterative denoising, where decisions about which tokens to commit or remask strongly affect output quality.
Full text
Diffusionlanguagemodels(DLMs)generatetextthroughiterativedenoising,wheredecisionsaboutwhichtokenstocommitorremaskstronglyaffectoutputquality.Existingcertainty-awarestrategiesrelyoninstantaneousconfidenceorentropy,buttheselocalsignalscanbemisleadingondifficultreasoningproblems.WeformulateDLMtest-timescalingasadaptiveallocationofbothinferencecomputeanddecodingstrategies.Ourtwo-stageapproachusesafastbasesamplertosolveeasyquestions,thenappliesfrequency-awareremaskingtotheremaininghardquestions.Thisstrategytrackshowofteneachtoken'spredictionchangesacrossdenoisingstepsandpreferentiallycommitsfrequentlychangingtokens,encouragingsamplestoexplorealternativereasoningpathsratherthancollapseontothemodel'spreferredtrajectory.ExperimentswithLLaDA-8B-InstructandDream-7B-Instructshowconsistentimprovementsoverastrongadaptiveallocationbaseline,withabsoluteaccuracygainsofupto3.73percentagepoints(LLaDA)and7.33percentagepoints(Dream)onMATH-500,andabsolutecoveragegainsofupto11.39percentagepoints(LLaDA)and10.98percentagepoints(Dream)onHumanEval.Onquestionsunsolvedbythebasecertainty-awarestrategyafter100samples,frequency-awareremaskingrecoverssolutionsfor3.75%to18.75%ofexamples.
View arXiv pageView PDF Add to collection