Hugging Face Papers ↗ · 2026-11-10
PolyRoute: A Twelve-Language and Code-Mixed Benchmark for Typed-Decision Routing, with Baselines and a Leaderboard
Research Hugging Face Papers Source language: Chinese
论文摘要(作者报告,未经本站复测):Typed-decision encoders answer a question about a text in one forward pass, but their behaviour outside English is rarely measured on a fixed yardstick.
Full text
Typed-decisionencodersansweraquestionaboutatextinoneforwardpass,buttheirbehaviouroutsideEnglishisrarelymeasuredonafixedyardstick.WereleasePolyRoute:322native-languagesupportmessagesin12languagesand9writingsystems,156code-mixedorromanisedmessages(Hinglish,Spanglish,Arabizi),967optionalnativetrainingrows,andaprotocolforathree-waydepartment-routingdecision.LabelscomefromQwen3.8-27Bwithablindsecondpass(91%ofcandidateskept);anindependentmodel,gemma4-31b,agreeson97.2%oftestitems,andthelargestcharacter5-gramsimilaritybetweenanytesttextandanytrainingtextis0.33.Aleaderboardof13systemsgives61.2%fortheEnglish-rootcheckpoint,77.3%fortheshippedmultilingualoneand96.6%forthebestfine-tune;TF-IDFwithlogisticregressionreaches91.9%.Onfourwithheldlanguagesthethreemultilingualsystemsscore77%,90%and93%.Oncode-mixedtexttheEnglishroot(85%)isnominallyaheadoftheshippedmultilingualcheckpoint(79%;p=0.134).Thesplitresolvesdifferencesonlyaboveabout4points.AlltextsandlabelsareLLM-generated.
View arXiv pageView PDF Add to collection