WorldDraft: If Code Builds the World, Can Video Models Just Render It?

研究论文 Hugging Face Papers 原文语言:中文

论文摘要(作者报告,未经本站复测):Multimodal coding agents decouple 4D world construction from visual generation, repurposing video models from world generators toward generative renderers.

全文

Multimodalcodingagentsdecouple4Dworldconstructionfromvisualgeneration,repurposingvideomodelsfromworldgeneratorstowardgenerativerenderers.Thisdivisionoflaborraisesafundamentalquestion:ifcodebuildstheworld,canvideomodelsjustrenderit?Areliablegenerativerendererhingesontwocomplementarycriteria:(i)worldpreservation,retainingprescribed4Dkinematicsandcamerapaths;and(ii)visualrealization,persistentlybindingtext-specifiedtargetattributesovertime.Toevaluatethiscapability,weintroduceWorldDraft,abenchmarkof432strictlypairedtestcasesacross48executable4Dworldfamiliesspanningsixdynamicchallengecategories.Undera3times3factorialdesign,WorldDraftorthogonallycrossesdraftgranularity(abstractproxytoprevis)andappearancealignment(neutraltoconflicting)underidenticaldynamics.Extensiveevaluationsacross12representativemodelsrevealthat:(I)worldpreservationandvisualrealizationremaininstarkoperationaltension;(II)richerdraftscaffoldinganchorsstructurebutparadoxicallyimpairsappearancefidelity;and(III)conflictingsourcecuestriggerseveresourceleakage(upto64%),exposingfragilecross-modaldisentanglement.Tomitigatethesebottlenecks,weproposeWorldDraft-Primer,aplug-and-playvisualpromptingstrategythatconsistentlyimprovesbothobjectivesacrossfrozenvideorenderers.Together,WorldDraftandWorldDraft-Primerlayafoundationforbridgingprogrammatic4Dsimulationwithvideogeneration,advancingphysicallygroundedgenerativerendering.

View arXiv pageView PDF Project page GitHub 13 Add to collection

正文以来源语言显示。

参考来源

← 返回每日简报