NexSight:用于细粒度代理式长视频理解的可验证任务

媒体报道 Hugging Face Papers 原文语言:中文

NexSight 构建了可验证的长视频任务,要求模型在提供证据的前提下回答计数、顺序和状态变化等细粒度问题。通过 SightForge 管线和 SightLoop 框架,模型必须在盲验证器可重建答案的情况下提交答案。实验显示,基线模型在 SightLine 上仅达 54.52%,而在提供标注证据时可提升至 79.35%;经过 6 K 示例微调的 Sight‑35B 将交互准确率从 21.29%…

全文

Fine-grained questions about long videos ask how many times an action occurs, in what order events unfold, or exactly when a state changes. Answering them can require finding brief moments scattered across hours of footage. Agentic models that decide when and where to look promise to find these moments, yet this ability is hard both to measure and to learn. Sparse supporting evidence may establish that an action occurred without establishing its total count; verifying timeline-wide claims requires checking for missed occurrences and double counting. Answer-only supervision, in turn, does not explicitly supervise the search for evidence. We present NexSight, which builds verifiable tasks to both evaluate and train agentic long-video understanding. Its agentic pipeline, SightForge, accepts a task only if a verifier blind to the draft answer can reconstruct it from evidence densely reacquired from the raw video and counterexample checks fail to refute it. This yields SightLine, 310 evidence-traceable tasks from 108 long videos, evaluated through SightLoop, a shared multiscale harness supporting temporal exploration, local inspection, evidence integration, and revisiting while recording returned observations. On disjoint videos, solver trajectories on verified tasks become step-level supervision for observation decisions. The strongest model reaches only 54.52% on SightLine, yet 79.35% when given the annotated evidence. With under 6K examples, Sight-35B nearly doubles its base model's interactive accuracy (21.29% to 42.26%), outperforms the 397B model of its family, begins using explicit spatial inspection after post-training, which its base model never uses, and gains over 8 points on two external long-video benchmarks.

View arXiv page View PDF Add to collection

正文以来源语言显示。

参考来源

← 返回每日AI报告