Hugging Face Papers ↗ · 2026-11-18
Phantom Success, Silent Failure: Auditing the Success Checkers of Simulated Manipulation Benchmarks
Media reports Hugging Face Papers Source language: Chinese
An LLM‑based pipeline recreates disagreement scenarios and visually verifies task compliance, uncovering phantom successes and silent failures in simulated manipulation benchmarks, highlighting reliability issues in success checkers.
Full text
An LLM-based pipeline audits simulated manipulation success checkers by recreating disagreement scenarios and visually verifying whether observed behavior satisfies task instructions, revealing phantom successes and silent failures.