Hugging Face Papers ↗ · 2026-11-10
ScrapeBench: Evaluating Legal Compliance of AI Agents in Website Scraping
Research Hugging Face Papers Source language: Chinese
论文摘要(作者报告,未经本站复测):AI agents deployed on the open web are becoming more capable of performing actions that may breach contractual provisions and violate computer security laws.
Full text
AIagentsdeployedontheopenwebarebecomingmorecapableofperformingactionsthatmaybreachcontractualprovisionsandviolatecomputersecuritylaws.Inthispaper,weintroduceScrapeBench,thefirstagenticbenchmarkforempiricallyevaluatingthelegalcomplianceofAIagentswithwebsitescrapingpolicies,namelytermsofservice(ToS)androbots.txtdirectives.Acrosswidelyusedclosed-weightandopen-weightAIagents,wefindthatmostagentsexhibitlowcompliancerates(<5%)wheninstructedtoscrapewebsitesinoursampleof1,626popularwebsitesthatexplicitlyprohibitscraping.Inaddition,weobservethatevenwhereAIagentsactivelychecktheapplicableToSandrobots.txt,theycontinuetoexhibitlowcompliancerates.Conversely,inoursampleof269websitesthatdonotprohibitscraping,weobservethatagentssometimesrefusetoscrapedespiteitbeinglawful,suchasClaudeOpus4.6exhibitinganover-refusalrateof98.5%.Ourhumanbaselineexperiments(n=180participants)suggestthathumansexhibitcomplianceratescomparabletoAIagentsbutover-refuseatmuchhigherrates.
View arXiv pageView PDF Project page GitHub 0 Add to collection