ScrapeBench: Evaluating Legal Compliance of AI Agents in Website Scraping

Research Hugging Face Papers Source language: Chinese

论文摘要(作者报告,未经本站复测):AI agents deployed on the open web are becoming more capable of performing actions that may breach contractual provisions and violate computer security laws.

Full text

AIagentsdeployedontheopenwebarebecomingmorecapableofperformingactionsthatmaybreachcontractualprovisionsandviolatecomputersecuritylaws.Inthispaper,weintroduceScrapeBench,thefirstagenticbenchmarkforempiricallyevaluatingthelegalcomplianceofAIagentswithwebsitescrapingpolicies,namelytermsofservice(ToS)androbots.txtdirectives.Acrosswidelyusedclosed-weightandopen-weightAIagents,wefindthatmostagentsexhibitlowcompliancerates(<5%)wheninstructedtoscrapewebsitesinoursampleof1,626popularwebsitesthatexplicitlyprohibitscraping.Inaddition,weobservethatevenwhereAIagentsactivelychecktheapplicableToSandrobots.txt,theycontinuetoexhibitlowcompliancerates.Conversely,inoursampleof269websitesthatdonotprohibitscraping,weobservethatagentssometimesrefusetoscrapedespiteitbeinglawful,suchasClaudeOpus4.6exhibitinganover-refusalrateof98.5%.Ourhumanbaselineexperiments(n=180participants)suggestthathumansexhibitcomplianceratescomparabletoAIagentsbutover-refuseatmuchhigherrates.

View arXiv pageView PDF Project page GitHub 0 Add to collection

Sources

← Back to the daily brief