Why we no longer evaluate SWE-bench Verified

公司动态 OpenAI 原文语言:英文

SWE-bench Verified is increasingly contaminated and mismeasures frontier coding progress. Our analysis shows flawed tests and training leakage. We recommend SWE-bench Pro.

原文摘录

SWE-bench Verified is increasingly contaminated and mismeasures frontier coding progress. Our analysis shows flawed tests and training leakage. We recommend SWE-bench Pro.

以下是来源提供的简短摘录,完整内容请查看原文。

参考来源

← 返回每日简报