IDEAS CONECTADAS
Atlas de conocimiento
Explora personas, perspectivas y sus fuentes originales.
1 personas · 1 fuentes · 1 opiniones expresadas
EN CONTEXTO
Human–AI performance tradeoffs in RE-Bench
Elige una perspectiva y vuelve a la conversación original.
AI agents outperform humans at short time budgets but not longer ones
Los sectores iguales orientan la lectura, no indican una clasificación.
Perspectivas 1–1 de 1 · Fuentes más recientes primero
Perspectiva seleccionada
AI agents outperform humans at short time budgets but not longer ones
In RE-Bench, the best AI agents scored 4× higher than human experts at a 2-hour budget, yet humans demonstrated better returns to extended effort and surpassed agents at both 8-hour and 32-hour time budgets.
Estas son perspectivas individuales, no una medida de consenso. El material fuente permanece en su idioma original.
Evidencia a favor
Extracto original
Best AI agents scored 4× higher than humans at a 2-hour budget, but humans had better returns to longer budgets and exceeded agents at 8-hour and 32-hour settings.
Contexto
RE-Bench : evaluate frontier AI agents on realistic ML research-engineering envs against human experts. 7 challenging, open-ended ML research-engineering environments. Each environment = (scoring function, starting solution, reference solution); each can be run with 8 or fewer H100 GPUs. Examples: optimize a kernel, run a scaling-law experiment, fix an embedding, fine-tune GPT-2 for QA, etc. Includes data from 71 eight-hour attempts by 61 distinct human experts. Human experts achieved non-zero score in 82% of 8-hour attempts; 24% matched or exceeded strong reference solutions.
Las fechas corresponden a las fuentes, no a cambios de opinión. Los textos sin traducción revisada se mantienen en su idioma original.