nafyi.
Thèmes / MLE-bench offline Kaggle benchmark

Point de vue reformulé

MLE-bench uses 75 Kaggle competitions as offline ML engineering benchmarks

MLE-bench evaluates ML engineering agents on 75 curated offline Kaggle competitions, testing model training, dataset preparation, experiment execution, and submission to grading scripts. Kaggle public leaderboards serve as human baselines. The best-performing setup—o1-preview with AIDE scaffolding—reached at least Kaggle bronze-medal level in 16.9% of competitions.

Lilian Weng·Publié

Derrière ce point de vue

Les traductions sont destinées à la lecture ; les extraits originaux restent la source evidence.

Harness Engineering for Self-Improvement

Extrait original

MLE-bench : evaluate ML engineering agents on offline Kaggle competitions. Contains 75 ML-engineering competitions curated from Kaggle.
Contexte

Tests training models, preparing datasets, running experiments, and submitting predictions to grading scripts. Uses Kaggle public leaderboards as human baselines. Best setup in the paper, o1-preview with AIDE scaffolding, reached at least Kaggle bronze-medal level in 16.9% of competitions. Includes resource-scaling and contamination analyses.

Ouvrir l’original · Paragraphe 142 (s’ouvre dans un nouvel onglet)Voir la page source