Themen / AI evaluation and generalization

Zugeordnete Aussage

Evolved harnesses generalize beyond their original benchmarks

A harness evolved on Terminal-Bench-2 transferred without further evolution to SWE-bench-verified, indicating it encodes general engineering experience in harness components rather than benchmark-specific optimization.

Hinter der geäußerten Meinung

Übersetzungen dienen dem Leseverständnis; die Originalauszüge bleiben die Quellenevidence.

Harness Engineering for Self-Improvement

Originalauszug

On Terminal-Bench-2, AHE achieved better than human-designed harness (OpenCode, Terminus-2, Codex) except for Hard tier and a few other self-evolve baselines (ACE, TF-GRPO). The same frozen harness, without further evolving, transfers to SWE-bench-verified, indicating that the evolved harness is able to encode engineering experience into harness components rather than doing benchmark-specific optimization.