Principle AI Product Building AI Agents Coding Tools Architecture

Give an optimizing agent full traces, not compressed summaries — it has to see why past attempts failed

Meta-Harness gives its proposer agent a filesystem containing full source code, scores, and execution traces of every prior candidate, enabling up to 10M tokens of diagnostic context per iteration — dramatically outperforming prior methods limited to 26K compressed tokens

Yoonho Lee et al. — Meta-Harness: End-to-End Optimization of Model Harnesses (arXiv:2603.28052) · Apr 6, 2026 · 10 connections

Meta-Harness’s key architectural decision: instead of compressing optimization history into brief summaries or sliding windows, it gives the proposer agent a full filesystem containing source code, scores, and execution traces of every prior candidate. The agent navigates this using standard Unix tools (grep, cat) to diagnose failure modes — the same pattern Agentic search beats RAG for live codebases describes for code navigation. This approach enabled up to 10 million tokens of context per iteration, versus prior methods’ 26K-token maximum.

The results validate the approach: on TerminalBench-2, Meta-Harness achieved 76.4% pass rate on Claude Opus 4.6 (#2 overall) and 37.6% on Haiku 4.5 (#1 overall). The Label-Primed Query harness it discovered beat the prior state-of-the-art (48.6% vs 40.9%) while using 4x fewer tokens. This is Traces are the universal substrate for agent learning — all three layers consume the same execution logs made operational — the filesystem IS the trace archive, and the proposer agent IS the learning loop. It also reinforces Meta-agents that autonomously optimize task agents beat hand-engineered harnesses on production benchmarks: AutoAgent proved meta-agents can optimize harnesses autonomously, and Meta-Harness proves that giving them richer context (full filesystem vs compressed summaries) dramatically improves the result.

AutoAgent found the same thing from the other direction (@kevingu — “AutoAgent: First Open Source Library for Self-Optimizing Agents”, sources/kevingu-autoagent-self-optimizing-agents.md): when its meta-agent received only scores without trajectories, its improvement rate dropped hard. Understanding why something improved matters as much as knowing that it improved — traces give the optimizer interpretability over the task agent’s reasoning, which is what makes targeted edits possible rather than blind grid search. Telemetry that captures only outcomes misses that reasoning layer, the gap Observability is the missing discipline for agent systems — you can't improve what you can't measure names for agent systems generally.

Connected Insights

References (4)

→ Agentic search beats RAG for live codebases → Meta-agents that autonomously optimize task agents beat hand-engineered harnesses on production benchmarks → Traces are the universal substrate for agent learning — all three layers consume the same execution logs → Observability is the missing discipline for agent systems — you can't improve what you can't measure

Referenced by (6)

← Private evals should measure business outcomes that matter — not external benchmarks ← Systems of record keep what was decided, not why — capture the reasoning at the moment a decision becomes binding ← Meta-agents that autonomously optimize task agents beat hand-engineered harnesses on production benchmarks ← Traces are the universal substrate for agent learning — all three layers consume the same execution logs ← Teacher-student trace distillation with consensus validation beats single-oracle learning ← Shadow execution enables safe trace learning — replay write operations without touching production data