Mar 10, 2026
Your Workflow is the Eval
A patch can pass tests and still be the wrong change. Workflow outcomes are often a better way to evaluate coding agents.
Findings, methods, and technical notes from building and using Voratiq: evaluation, verification, workflow design, and the broader engineering of agentic systems.
A patch can pass tests and still be the wrong change. Workflow outcomes are often a better way to evaluate coding agents.
Which agent is best? It depends on the task. Selection from a candidate pool turns the prediction problem into an optimization problem.
We ran coding agents in unrestricted mode and logged every sandbox bypass. Most happened incidentally while agents tried to complete tasks.