An AlphaEvolve evaluation harness must process candidate code updates systematically through a deterministic execution pipeline. Rather than executing and grading a solution all at once, the harness divides the pipeline into three progressive testing gates:
Evaluation gates
The following diagram maps out how candidate programs flow through the three distinct testing tiers before returning structured metadata to AlphaEvolve.
Solution validation
The evaluation loop starts by enforcing hard structural constraints without running the code's core logic. The harness checks the candidate solution for basic compile-time syntax correctness, parsing bugs, and structural security violations.
Success pathway: If the program passes all validation checks, the code safely progresses down to the solution verification runtime loop.
Failure pathway (Short-circuit): If any validation test fails, the harness immediately breaks execution. It bypasses the resource-heavy verification and performance testing tiers completely to protect system infrastructure. The loop drops straight to feedback production, asserting a massive flat search penalty (such as
-1e12) alongside zero-valued performance scores.
Solution verification
Once a candidate program is validated as syntactically sound and safe to execute, the harness launches it within an isolated environment to enforce soft functional constraints. This phase runs your standard unit and functional correctness checks.
Instead of outputting a pass-or-fail binary signal, the harness evaluates the program based on the exact number or percentage of functional tests passed. This counts toward an overall penalty score, providing AlphaEvolve with a dense numerical search gradient to incrementally correct broken logic structures over successive generations.
Solution evaluation
When a candidate program proves to be both syntactically safe and algorithmically correct, it reaches the performance benchmarking layer.
The harness runs empirical tests, direct product logic calculations, or analytical simulations to quantify your target business optimization metrics (such as algorithmic execution speed, cloud infrastructure memory footprint, or output calibration bounds).
Comprehensive evaluation feedback
At the end of the execution pipeline, the harness aggregates data from all completed phases into a structured feedback payload. This payload is returned directly to AlphaEvolve to guide parent selection for subsequent mutations.
The standardized feedback output consists of the following three primary components:
Overall hill-climbing score: A single, combined scalar value calculated deterministically by the harness that AlphaEvolve maximizes directly during optimization.
Granular optimization scores and penalties: The individual sub-metrics, execution times, and soft constraint penalty breakdowns. This explicit data distribution allows the engine to reason clearly about underlying engineering tradeoffs.
Textual insights: Raw structural logs and failure mode details passed back into future prompt contexts, allowing the LLM to learn explicitly from its generation errors.