🔴 Required Information
Describe the Bug:
When using the optimizer with conversation_scenario (user simulation) eval cases, local_eval_sampler.py crashes with TypeError at line 292 because eval_metric_result.score can be None for some per-invocation results.
The initial baseline evaluation completes successfully, but during the GEPA reflection loop's internal re-evaluation, some per-invocation metric results return None scores. The code calls round(eval_metric_result.score, 2) without a None guard.
Steps to Reproduce:
- Create an eval set using
conversation_scenario (user simulation)
- Run the optimizer (e.g.
AgentOptimizer.optimize())
- The first baseline eval passes, but iteration 1's re-evaluation crashes
Expected Behavior:
The optimizer should handle None scores gracefully (e.g. treat as 0.0 or skip) and continue the GEPA loop.
Observed Behavior:
File ".../google/adk/optimization/local_eval_sampler.py", line 292, in _format_eval_result
"score": round(eval_metric_result.score, 2), # accurate enough
TypeError: type NoneType doesn't define __round__ method
Environment Details:
- ADK Library Version: 1.28.0
- Desktop OS: macOS
- Python Version: 3.13
Model Information:
- Are you using LiteLLM: No
🟡 Optional Information
Root Cause Analysis:
local_eval_sampler.py:292:
eval_metric_results.append({
"metric_name": eval_metric_result.metric_name,
"score": round(eval_metric_result.score, 2), # ← crashes when score is None
"eval_status": eval_metric_result.eval_status.name,
})
Suggested Fix:
"score": round(eval_metric_result.score, 2) if eval_metric_result.score is not None else 0.0,
Related Issues:
How often has this issue occurred?:
Always (100%) when using conversation_scenario with the optimizer.
🔴 Required Information
Describe the Bug:
When using the optimizer with
conversation_scenario(user simulation) eval cases,local_eval_sampler.pycrashes withTypeErrorat line 292 becauseeval_metric_result.scorecan beNonefor some per-invocation results.The initial baseline evaluation completes successfully, but during the GEPA reflection loop's internal re-evaluation, some per-invocation metric results return
Nonescores. The code callsround(eval_metric_result.score, 2)without aNoneguard.Steps to Reproduce:
conversation_scenario(user simulation)AgentOptimizer.optimize())Expected Behavior:
The optimizer should handle
Nonescores gracefully (e.g. treat as 0.0 or skip) and continue the GEPA loop.Observed Behavior:
Environment Details:
Model Information:
🟡 Optional Information
Root Cause Analysis:
local_eval_sampler.py:292:Suggested Fix:
Related Issues:
_EvalMetricResultWithInvocationfails withconversation_scenario(user simulation) - expected_invocation is None #4283 — same root cause (conversation_scenarioproducingNonevalues) but manifesting inagent_evaluator.pyHow often has this issue occurred?:
Always (100%) when using
conversation_scenariowith the optimizer.