Evaluate coding agents with confidence: Tests, trajectories & SambaEval

Evaluating coding agents with tests as ground truth

Coding agents have a luxury most agents don’t: ground truth. The tests pass or they don’t. In this session, Kwasi Ankomah builds a full evaluation harness around that signal and uses it to choose the best SambaNova model for the job.

About this webinar

Most agents are hard to evaluate because “correct” is subjective. Coding agents are different — a test suite gives a clear, objective answer. That makes them the perfect place to build a rigorous evaluation practice.

In Part 3, the agent gained a real environment and learned to ship a PR that passes CI. In Part 4, Kwasi Ankomah measures how well it actually performs. He builds the evaluation harness end to end — pass@k over a task set, trajectory evaluation, and full tracing — then uses SambaEval to compare SambaNova models and pick the best one for the executor role. Evaluation is the bridge to improvement, and this session sets up everything the rest of the series builds on.

What you will learn

• Why tests make coding agents uniquely measurable, and how to use them as ground truth
• How to measure pass@k across a task set to get a reliable view of agent performance
• How to run trajectory evaluation to check whether the agent took a sensible path, not just whether it got there
• How to add full tracing so every run can be inspected and debugged
• How to use SambaEval to select the best SambaNova model for the executor role

Why coding agent evaluation matters

Agents are non-deterministic — the same task can succeed once and fail the next time. A single passing demo says very little. Measuring pass@k across many tasks shows how reliable the agent really is, while trajectory evaluation catches agents that reach the right answer for the wrong reasons. Without this foundation, there’s no way to tell whether a prompt change, model swap, or new tool actually made things better.

Who should attend

AI and ML engineers building or shipping coding agents, data scientists responsible for evaluating LLM systems, and technical leads choosing models and evaluation tooling for production agents. Familiarity with earlier parts of the series is helpful but not required.

Kwasi Ankomah is the Lead AI Architect at SambaNova Systems, where he builds production agentic AI systems on purpose-built RDU inference infrastructure. With 15 years of experience spanning financial services, consulting, government, and tech startups, Kwasi bridges deep technical expertise with a hands-on understanding of real business challenges. He specializes in deep agent architectures, multi-agent orchestration, and context engineering. Connect with Kwasi on LinkedIn →