A demo proves it can work. Evaluation proves it does — and keeps doing it.

Agents are non-deterministic, act in many steps, and reach the same outcome by good and bad routes alike — which breaks almost every assumption behind conventional software testing. The same input does not give the same output; a correct answer can be produced through a forbidden path; a one-in-twenty failure is invisible in a demo and certain at scale; and the most common metric, whether it looked right, is the one that tells you least. This guide covers what to measure and what not to, the levels of testing from a single tool contract to a live release, how to build fixtures the business believes, how to grade open-ended work without fooling yourself, trajectory scoring, simulated users and injected faults, adversarial testing, the statistics you cannot skip, release gates, production monitoring, human review, independent validation, and how evaluation itself fails. Five lenses per concept: what it requires, how to architect it for a bank, a flow diagram, a plain-English reading, and what it hands off next.