The demo proves possibility, not operability
An agent demo usually answers one narrow question: can a model complete the intended task under favorable conditions? Production introduces a different set of questions. What happens when context is missing, a tool fails, permissions differ, an instruction is ambiguous, or the model’s cost and latency change?
The architecture must make those conditions visible and manageable. That means treating the agent as a distributed software system with uncertain decisions rather than as a single API call.
Separate reasoning from authority
A model can propose a plan without receiving unrestricted authority to execute it. Production systems should distinguish read actions, reversible actions, consequential actions, and actions that require human confirmation.
Tool contracts, identity, permissions, data boundaries, and approval policies should be explicit. The user experience must also show what the system intends to do before an irreversible step.
Evaluate the workflow, not only the answer
Answer quality benchmarks are useful, but an operational agent should also be evaluated for tool choice, task completion, escalation behavior, source use, latency, and cost. Tests need representative workflow states, including failure and ambiguity.
A strong evaluation system combines offline test suites, release simulations, production signals, and review of sampled interactions. The purpose is continuous evidence rather than a single score.
Design for recovery
Tools time out, APIs change, and business rules evolve. Agent workflows need idempotency, checkpoints, retries, compensation paths, and a clear way for a person to resume or correct work.
When recovery is part of the original design, the system becomes easier to trust and less expensive to operate.
What evidence would your team need to operate this capability with confidence?