AI makes usage less predictable
Traditional SaaS workloads are often easier to connect to a user action or transaction. AI features can introduce variable token use, background tasks, retrieval, tool calls, and repeated evaluation. Cost and capacity therefore need to be visible at the tenant, feature, workflow, and model level.
Usage metering should be designed as a platform capability early enough to inform pricing, limits, routing, and product decisions.
Tenant isolation includes context
Tenant boundaries must cover more than database rows. They also apply to vector indexes, prompts, cached context, evaluation datasets, model credentials, tool access, logs, and support workflows.
Permission aware retrieval and traceable source use are especially important when an assistant can synthesize information across many systems.
Model flexibility is an architectural concern
Model quality, cost, latency, availability, and deployment requirements change. A product should avoid spreading assumptions tied to a specific provider throughout the application.
A deliberate model and tool abstraction can support routing and replacement, but it should remain narrow enough to preserve useful provider capabilities rather than reducing every model to the lowest common denominator.
Evaluation belongs beside delivery
AI product releases need evidence about behavior, not only code correctness. Evaluation datasets, expected behaviors, safety tests, and production feedback should move through the delivery process with the application.
This creates a more useful definition of done: the feature is released, its behavior is measured, and the team knows how it will be monitored.
What evidence would your team need to operate this capability with confidence?