Agent demos usually emphasize what the model decides. Reliable agent systems are shaped just as much by what the surrounding software refuses, records, retries, and hands back to a person.

The most valuable engineering is often deliberately ordinary: typed inputs, narrow tools, explicit state, authorization checks, timeouts, idempotency, and useful logs.

Make tools small and specific

A tool should represent one bounded capability with a clear input and a predictable result. Avoid giving a model a large internal API surface and hoping a detailed prompt will enforce the right sequence.

Small tools make permissions easier to reason about. They also make it possible to validate arguments, test failure behavior, and attach useful telemetry to each action.

Treat state as product data

If a workflow can pause, retry, branch, or require approval, its state should not live only in the model context. Persist the decisions and identifiers the product needs to resume safely.

Useful state includes the current step, tool results, approval status, retry count, relevant resource versions, and the reason a workflow stopped.

Validate at every boundary

Structured model output is an input to your application, not a trusted internal object. Validate it before it reaches a tool. Validate tool results before they become context for another model call.

Business rules and authorization remain deterministic application concerns. A model can suggest an action; it should not redefine who is allowed to perform it.

Design the recovery path first

Ask what happens when a tool times out, returns partial data, or succeeds after the client has retried. Decide which operations are safe to repeat and which require a stable idempotency key.

Some failures should retry automatically. Others should stop with a concise explanation and enough state for a person to continue.

An external side effect needs a stable operation identity. If a tool call times out after the server commits, “retry once” can create a duplicate charge, message, or job. The workflow should send an idempotency key where the downstream API supports one and persist the key with the step state.

The state machine should distinguish at least:

not_started -> running -> succeeded
                    \-> failed_retryable
                    \-> failed_terminal
                    \-> awaiting_approval

“The model will remember what happened” is not a recovery protocol.

Observe decisions, not private thoughts

You rarely need to retain unrestricted prompts and responses forever. Capture the operational facts required to understand the workflow: tool chosen, validated arguments, result status, latency, model version, policy decision, and final outcome.

This creates evidence for evaluation and debugging without turning every trace into a new data-liability problem.

Reliability is a design choice

An effective agent does not need to look maximally autonomous. It needs to complete a valuable task with clear limits and recover predictably when the world is messy. That outcome comes from good product and systems engineering around the model.

References