An AI agent is not a prompt plus a loop. It is a distributed workflow that receives uncertain input, chooses actions, touches tools, and must explain what happened. The architecture becomes trustworthy when each boundary has an owner.

The short answer

Use separate components for planning, policy, tool execution, durable state, and verification. Let the model propose an action. Let application code validate it. Let a permission layer approve or reject it. Let a tool adapter perform one bounded operation. Then observe the result and verify the expected state change.

A useful control loop

request -> context -> proposal -> policy -> action -> observation -> verification

The arrows are responsibilities, not marketing labels. Context selection should not secretly execute tools. Policy should not be bypassed because a model returned valid JSON. Observation should record what actually happened, including a partial failure. Verification should compare the result with an explicit expectation.

Tools need contracts

A tool should declare its name, input schema, authorization requirement, side effects, timeout, retry behavior, and result shape. “Run command” is too broad for a safe interface. Prefer a tool such as run_tests with an allowed working directory and a time limit.

For a file-editing tool, validate the path, show the diff, reject paths outside the workspace, and create a checkpoint before writing. For a payment tool, use an idempotency key, store the provider response, and require a stronger approval level than a read-only lookup.

Durable state beats chat history

The transcript is not a reliable job database. Store task status, selected context, tool attempts, approvals, results, and verification outcomes in durable state. A worker restart should not make the agent forget whether an external request may already have succeeded.

This is the same reasoning used in reliable backend queues: the system needs a state machine, not just a sequence of messages. Store enough information to resume safely, but redact secrets and avoid copying entire private documents into every event.

Memory is a policy decision

Agent memory can mean files, a database, embeddings, or task history. Each option has different deletion, access, and freshness behavior. A vector search result is not automatically true, and a saved task note may outlive the permission that allowed the original request.

Define what can be remembered, who can read it, how it expires, and how a user deletes it. Retrieval should return source references so the model can distinguish a current record from an old suggestion.

Verification must be independent enough

Asking the same model whether its own result is correct is useful but insufficient for high-impact actions. Verify with a database constraint, a test command, a second system, a checksum, or a human approval depending on the risk.

For a code change, run the relevant tests and inspect the diff. For a data update, query the resulting record and compare the expected fields. For an email, show the final recipient and content before send. Verification should produce evidence a reviewer can understand.

Where agents fail

Agents fail at boundaries: ambiguous instructions, stale context, malformed tool arguments, permission confusion, timeouts, duplicate retries, and partial completion. Design each failure as a named state. “Unknown” is often safer than “failed” when a provider may have accepted the request.

Start with one tool, one approval rule, and one recovery path. Expand only after the logs and tests explain the current system. Continue with handling partial failure in multi-step agent plans.

A small implementation boundary

In a Node.js service, keep the model adapter separate from the tool registry. The adapter can translate provider-specific tool-call formats into one internal command. The registry can validate the command against a schema and look up the permission required for that tool. The worker can then execute the approved operation and write an event containing the outcome.

That separation makes testing more realistic. You can feed a malformed tool call to the validator without contacting a provider. You can simulate a timeout at the worker boundary without pretending the database is unavailable. You can replay an observation and check that verification produces the same decision. These are ordinary backend tests applied to an uncertain input source.

Do not add a second agent merely because the first one is unreliable. Add another component only when it owns a distinct responsibility, such as retrieval, policy, or verification. More agents multiply context transfer, retries, and debugging paths. A single well-bounded workflow is usually the better first release.

When the system grows, keep ownership obvious. The planner owns a proposal, the policy service owns permission, the worker owns execution, and the verifier owns the claim that the result meets the task contract. This prevents a convenient model callback from becoming an unreviewed path around authorization. It also gives a new engineer a map of where to add a test when a failure is discovered.

The architecture should make the safe path the easy path. A tool call with an unknown name should fail closed. A missing approval should produce a waiting state. A result that cannot be verified should remain visible as incomplete. These defaults reduce the chance that a timeout or provider change turns into an invisible side effect.

They also make incident review faster because every decision has a clear owner.

That clarity matters when several systems fail together.