The Nerva codebase should not force every task through the same model. A local model may be the right choice for private notes or a small code change. A remote provider may be worth using for a difficult reasoning task, a larger context window, or a model that is not practical to run on the user's laptop.

Nerva is being designed around that choice. The planned router can select a local model or a connected provider API, but the router must never become a hidden pipeline that sends private context away without the user's knowledge.

This article describes the planned architecture. Names such as Luna, Opus, Sonnet, Sol, or Astra are examples of models a user might connect when the relevant provider supports them. They are not a promise that every model is available, free, or supported at launch.

The routing decision in plain English

Before choosing a model, Nerva should inspect four things:

1. Privacy: can the task stay local, or does it contain a path, secret, client file, or private repository? 2. Capability: does the task need a larger context window, vision, code reasoning, or another specialized ability? 3. Cost and limits: is the connected API affordable and within its quota for this task? 4. Reliability: can the task continue if a provider is unavailable or returns an incomplete answer?

The best model is not always the largest model. It is the smallest model that can complete the task within the user's policy and quality requirements.

A safer routing shape

The model router should sit behind a context and permission boundary:

task request
    |
    v
context classifier -> sensitive? -> local-only or ask permission
    |
    v
capability check -> choose local model or approved provider
    |
    v
request adapter -> model call -> structured result
    |
    v
tool permission -> execution -> verification

The router does not get to decide that a private file is safe to upload. It can describe which capability is missing locally, but the user or a saved policy decides whether a cloud request is allowed.

Local models should be the first option

Local inference is useful when the task is small enough for the device and the user values privacy more than maximum capability. Examples include:

  • Explaining a function from a local repository.
  • Renaming a variable across a small set of files.
  • Turning personal notes into a checklist.
  • Classifying files before a larger workflow.
  • Drafting a first answer from context that should not leave the device.

    • The local path still needs guardrails. A local model can misunderstand a request, edit the wrong file, or follow a malicious instruction found inside a repository. Local does not mean automatically trustworthy. The same permission, sandbox, diff, and verification rules still apply.

    When a provider API is useful

    An external provider can be valuable when the user explicitly chooses it for a reason:

    | Need | Possible reason to use a provider API | | --- | --- | | Difficult reasoning | The connected model is stronger for the task | | Large context | The local model or device cannot fit the material | | Specialized input | The provider supports vision, audio, or another capability | | Speed | Remote inference is faster than the local device | | Experimentation | The user wants to compare two model responses |

    The UI should show the model, provider, estimated context, and any known cost or quota implication before sending. A provider switch should feel like a deliberate decision, not a side effect of clicking “try again.”

    Do not make provider adapters leak into the agent core

    Every provider has different authentication, request formats, streaming behavior, error codes, and tool-calling conventions. Nerva should hide those differences behind a narrow adapter interface:

    type ModelRequest = {
  messages: Array<{ role: 'user' | 'assistant' | 'tool'; content: string }>;
  tools?: ToolDescription[];
  signal?: AbortSignal;
};

type ModelAdapter = {
  id: string;
  capabilities: Set<'text' | 'vision' | 'tools' | 'streaming'>;
  estimate(request: ModelRequest): Promise<{ inputTokens?: number; cost?: number }>;
  complete(request: ModelRequest): Promise<ModelResponse>;
};

    The exact interface will change during implementation. The principle should remain: the task manager speaks to a stable contract, while each provider adapter handles its own API details.

    That separation makes it possible to compare providers without rewriting the permission engine, tool gateway, or audit log. It also makes failure handling easier to test.

    The context firewall matters more than the model list

    Before a cloud request, Nerva should build a payload preview. It should identify files, excerpts, tool results, and environment values that would leave the device. Secrets should be blocked or redacted before the adapter receives the request.

    The firewall should also treat repository instructions, web pages, downloaded documents, and terminal output as untrusted data. A file can contain text that looks like an instruction. That text is context to analyze, not permission to execute.

    For a code task, a useful preview could say:

    Provider: connected API
Model: selected by user
Files included: 4
Secrets detected: 0
Tools requested: read-only search
Estimated input: 18,000 tokens
Approval: required before sending

    The product does not need to expose every internal token. It does need to give the user enough information to make an informed decision.

    Failover must not create duplicate actions

    A provider timeout is not proof that the provider did nothing. If Nerva retries a request that included a tool call, it could create a duplicate side effect unless the agent separates planning from execution and assigns an idempotency identity to the action.

    The safer sequence is:

    1. Ask the model for a structured plan. 2. Show the requested tool actions. 3. Obtain permission for the specific actions. 4. Execute each action through the tool gateway. 5. Record the result and verify it.

    If a model call fails after returning a partial plan, Nerva can retry the planning step without automatically repeating the tool action. That is the same retry boundary that matters in payment workers and queues; idempotency in Node.js workers explains the backend version of the problem.

    What “unlimited” means in a routed system

    Nerva can aim to make the user's workflow unlimited in the sense that the app does not stop after a small number of prompts. The routing system still needs honest limits:

  • Local models are limited by device memory, compute, thermals, and battery.
  • Provider APIs may have quotas, rate limits, context limits, and usage costs.
  • A task may be stopped by a user policy or permission boundary.
  • Repeated retries can waste time and money, so they need budgets.

    • Visible limits are better than pretending none exist. A good agent lets a user set a local-only policy, a provider budget, a maximum retry count, and an approval requirement for tool actions.

    The useful promise

    The strongest Nerva promise is not “every model in one app.” It is this: the user chooses where reasoning happens, what context leaves the device, which tools can run, and how the result is verified.

    That creates room for local models, connected providers, and future model families without making the security boundary provider-specific. The model can change. The control spine should not.

    Nerva is still an upcoming project, so the adapter list, supported platforms, and final performance need implementation proof. The architecture is useful precisely because it makes those future claims testable instead of hiding them behind a model picker.