Running a local model is not a contest to find the largest file that a laptop can barely load. A useful Nerva setup should leave enough memory for the operating system, editor, repository tools, and the task itself. The right model is the one that gives acceptable answers at a sustainable speed.

Start with the task, not the model leaderboard

A small local model may be enough to summarize a function, classify files, or draft a test outline. A harder architecture review may need a larger model or an approved cloud route. Decide the quality requirement first, then choose the smallest model likely to meet it.

Useful checks include:

  • Available memory while the editor and tools are open.
  • Context length needed by the task.
  • Tokens per second for the expected workflow.
  • Whether the model follows structured output reliably.
  • Whether a local answer can be verified with tests or a diff.

Quantization is a tradeoff

GGUF quantization can make a model practical on consumer hardware by reducing memory use. It may also change output quality. The practical question is not whether a model is “lossless,” but whether it remains accurate enough for the task and whether the result is checked before use.

Nerva should show the selected model, context size, and local route clearly. A user should be able to stop a slow inference or switch routes without losing the task history.

For private code, local inference can be an excellent default. It is still not a substitute for permissions: a local model can propose the wrong edit or follow unsafe instructions found in a repository. Local model selection and action control need to work together.

The useful design question

The useful question here is not whether an AI agent can produce an impressive demo. It is whether the workflow stays understandable when the task is incomplete, the repository is messy, or the model is wrong. For this part of Nerva, the focus is choosing a model around the laptop and the task. That means each state should have a visible owner, a small input, and a result that another part of the system can inspect.

A concrete scenario

Consider testing a quantized GGUF model on a normal development machine before judging it by a leaderboard. A good Nerva run would first state the scope, identify the files or operation it needs, and show the next proposed step. It would keep the original request separate from model-generated instructions. If an action needs approval, the user would see what will happen before it happens. Afterward, the task history would retain the command, result, and verification status instead of reducing everything to a confident paragraph.

This approach also makes a failure useful. A denied permission, unavailable provider, missing package, or changed file is information about the task state. Nerva should preserve it and explain the smallest recovery option. It should not silently broaden the request, retry a side effect, or claim completion because a model supplied plausible-looking text.

What I would test first

For an early implementation, I would test three paths: a normal successful run, a denied action, and an interrupted run resumed after the workspace changes. The key test is whether speed, context size, or reasoning quality is the real bottleneck. In each path, the interface should make clear what Nerva knows, what it proposed, what actually ran, and what remains unverified.

Useful acceptance checks include:

  • the user can inspect the selected context before model execution;
  • sensitive paths and credentials are excluded or redacted;
  • every side effect has a narrow permission decision;
  • failures remain attached to the step that caused them;
  • a final answer links to evidence instead of implying that unrun work succeeded.

Nerva is still an upcoming project, so these are design and implementation goals rather than claims that a finished product already provides them. That distinction matters: a trustworthy agent should earn confidence through observable behavior, not through a feature list.

A small review checklist

A practical review should end with a small decision, not a vague promise. Write down the scope, the expected result, the permission required, and the check that will confirm it. Then try the same flow with a refusal or missing dependency. If the interface explains that case clearly, the design is ready for a more realistic prototype; if it hides the problem, adding another feature will only make the workflow harder to trust.

The design is informed by the boundaries I have had to think through in CryptoEx and SentinelFi: backend work becomes easier to review when context, permissions, side effects, and verification are explicit.