Most AI developers treat their production calls as exhaust — logs that pile up, get ignored, and eventually get rotated out. We think that’s a massive missed opportunity.
Every API call your Claude Code session makes contains a real prompt, from a real workflow, solving a real problem. That’s not exhaust. That’s a test suite. And with the new Neurometric skill for Claude Code, we’re using it as one.
Logging Is the Floor, Not the Ceiling
There are plenty of tools that will capture your AI calls. Neurometric does that too — routing every request from your Claude Code sessions through our gateway at api.neurometric.ai, where prompts, completions, token counts, model metadata, and latency are all recorded automatically, with zero changes to your code.
But logging is just where we start. What happens next is the part that matters.
Model and Test Time Compute Evaluation
As Neurometric captures each call, it runs that same prompt against different combinations of models and thinking algorithms. Not in theory — against your actual prompts, from your actual sessions, in real time.
This is the distinction we care about: it’s not observability, it’s evaluation. The question we’re answering isn’t “what did your agent do?” It’s “did it use the right model, with the right reasoning approach, to get the best result?” If you are optimizing for latency or cost, the only real way to evaluate your prompts is to test them against the jagged frontier of system performance.
Test time compute — the idea that you can get dramatically better outputs by giving a model more time to reason, rather than simply swapping in a larger model — is one of the most important levers in modern AI. But most developers never touch it, because there’s no easy way to run systematic comparisons across model and compute configurations against real workloads. Neurometric makes that comparison automatic.
When your Claude Code session makes a call, we capture it and evaluate it. You find out not just what happened, but what could have happened with a different model or a different reasoning strategy.
Installation in Three Steps
Setup is minimal by design. Clone the plugin, export your API key, and launch Claude Code with the plugin flag:
bash
git clone https://github.com/neurometricai/neurometric-plugin ~/.claude/plugins/neurometric
export NEUROMETRIC_API_KEY="sk_live_your-api-key"
claude --plugin-dir ~/.claude/plugins/neurometricNo configuration files. No code changes. The gateway handles everything from there.
Two Commands Built In
The skill ships with two slash commands available from any Claude Code session.
/neurometric-status verifies that the gateway is reachable, your API key is valid, and the environment is configured correctly — useful at install and whenever something feels off.
/neurometric-replay fetches your most recent captured calls directly in the terminal. Pass an optional count to control how many to display. It turns your terminal into a live window into your evaluation history.
More coming soon as we add them to the API. For now, to get model recommendations you have to login to the app.
Why This Changes the Development Loop
The standard Claude Code workflow is fast. Write, run, iterate. But there’s a hidden assumption baked in: that the model you’re calling is the right one for the task, with the right reasoning configuration. Most of the time, that assumption is never tested.
Neurometric closes that gap. Every Claude Code session becomes an ongoing evaluation — not just a record of what your agent did, but a continuously updated answer to a harder question: is this the best your AI can do?
Get started at neurometric.ai/skill.
This reframes the entire AI development paradigm: moving from "what did my agent do?" to "what could my agent do?" By turning every production call into a live, multi-model test, Neurometric transforms deployment from a static endpoint into a dynamic optimization loop, the difference between logging history and engineering the future.