The Product SystemThe Vault
Build Your Own Agent Harness: 4 Ideas Worth Stealing From NVIDIA's SoL-Pi Paper, the infographic in this PDF

AI

Build Your Own Agent Harness: 4 Ideas Worth Stealing From NVIDIA's SoL-Pi Paper

Four token-saving mechanisms from NVIDIA's SoL-Pi paper, the one I built in first, and why owning your agent harness lets you test new research the same day.

Want this as a PDF?

You'll also get The Product System, my weekly note on leading product in the AI era. Unsubscribe anytime, and I'll never sell your email. How I handle your details

1

The full infographic, as a PDF

The exact one from the post, sized to print or keep within reach.

2

A copy in your inbox

The download link lands in your email too, so it's there when you need it. No follow-up campaign.

3

The Product System, weekly

My note on leading product in the AI era. One idea a week that compounds. Unsubscribe anytime.

Build your own harness, people.

A new NVIDIA paper, SoL-Pi, reports cutting the tokens an agent harness uses by 44.7% to 49%. It sits on top of the harness you already have. It cuts your costs, and nothing else changes.

A harness is the layer around an AI model that runs the work. It decides what goes into the model's context, which tools get called, what comes back and what gets checked. If you build or manage anything that runs agents, the harness is where your token bill gets decided.

The four mechanisms in this paper are the part that matters. They did not come from a whiteboard. The researchers ran automated loops and tested variant after variant. Four survived. Here is what each one does, which one I took, and why the paper is really an argument for owning your harness.

Action Fusion: one round trip instead of two

The usual pattern is two steps. The agent makes an edit, then makes a second call to check the edit worked.

Action Fusion joins them. The edit carries its own check, and both run in the same call. One round trip instead of two.

That sounds small until you count how often an agent edits something in a long run. Every call carries context with it, so halving the calls for the most common step adds up quickly.

ObservationPack: send the big result in full once

Tools return big results. A log file, a long search result, a page of test output.

In a normal harness that result could travel full size twice. ObservationPack sends it full size once. After that it shrinks to a reference and a short excerpt.

The model still knows the result exists and can point back to it. It just stops paying to read the whole thing again.

Online Context Compact: compress only when it pays

Context windows fill up, and a common fix is to compress them on a schedule.

Online Context Compact adds a condition. The context window compacts itself, but only when doing so actually saves tokens. Compressing costs something too, so compressing when it saves nothing is just another expense.

It is a decision rule, not a habit. Act when the saving is real, skip it when it isn't.

Evidence-Preserving Reducer: a summary you can check

This is the one I took.

A cheap model compresses a long log into a summary. The summary is accepted only if every quote in it can be found in the original. If a quote can't be found, the summary is rejected.

The risk with any summary is that it sounds right and isn't. This mechanism doesn't ask you to trust the cheap model. It checks the cheap model's work in a way that can't be argued with: the quote is either in the original or it isn't.

Worst case, you lose the shortcut. You never lose the evidence.

That is why it was the first one I built in. Its worst case is cheap, and it never trades accuracy for savings.

Take the exact figures with a grain of salt

Those savings, 44.7% to 49% fewer tokens, are their numbers on their own benchmark. The paper also reports API cost falling by about a third. Treat both as a direction, not a promise for your setup.

The code is MIT licensed and on GitHub under NVlabs if you want to dig in.

Why this is the argument for building your own harness

I run my harness on 2x Claude Max, 4x ChatGPT, Gemini and OpenRouter subscriptions across 3 Macs. Token costs add up fast.

I assessed all four mechanisms the morning I read the paper, and I had already built one of them in by the time I wrote about it.

That speed is the whole point. Without a harness of your own, a paper like this is something you read and admire. You are left guessing whether any of it applies to you.

With one, you can steal from it. You run the idea against your own setup and the answer comes back the same morning.

The next paper drops, and you ask your harness what is useful. You build what fits and leave the rest. The question you ask is the one you would ask of any roadmap item: does this fit how we work, what does it cost, and what happens if it fails?

Your harness gets better with every paper or repo you steal from. A harness you rent can only improve when someone else decides it should.

Do you have a harness? If not, what is stopping you?

Questions people ask

What is an agent harness?
An agent harness is the software layer around an AI model that runs agent work. It controls what goes into the model's context, which tools it can call, how results come back and how work gets checked. The harness has a large effect on how many tokens an agent uses, and so on what it costs to run.
What is SoL-Pi?
SoL-Pi is a research paper from NVIDIA describing a layer that sits on top of an existing agent harness to cut token use. The researchers found it by running automated loops that tested many variants, and four mechanisms survived. The code is MIT licensed and published on GitHub under NVlabs.
What are the four mechanisms in SoL-Pi?
Action Fusion runs an edit and its check in the same call. ObservationPack sends a big tool result in full once, then replaces it with a reference and a short excerpt. Online Context Compact compresses the context window only when that saves tokens. The Evidence-Preserving Reducer accepts a cheap model's summary of a long log only if every quote in it can be found in the original.
How much does SoL-Pi reduce agent costs?
The paper reports 44.7% to 49% fewer tokens, and API cost down by about a third. Those are the authors' figures on their own benchmark, so treat them as a direction rather than a promise for your own setup. The only way to know what it saves you is to test it against your own harness.
Which SoL-Pi mechanism should you adopt first?
The Evidence-Preserving Reducer is the lowest-risk starting point. Its summary is only accepted when every quote checks out against the original, so the worst case is that you lose the shortcut, never the evidence. That makes it a cheap bet to try.
Why build your own agent harness?
Owning your harness means you can test new research against your real setup and get an answer the same day. Without one, a paper like SoL-Pi is something you read and guess about. With one, each new paper or repo is something you can take ideas from, so the harness keeps improving.