When a token costs less than grep, what you are buying is the same answer twice.

The numbers are on the author’s side

The strongest essay on Hacker News this week, by a comfortable margin, was jyn’s “Tokens too cheap to meter”. The argument is simple and the numbers are hard to wave away. Model efficiency per task has improved by something like a hundredfold in a year, hardware adds a little, inference software adds a little more, and the author multiplies it out to roughly two and a half orders of magnitude of decline in the cost of a token. Specialised small models are already quoted at tens of dollars per billion tokens. The same morning the essay was posted, OpenAI shipped GPT-6 Luna at ten cents per million input tokens, about half the price of what it replaced, as announced.

The provocative line in the piece is the one everybody quoted: at current rates, calling a language model will soon cost less than running a simple command-line tool. A call to a cheap model is only four or five orders of magnitude more expensive than a grep, one commenter worked out, and the gap is closing.

The best reply on the thread invoked Stein’s Law: if something cannot go on forever, it will stop. Compiled software like grep is a floor that models will approach and never cross. That is probably right about the physics. It also concedes the wrong point, because it accepts that cost is the axis the decision turns on.

What you actually pay for when you run grep

Grep earned its place because it gives the same answer on the second run, and the third, and on the machine of the engineer who joins next year. Its failure modes are enumerable. When it is wrong, it is wrong the same way every time, which means a person can find the wrongness once and fix it forever.

A model call, at any price, sells something different: a reasonable answer. For a very large class of work (drafting, summarising, triage, first-pass review, reading a stack trace nobody has seen before) a reasonable answer is exactly what is wanted, and cheap tokens are a gift. For another class, the class that production systems are mostly made of, a reasonable answer that differs from yesterday’s reasonable answer is the most expensive thing that can happen. It is expensive precisely because it is reasonable. A wrong answer that looks wrong gets caught. A different answer that looks fine ships.

So the mechanism is this. The cost of an operation in a production system is the price of running it plus the expected cost of it being different from what you relied on, and the second term is usually the larger one by several orders of magnitude. Cheap tokens drive the first term towards zero. They do nothing to the second, and by making it tempting to route more operations through a model, they multiply how often you pay it.

Three stories from the same front page

The week supplied its own examples, and none of them are about price.

DrivingBench put frontier models in control of a Toyota Corolla’s steering, throttle and brakes on a cone course. GPT-6 Astra was the only model to finish, and that is a genuinely impressive result. Look at how it finished, though, as published on the leaderboard: the first attempt ended at 49 per cent of the course, the second completed it in five minutes and twenty-two seconds, spending 246.6 million tokens for $7.74. The price was trivial. The interesting number is the spread between two runs of the same model on the same course, and no price cut changes it. In a benchmark that spread is a footnote. In a system with people downstream of it, the spread is the product you are actually shipping.

The second story is smaller and more instructive. A developer found that a popular coding agent’s support for project instruction files sat behind a remote feature flag, so with telemetry off the file was silently skipped. The fix was quick. The lesson is about what “the same input” means once a model is in the loop: the output depended on a flag the user could not see, served from a machine they did not own. Two engineers with identical repositories got different agents. Nobody paid a cent more, and nobody could have told from the answers.

The third is the one the essay’s own comment thread circled without naming. If tokens become cheaper than a tool call, the tempting move is to stop writing the tool and ask the model each time. The directory listing becomes a description of what is probably in the directory. The date parser becomes a prompt. The price argument says this is fine. The same-answer-twice argument says every one of those replacements has quietly turned an enumerable failure into a statistical one.

Buy the model once, run the tool forever

There is a pattern that gets the benefit of cheap tokens without paying the second cost, and it is the one we reach for first when a client wants a model somewhere in a production path. Use the model to write the deterministic thing, then run the deterministic thing.

A model that writes a parser, a validation rule, a SQL query or a migration script, which a person reviews once and a test pins down, has spent its tokens where tokens are good (the reasonable first draft) and left the repetition to something that cannot drift. The alternative, the model called fresh on every one of the ten thousand runs, pays for its reasonableness ten thousand times, and each of those is an opportunity for the answer to be reasonable in a new way.

This also answers the price argument on its own terms. The cheapest token is the one you never need to spend again, because its output was compiled into something that runs for free.

A worked example, as a composite

Here is the shape this takes in practice, drawn as a composite of the kind of back-office work we see rather than any one client. A finance team wants incoming supplier invoices coded to the right ledger account. There are about forty accounts and a few thousand invoices a month.

The first version everybody builds sends each invoice to a model with the chart of accounts in the prompt and takes the answer. It is impressive in the demo and cheap at this week’s prices; a few thousand calls a month on a small model costs less than the coffee in the finance office. It is also right most of the time, which is the problem. When it is wrong, it is wrong in a different place each month, and a miscoded invoice looks exactly like a correctly coded one until the quarter closes and somebody asks why the software line has doubled.

The second version uses the same model differently. It reads the last year of human-coded invoices and proposes a set of rules: this supplier always goes here, anything mentioning a subscription with these words goes there. A person reads the rules, deletes the three that are nonsense, and they become code with tests. The rules cover perhaps four fifths of the volume, deterministically, and the model only sees what the rules decline to handle. Those go to a queue a person clears in a few minutes a day, with the model’s suggestion attached.

The second version spends fewer tokens, which is nice and beside the point. What it buys is a system where the same invoice gets the same code every month, where a mistake is a rule someone can find and fix, and where the model’s variance is confined to the one place a person is already looking. That is the whole argument in miniature.

Where this argument is wrong

Some work has no deterministic version worth writing. Reading free text from customers, triaging a novel error, deciding which of four hundred alerts is interesting at three in the morning: for those, the model is the tool, variance and all, and cheaper tokens are straightforwardly good news. The right response there is to put a person’s decision at the point where the answer starts to matter, and to log enough that a divergence can be found afterwards.

And determinism itself can be oversold. Plenty of compiled software is nondeterministic in the ways that bite (clocks, ordering, network). So the claim here is a narrow one: variance is a property you should decide to accept, per operation, and falling prices are currently making that decision for people without anyone noticing.

What to do this quarter

Take the list of places your team now calls a model in a production path, including the ones inside agents and scripts that nobody thinks of as “the AI feature”. For each one, write down a single answer to a single question: if this returned a different but plausible result tomorrow, who would notice, and how long would it take?

Sort the list by that answer. The top of it, where nobody would notice for weeks, is where the model should be writing a tool rather than being the tool. The bottom, where a person reads every output anyway, is where cheap tokens are pure upside and you should spend them freely.

Then do one small, specific audit that this week’s instruction-file story suggests. For every configuration a model depends on (instruction files, system prompts, feature flags, tool lists), check that you can prove it was loaded on a given run. Hide a word in it if you have to. If you cannot prove it, you do not know what your system was told, and no price on earth makes that cheap.

Tokens are going to get cheaper. The author is right about that, and the curves agree. What they will not do is get more repeatable on their own. That part is still an engineering decision, and it is still yours.