Nobody gets fired for not using AI. People get fired for what they approved.

The two fears, and which one is real

There are two fears in circulation right now and they point in opposite directions.

The first is the one Fireship and Theo and every LinkedIn feed are selling: you are behind. The models moved again, the people using them properly are producing five times what you are, and if you do not get on this you will be the person the organisation quietly stops giving interesting work to. There is a version of this fear that is accurate and we will get to it. But as a career risk it is slow. Nobody gets walked out on a Tuesday for having been sceptical of a tool.

The second fear is the one that actually ends careers, and almost nobody is articulating it because it is less flattering to the tools. On Thursday CNN reported that the US military had what it called a close call after an AI-generated intelligence report contained hallucinated content that made it some distance up the chain before somebody caught it. We do not know the details beyond what was reported and we are not going to pretend to. But read the top comment on the Hacker News thread, because it is the whole essay in three sentences: AI is not responsible. People are responsible. As soon as people choose to remove their own accountability, that is when the bad stuff happens.

The same week, OpenAI launched Astra for Law, a model tuned for legal work. The launch post, as several commenters noted, does not mention hallucination once. The most upvoted question under it was not about capability. It was: if I use this to write my contracts, who is getting sued when it is wrong? That question has an answer, and the answer is you.

Where the signature lands

Every organisation has a small number of places where a person converts information into a commitment. A lawyer signs the brief. An officer signs the assessment. A reviewer approves the change. A controller approves the payment. A doctor signs the order. These are the seats where the organisation has decided, in advance, that a named human stands behind the outcome.

For the whole history of office work, the person in the seat also produced most of the information that went into the decision, or supervised the junior who did. That has quietly stopped being true. The brief was drafted by a model. The assessment was summarised by a model. The pull request was written by a model. The person in the seat is now signing for work they did not do, produced by a process they did not watch, at a volume that does not leave time to check.

This is the structural fact under the anxiety. The tools did not remove the seat. They removed the slack around the seat, the four hours of drafting during which a person used to notice things. And in a lot of organisations they removed it faster than anybody redesigned what the person in the seat is supposed to do with their eight seconds.

Martin Fowler’s short piece this week, the one titled simply that he does not like LLMs, is being read as a grumpy-elder post. It is more useful than that. His complaint is that they confidently bullshit him, often while giving useful answers, with the same assurance either way. That is not a complaint about capability. It is a description of the exact property that makes the approval seat dangerous: the output does not carry a signal about its own reliability. A junior who is unsure looks unsure. A model that is unsure looks like a model that is sure.

What the people who survive this will have done

We build systems where a model is allowed to act, inside a gate a person controls, and the entire design problem is the seat. So here is what we have learned about occupying one well, offered to the lawyer and the officer and the reviewer as much as to the engineer.

The first thing is to refuse to sign for what you cannot see. That sounds obvious and it is violated constantly, because the tools present a finished artefact and hide the process. Insist on the provenance: what was the model given, what did it produce, what did it check, what did it say it was unsure about. If the tool cannot show you that, the tool is asking you to carry liability for a process it will not disclose. The correct response to that is not faster reading.

The second is to make the model’s abstentions visible and to treat them as the most valuable line in the output. A model that has been built to say “I could not verify this citation” or “this clause is outside the examples I was given” hands the seat exactly the information it needs to spend its attention well. A model that has been built to always answer hands the seat nothing. Most vendors ship the second kind because it demos better. Buy the first kind, or add the gate yourself.

The third is the record. When something goes wrong, and it will, the difference between a career ending and a process improving is whether there is a log that shows what the seat was shown, what it decided, and why. Approvals without a record are how organisations find scapegoats. Approvals with one are how they find bugs. If you are the person in the seat, you want the record more than your employer does.

The fourth is the one that answers the first fear honestly. You do need to understand the tools. Not to produce more, though you will, but because the person who understands how a model fails is the only person who can occupy the seat competently. A reviewer who has never watched an agent confidently rewrite a retry loop it did not understand cannot review agent-written code. A lawyer who has never seen a model invent a citation cannot sign a model-drafted brief. Falling behind is a real risk, and the reason is not productivity. The reason is that the seat is about to be occupied exclusively by people who know what they are looking at.

What it looks like when it works

A composite from the systems we run, with the details changed. A finance team processes vendor invoices through a model that reads the document, matches it to a purchase order, and proposes a payment. The person in the seat sees, for each proposal, four lines: the amount, the matched order, the confidence of the match, and a list of anything the model could not reconcile. Most proposals show an empty list and take three seconds. A handful show one line, “invoice quantity 12, order quantity 10”, and take thirty. Once or twice a day one shows “no matching order found” and gets routed to a human who investigates.

The team approves several hundred payments a day this way, which is roughly six times what it did by hand, and the person in the seat is not faster at reading invoices. They never read the invoice. They read the abstentions. The model was built to say what it did not know, the seat was built to show only that, and the record of every approval, with what the seat saw, is kept for the auditor.

When a duplicate payment went out in the spring, because a vendor had sent the same invoice under two numbers, the investigation took twenty minutes. The record showed the seat had been shown two proposals, each with an empty abstention list, three days apart. Nobody was blamed, because nobody had been shown anything that should have stopped them. The model was changed to flag matching amounts to the same vendor inside thirty days. That is what the seat is for: not to be infallible, but to make failure legible enough that the next one does not happen.

Compare that with the version most organisations are building, where the same person is handed a finished payment run and asked to approve it by five. That seat has the same liability and none of the information, and when the duplicate goes out the investigation finds a name rather than a bug.

The uncomfortable version for managers

If you run a team, the obligation runs the other way. You have almost certainly increased the volume flowing into your approval seats this year without increasing the time, the tooling, or the authority of the people in them. You may have done it with a slide that said something about productivity. When one of those seats signs something that goes wrong, the organisation will look at the name on the signature, not at the slide.

The remedy is not to slow down. It is to treat the seat as the product. Decide, explicitly, which decisions a model may take alone, which need a person, and which need a senior person. Write it down. Give the seats the provenance, the abstentions and the record described above. Cap the volume per seat per day, because a person who approves three hundred things approves none of them. Then, and only then, let the models run.

The last month of discourse has been about who is behind. The next year is going to be about who signed. Make sure that when it is your name, you were shown enough to deserve it.