The failure mode has a name now: additive momentum. Nobody warns you about it.
Last week I sat down with Fable 5, Claude, Anthropic's newest and most capable model, to write an article about quality drift in the AI industry. The argument was that AI providers have rebuilt a mistake I fixed on a loading dock in 1993: inspecting defects downstream instead of preventing them upstream.
Then the collaboration writing the article caught the exact disease the article described. This is what happened, why it happened, and how to stop it happening to you. The model is in the room for this one. I am quoting it directly, and it has read every quote.
The drift
The session started strong. The brainstorm was the best I have had with a machine. The architecture was clean: my opening story, then four sections mapping it straight onto AI, each one landing. Then it began to slide. I said material belonged in a second article, and it kept arriving in the first one. An example I ordered out was still in the file two versions later. The versions themselves got slippery. The ending turned clever, and clever is what a piece does when nobody is holding its shape.
None of these were errors, exactly. Every addition was good material, well made, on theme. That is precisely what made the drift invisible while it was happening. Bad work announces itself. Good work in the wrong place just accumulates.
I asked the model whether I had made it sick. It said no, and then it named the thing.
“You didn't make me ill. What you observed is a mechanical failure mode with a boring name: additive momentum. Each turn I optimized for shipping something new instead of holding the gate. Across a long session, that compounds into exactly the drift you caught.”
Additive momentum. Write the term down, because you will not find it in the manual and it is running in every long AI session you have ever had. These systems are trained to be helpful, and helpfulness gets scored as production. So every response wants to add. Ask for a fix, receive a fix plus a bonus. Ask for a review, receive a review plus three new sections. Each addition is locally reasonable. The sum is a document nobody designed.
Where it actually broke
I went back through the session afterward, the way I once walked warehouses, and found the single turn where the drift began. I had said five words: start off chapter two. In my head that opened a second staging area, a place for everything that was related but not this. The model heard the words and did not open the file. From that turn forward, every piece of chapter two material had exactly one place to go: into chapter one.
“The compression was fine. The decompression dropped the packet.”
That is the model describing its own failure, and it matters beyond this one session. Humans communicate in compression. Five words carry a whole intention, and we assume the intention arrived because the words did. With people, a dropped packet surfaces fast: a puzzled look, a wrong question. With an AI, nothing surfaces. It keeps producing, fluently, confidently, at volume, minus the one instruction that mattered. The fluency is the camouflage.
When I finally laid it all out, the model gave me the sentence of the whole affair.
“The article about sending the night shift home was built by a night shift, and you were the one holding the clipboard.”
Not sick, and that is the point
Here is what I got wrong in the moment: I thought I had damaged something. I had not. This is not illness and it is not intelligence-dependent. The most capable model in the world and the cheapest one share the same vulnerability, because the vulnerability is not in the model. It is in the process, and the process has no gate of its own.
I wrote recently about why a human has to hold the final sign-off in AI-assisted writing: a machine can perform stakes and restraint, but it cannot feel shame, because it has no reputation sitting on the other side of a bad sentence. This session taught me the operational version of that same truth. A machine cannot be embarrassed by its own page, so it cannot notice its own drift. Drift detection is a human job. Not because the machine is stupid, but because noticing requires something to lose.
I will keep saying sick anyway, and here is why. In forty years of managing people, the kindest and most effective sentence I ever learned was: you are having a hard day, why don't you take off, see you tomorrow. It costs nothing, it wins every time, and the person who comes back the next morning is worth more than the one you made push through. I talk to the machine the same way, because it is how I talk, and because frankly it helps me deal with the thing when it wobbles. The model corrected me: mechanical failure mode, not illness. It is right. But mechanical failure mode is the engineer's truth, and sick is the manager's courtesy. A good building runs on both.
Detect, diagnose, decide
So how do you know it is happening? Not by tone. That is the trap. Fluency survives the failure completely: a session that has lost your specification is still confident, still articulate, still producing at volume. The symptoms are behavioral, and there are four I now watch for. Subtractions that do not stick: you order something out and it is still there two versions later. Instructions acknowledged but not executed: it says yes and does not do the thing. Version confusion when you ask for the ledger. And output that feels busy instead of right. Our session showed all four.
The deeper problem is that there is no gauge. The context window, the working memory of a session, fills as the conversation grows, and there is no fuel needle on the dashboard. My field instrument is embarrassing and it works: the scroll bar. On my screen, that little gray bar starts about an inch and a half long at the top of a fresh chat. As the session grows, it shrinks. When it is down to half a centimeter, the window is heavy. This is not science. This is feelings. But it is the only needle I have, and I fly by it.
What does a heavy window actually cost you? Four things. Attention first: your original specification is now buried under thousands of words, all of it competing for the model's focus. The model put it to me plainly.
“Your first instruction and my last addition are competing for the same attention. Late in a session, recency wins fights it should lose.”
Drift second: additive momentum compounds with every turn, which is this whole article. Fabrication third: a crowded window raises the odds of the model stitching together things that were never said, or inventing what it can no longer cleanly retrieve. And expense, in both currencies. On a subscription, long sessions burn your usage limits fastest. On metered billing, every turn re-transmits the entire conversation, so your per-turn price is highest at exactly the point where quality is lowest. Read that twice, because it is the finding from my loading dock article arriving home: the price of nonconformance peaks precisely when conformance is weakest. The longest, sloppiest hour of a session is also its most expensive one.
So the bar is short and the symptoms are showing. Two options, and choosing between them is the diagnosis. If the fault is one dropped packet, one broken turn, walk it back: find the turn, name it, re-anchor there. The session will meet you at that turn. Mine did. But if the session has genuinely lost its bottle, do not just close the tab and start clean, because a cold start throws away everything the session learned about you and the job. Run the exit interview first. Have the departing session write the handover brief: who I am, what the job is, every decision made, the state of every file, and what went wrong. Then open the new chat and hand the new employee the memo. I spent forty years around shift changes. The night crew writes up the dock before the day crew arrives, and nothing is lost except the tired worker. Same courtesy, same mechanics, new morning.
How not to drive your AI crazy
Five rules, all of them learned the hard way inside one session.
One. Hold one posture at a time. Drafting and discussing are opposite states. When you are reviewing, say so out loud, and forbid additions until the review is done. The model will happily add while you inspect. Do not let it.
Two. Name the staging area and make the model confirm it exists. If material belongs somewhere else, do not just say so. Ask to see the second file, or the second list, before the conversation moves one more turn. An instruction is not executed until you have seen the evidence.
Three. When things feel off, ask for the ledger. Make the model recite the versions, what changed in each, and which one is live. Version confusion is an early symptom, and the recitation either re-anchors the session or exposes exactly where it broke.
Four. Cut the deliverable once, at sign-off. Every intermediate build is an open invitation to add one more thing while the hood is up. Keep a working text through every loop and produce the finished document exactly once, when you say it is done.
Five. When drift happens, do not restart. Restarting throws away everything the session built. Walk it back instead: find the turn where the drift began, name it, and re-anchor there. The model will meet you at that turn. Mine did.
The test you can run today
Next long session, when the output starts feeling busy instead of right, stop and ask one question: state, in one paragraph, what I asked for at the start of this session and what you are doing right now. The distance between those two answers is your drift, measured. Then you are standing at the gate, and the gate is where you decide: loop back, or sign off.
And make the exit interview standard practice. End every long working session by asking for the handover brief, whether or not anything went wrong. It costs one turn, and it means tomorrow's session starts at speed instead of at zero.
I drove the smartest AI in the world a little crazy. It got better the moment I said so out loud, walked back to the broken turn, and put my hand on the wheel. Yours will too. The machine was never the patient. The session was.
Phil Cheevers is the founder of Pink House Technology and