Most AI pilots that fail did not fail technically; they stalled because nobody owned the decision to put them in front of real work. But that is the polite version. The blunter one is that the pilot did not fail at all. It succeeded, completely, at the job it was actually given, which was to win a budget. The job it was never given was to change how anyone works.
We have been putting AI into production since 2022, mostly for mid-market businesses in travel, luxury retail and wellness, and we now start every engagement by asking what the last pilot was for. Not what it did. What it was for. The answer is almost always some version of "to show the board we were doing something about AI". Judged by that standard, the pilot worked. The board was shown. Then everyone went back to work, and the work had not changed.
A pilot is optimised for the wrong audience
Picture the usual case. A travel operator has a team of six answering supplier queries by email. Someone builds a pilot that drafts the replies. The demo is run for the operations director, on twenty carefully chosen threads, and the drafts are good. Budget is approved. Six months later the drafts are still being generated, in a tool that sits in a second browser tab, and the six people are still writing their emails by hand, because the tab is one more thing to check and the drafts are wrong just often enough that checking takes as long as writing.
Nothing about that pilot was incompetent. It was built for the person who approves budgets, and it impressed them. The people it needed to impress were the six, and nobody asked them anything. This is the structural problem with the word "pilot": it names a phase, and a phase has a sponsor. Production does not have a sponsor. It has users, and users are a harder audience because they do not care whether the thing is clever. They care whether it is wrong, and what happens when it is.
The properties that win budget are the ones production punishes
A pilot wins budget by being impressive: broad, fast, confident. Production needs the opposite. Narrow, so that the failure modes are knowable. Slow at first, so that a person sees every action before the business feels it. Uncertain, so that the system says "I am not sure about this one" and hands it to someone rather than guessing. The pilot that impressed the board would be an embarrassment in production, and the system that survives production would have looked timid in the demo. Teams that do not understand this build one and then wonder why it cannot become the other.
The clearest sign is permissions. In almost every pilot we inherit, the system runs with whatever access the developer had that afternoon. Nobody decided what it was allowed to do, because in a demo the question does not arise. In production it is the first question, and answering it usually means rebuilding the thing.
"Human in the loop" is where responsibility goes to hide
Pilots stall in the gap between IT, a vendor and a sponsor because each assumes one of the others will decide when it is ready. The phrase that lets them all off the hook is "there is a human in the loop". It sounds like a control. It is a description of the fact that a person exists somewhere near the system. It does not say which person, what they see, what they can change or what is recorded when they do. When something goes wrong, the loop turns out to contain nobody in particular.
Production means a named person putting their name on a workflow. They will do that only when they can see what the system did, why it did it and how to undo it. That is not a feature you add at the end. It is the design.
The Tuesday test
We use one test to tell a pilot from production, and it has nothing to do with the technology. Whose Tuesday is different? Name a person in the business who does something differently on an ordinary Tuesday afternoon because the system exists. If nobody, it is a pilot, whatever the project tracker says. If one person, it is in production, however small it looks. One desk in production is worth more than a department in pilot, because the one desk generates the evidence that lets you add the second.
What to do with the pilot you already have
Do not throw it away and do not scale it. Ask the six people. Sit with them for two days and find the one step in their day they would give up tomorrow, then rebuild the pilot to do only that, inside your own environment, with a review step in front of it and a way to undo it. Let one person use it for real, for a month, with every action reviewed. Count the corrections. When the corrections fall to a level the reviewer is happy with, give it to the second person. That is production. It is smaller than the pilot and it changes more.
The vendors are not the problem here. The models are good enough and have been for a while. The gap between a pilot and production is not a model gap; it is an ownership gap, and it sits on the buyer's side of the table. That is why the next release will not close it.