A pilot is designed to answer the question “can this work?” It almost always can. The honest problem is that nobody commissioned an answer to the harder question underneath it: will this be used, by these people, on an ordinary Tuesday, when we are not in the room?
That gap is not a technology failure. It is a scoping failure, and it is predictable enough to design around.
The pattern
A pilot is run by the people who most want it to succeed, on data they have cleaned, for users who volunteered. It is judged on whether the output is impressive. On every count, the conditions are the opposite of the ones production will impose.
Three things reliably go missing.
An owner after launch. The pilot has a sponsor. The system needs someone whose job it is to run it — to watch it, to answer for it when it is wrong, and to decide what happens next. Pilots that end without that person named do not fail; they simply stop.
The unhappy path. Demos show the case the model handles. Production is mostly the other cases: the malformed input, the exception the policy did not anticipate, the request that should be refused. A pilot that never shows what happens when the answer is wrong has not been tested, only exhibited.
The surrounding work. The model is one step in a process that has a before and an after. If the step before it still arrives as an email, and the step after it still needs a person to re-key the result, the model has saved nothing that anyone will notice.
Why it happens
Because a pilot is cheap to say yes to and a production system is not, organisations reach for the pilot as a way of deferring the decision. That is reasonable — until the deferral becomes the strategy.
The incentives then quietly align against finishing. A pilot that succeeds creates work: integration, controls, training, a support rota, a budget line, someone’s changed job. A pilot that merely concludes creates a slide. The second outcome is easier for everyone, which is why so many programmes produce a steady supply of promising pilots and no systems.
There is a structural version of this too. The further AI moves into an organisation, the more it meets functions that were never part of the pilot — security, records, legal, procurement, the people whose workflow changes. Each has a legitimate veto and none was consulted. Delay then looks like obstruction, when it is really a conversation that should have happened at the start.
What to do instead
Scope the pilot to end in a decision, not a demonstration. Write down in advance what result would make you build it and what result would make you stop. A pilot that cannot fail cannot inform anything.
Build the riskiest part first, on real data. Not the most impressive part — the part most likely to kill the project. If the data is unusable, or the accuracy is unacceptable in the exceptions that matter, you want to know in week two rather than month five.
Name the operator before you write any code. Someone whose existing job this changes, who will run the system afterwards, and who is in the room for the decisions. If no such person can be named, that is the finding, and it is worth more than the pilot.
Bring the vetoes in early. Security, legal and records are cheaper as participants than as reviewers. Ask them in week one what would make them say no, then design to it.
Measure the process, not the model. Take a baseline of how long the work takes end to end today. Model accuracy improves the step; only the end-to-end number tells you whether anything reached the business.
Where this is going
The stock of unfinished pilots is becoming its own problem. Boards that approved exploratory work two years ago are now asking what came of it, and the honest answer in many organisations is a portfolio of things that worked and were never adopted.
The useful consequence is that the question is shifting from “what could we do with AI?” to “what have we actually put into service?” That is a harder question and a better one. It rewards organisations that finish few things over those that start many — and it means the next phase of this work looks less like experimentation and more like ordinary, unglamorous delivery.
