Most AI pilots don't fail at the demo. They fail after it.

MIT's NANDA initiative looked at more than 300 public AI deployments, plus surveys of 153 executives and interviews with 52 leaders, for its "State of AI in Business 2025" report. The number that made headlines: 95% of generative AI pilots produced no measurable return.

Most people read that as "AI doesn't work yet." That's not what the report found. The pilots that stalled weren't failing at the task — a support bot answering tickets, a document summarizer, a lead-qualification agent usually does the thing it was built to do in the demo. They fail to survive contact with the rest of the business afterward.

The pattern behind the number

We see the same shape on almost every automation engagement, and it has nothing to do with model quality:

• **Nobody owns it after launch.** A pilot gets built by a vendor or a hackathon team, it works for six weeks, then the person who understood it moves to another project. No one is watching the logs, so nobody notices when it starts silently failing on 15% of cases.
• **It doesn't touch the systems of record.** The agent drafts a summary or a recommendation, but a human still has to copy it into the CRM or the ERP by hand. The automation removed the thinking step but kept the busywork step, so the time saved doesn't show up anywhere measurable.
• **It doesn't learn from correction.** Someone fixes the agent's output every day, but those fixes never make it back into a prompt, a rule, or a retrieval source. The system makes the same mistake in month six that it made in week one.

None of these are model problems. You could swap in a better model tomorrow and the pilot would still die the same way, because the failure is organizational, not technical.

Why "the demo worked" is the wrong bar

A demo proves the model can do the task under ideal conditions with someone in the room to catch edge cases. Production asks a different question: what happens when the input is malformed, the API is down, the customer asks something nobody scripted for, and there's no one watching?

Harvard Business Review's analysis of automation projects found that companies doing structured process redesign before automating saw roughly triple the ROI of companies that automated their existing process as-is. That matches what we see: the pilots that make it past the demo are usually the ones where someone rewrote the workflow around the automation, not the ones where the automation got bolted onto the old workflow.

Timelines back this up too. A narrowly scoped, already-standardized task — categorizing inbound emails, matching invoices to POs — can show ROI in six to ten weeks. Anything that requires cross-team process change, new approval chains, or a shift in who's accountable for what, realistically takes six to twelve months. Pilots that get killed at the ninety-day mark are usually the second kind of project being judged on the first kind's timeline.

What to check before you scale a pilot

If you're deciding whether to take a pilot from "it worked in the demo" to "it runs in production," ask:

1. **Who gets paged when it breaks?** If the answer is "nobody, it just stops," you don't have an owner yet. 2. **Does a human still re-key the output anywhere?** If yes, you've automated the easy 80% and kept the annoying 20%, which is often where the labor cost actually was. 3. **Where do corrections go?** If fixing a bad output doesn't change what the system does next time, you have a demo, not a system. 4. **What's the standardized version of this process?** If three people do the task three different ways today, automating it will just make one of those ways happen faster and more often — including the wrong one.

This is also where we tell clients not to automate. If a process changes shape every quarter, or three departments each have a veto over how it should work, building an agent for it now just adds a fourth thing that breaks when the process shifts again. Fix the process first, or accept that the automation will need a rebuild on the same cadence as the process.

The 95% figure isn't a verdict on the technology. It's a measure of how many pilots got handed to nobody, plugged into nothing, and left to quietly rot. The 5% that worked mostly did one boring thing: someone was still responsible for it three months after launch.