In June, a three-person carpentry shop (hypothetical, but you may know one like it) is still paying for the AI quoting tool it set up in March. The first five draft quotes were promising. Then one missed the upcharge for paint-grade versus stain-grade trim, the owner fixed it by hand, and the following week he was too busy to try again. He's still writing quotes at the kitchen table after dinner.
Nothing broke. Nobody decided to stop. The pilot simply drifted out of use, which is what a stalled pilot usually looks like: not a failure anyone announces, just a tool that quietly stops being opened. The causes are ordinary, and so are the fixes.
Six places AI pilots stall (and why projects fail)
1. Nobody owns it
What you'll notice: questions about the tool get answered with “I thought you were handling that.” The boring fix: one named person, with an hour a week blocked on their calendar and the authority to change the rules. In a small shop that's often the owner. In an accounting firm it might be the office manager rather than a partner, because she'll actually have the hour.
2. The awkward cases were ignored
What you'll notice: the tool handles the easy cases and fumbles the odd ones, and each fumble costs more trust than each success earns. The boring fix: before launch, collect the awkward examples on purpose: the trim upcharge, the client with three business entities, the bank feed full of transfers between a client's own accounts. Decide for each one whether the system handles it or hands it to a person.
3. Nobody measured
What you'll notice: opinions instead of numbers. One person says it's great, another says it's useless, and neither has a count. The boring fix: record a baseline before launch (how long a quote takes, how many bank lines get recategorized at review) and log the same numbers every week.
4. Staff were never shown how
What you'll notice: people keep doing it the old way, or use the tool for things it was never set up to do. The boring fix: a 30-minute session built on real cases from your own work, a one-page cheat sheet, and a clear answer to “what do I do when it gets it wrong?”
5. The scope kept growing
What you'll notice: the pilot was meant to categorize bank feeds for ten bookkeeping clients. By week four it's also supposed to draft monthly reports and answer client emails, and none of it is finished. The boring fix: write the scope down, including what's out of it, and add nothing until the first job is live.
6. The output landed in the wrong place
What you'll notice: the tool produces a decent draft or category, but it sits in the vendor's dashboard, so someone copies it into the invoicing or accounting software by hand. After a few weeks they decide it's faster to skip the tool. The boring fix: during the side-by-side run, check where every output lands. If it isn't in the software where the work happens, fix that connection before go-live; build, buy or skip covers when connecting tools is the real project.
Before and after
A pilot that drifts vs. a pilot with a go-live bar
Drifting pilot
- 01Switched on for “a few clients” in January
- 02Staff quietly fix its errors during review
- 03No count of how often it was right
- 04Busy season arrives and everyone reverts to manual
ResultStill “in testing” in May
Pilot with a go-live bar
- 01Ten named clients, categorization only
- 02Every correction logged with a reason
- 03Bar agreed up front: 19 of 20 lines right, two weeks running
- 04Weekly review fixes the rule behind repeated errors
- 05Live for those ten clients before busy season
ResultRoutine lines run on their own; reviewers take exceptions and sign off at month-end
The pilot charter: one page before you start
Before any pilot begins, fill in these seven lines. If you can't fill one in, the pilot isn't ready to start, and you've just saved yourself three months of drift.
| Line | Example: carpentry quoting pilot |
|---|---|
| The one job | Draft quotes for built-ins from site notes and measurements |
| Out of scope | Decks, repairs, anything structural, and sending quotes to clients |
| Owner and weekly slot | The owner, Mondays from 7:00 to 7:20 AM |
| Known awkward cases | Paint-grade vs. stain-grade trim, scribe work, client-supplied materials |
| Baseline | The last five quotes took about 90 minutes each |
| Go-live bar | Five quotes in a row needing under 10 minutes of edits and no missed line items |
| Stop date | If the bar isn't met by [date], cancel or rescope |
The stop date matters more than it looks. Without one, a pilot can't fail, which means it can't succeed either. It just continues, costing a subscription and a little goodwill every month.
Flowchart
From pilot to daily use
- StartCharter written: one job, named awkward cases, what's out of scope
- StepRun side by side: the system and your team do the same work
- StepLog every difference between the two, with the reason
- DecisionHas it met the go-live bar two weeks running?NoA personOwner fixes the most common cause of misses and reruns; the stop date still appliesYes
- StepGo live on routine cases; people keep the exceptions
- A personWeekly 20-minute review of errors and hand-offs
- DoneIn daily use, with a record showing it works
The weekly review that keeps it alive
Twenty minutes, same time every week, same three questions:
- 01What did it get wrong, and why? Sort errors by cause: missing information, a rule that needs changing, or a case that should always go to a person.
- 02What did people work around? A workaround is an error nobody logged. Ask the people using it directly, not just the owner.
- 03What one change will we make this week? One, not five. A single change is easy to check next Monday.
Keep a running log of every change and why it was made. After a month that log becomes the system's real documentation, and the best possible answer to “why does it do that?”
Restarting a pilot that has already stalled
If you're reading this with a stalled pilot on your hands, you don't need to start over. You need one honest week:
- 01Pull the last ten real items the tool should have handled, whether it touched them or not.
- 02Run them through it now and note, for each one, what it got right, what it got wrong and why.
- 03Write the charter from the section above, using what those ten items taught you. Cut the scope to the cases it handled well.
- 04Name the owner and book the weekly slot before anyone touches the settings.
- 05Set a stop date four to six weeks out. If the bar isn't met by then, you'll stop with a clear reason instead of another quiet month.
For the carpentry shop, that week would probably show the drafts were close on cabinet boxes and weak on trim. Narrowing the pilot to boxes, with trim added by hand, gives it a job it can win.
When stopping is the right call
Not every pilot deserves rescuing. Stop, or shrink the scope, when the go-live bar still isn't met after two rounds of fixes, when the cases the system handles well turn out to be the ones that took no time anyway, or when reviewing its work takes longer than doing the work by hand.
A stopped pilot with a clear reason is useful information. It tells you what to try next, or that this task belonged with a person all along. The carpentry shop might find quoting too varied to automate yet, but change orders, which are short and frequent, a far better fit.
The pilots that make it are usually the ones that were scoped well before they started. If you're choosing what to pilot, a simple scoring test helps, and the plan at the end of an AI opportunity audit is essentially a charter like the one above. Once you're live, human-in-the-loop patterns help you decide how much review to keep.