Somewhere in the last two years, most small and mid-sized companies tried an AI tool. Usually a chatbot. The demo was impressive, the setup took an afternoon, and for a few weeks it answered questions on the website. Then a customer asked it something real, it gave a confident wrong answer, and someone quietly switched it off. The owner drew a reasonable conclusion: AI doesn’t work for a business our size.
The conclusion is wrong, and the research on why is more useful than it sounds. The chatbot was the visible tenth of a system. Three other parts were never built: a connection to the business’s real data, a handoff to a person, and someone whose job it was to watch it. All three are ordinary engineering. Here’s how the businesses that get value from AI put them in place, and how we do it.
First, the good news inside a bad number
of organizations get zero measurable return from their generative AI pilots
MIT’s Project NANDA reviewed more than 300 public AI initiatives, interviewed 52 organizations, and surveyed 153 senior leaders in the first half of 2025. Ninety-five percent of organizations had nothing to show for their pilots. The report is just as clear about the other 5%: they picked one narrow, high-value job, wired the tool into the systems people already use, and let it learn from corrections. Same models, different plumbing.
Source: MIT NANDARAND reached a matching conclusion from the engineering side. Its interviews with 65 experienced AI engineers found the leading cause of failure was that nobody had agreed what problem the project was meant to solve. Missing data came second, then chasing the technology for its own sake, then missing infrastructure. Four of the five root causes are decisions rather than code, which is what makes them fixable.
Fix one: connect it to your systems before it talks to anyone
Ask an owner what the chatbot knew, and the honest answer is usually the FAQ page. It couldn’t check whether Tuesday at three was free, whether an invoice had been paid, or whether the part was in stock. That information lives in a booking tool, an accounting system, a spreadsheet, and someone’s head. The bot could only recite, and customers worked that out in one conversation.
of AI projects without AI-ready data will be abandoned through 2026
Gartner’s survey of 248 data leaders found 63% either don’t have the data practices AI needs or aren’t sure. For a company of twelve or two hundred that means the same concrete thing: the answers a customer wants sit in four or five systems, and the assistant has to be able to read them. That’s the first thing we build. We map where each answer lives, connect the assistant to those systems with read access, and only then let it near a customer. When someone asks where their order is, it looks the order up. When it can’t find an answer, it says so instead of inventing one.
Source: GartnerFix two: give customers a way to reach a real person
The second gap is the one customers notice first. If the bot can’t answer a question, most people will forgive it once. If it can’t answer and there’s no way to reach a person, they leave. What they want is simple: let the assistant handle the easy things, and open a door to a real person the moment it can’t.
of customers say a company using AI for service must offer a way to reach a human
Gartner surveyed 3,566 customers in early 2026. In the same survey, half said AI made their interactions easier, so this isn’t hostility toward the technology. People are happy to let an assistant handle the routine, provided the hard cases reach someone who can decide.
Source: GartnerKlarna learned this in public. In February 2024 it said its AI assistant was doing the work of 700 agents. By May 2025 its CEO was telling Bloomberg that cost had been too dominant a factor, that the result was lower quality, and that the company was hiring people back. Klarna kept the AI running and put humans back on the cases it couldn’t handle. That split is the one to copy at any size, and Gartner’s numbers show where the line falls: back in 2023, chatbots resolved only 17% of billing disputes.
So we build the handoff first. The assistant knows the edge of what it can do. When a question falls outside that edge, or the customer asks for a person, the whole transcript lands with a named human in a channel they already read: the inbox, a Slack channel, a text message. The customer never repeats themselves, and the person picking up already knows what was tried.
Fix three: someone has to own it
The third gap is the quietest. The chatbot was bought by whoever saw the ad, set up in an afternoon, and from then on belonged to no one. Nobody read the transcripts. Nobody updated it when prices changed in March. Nobody noticed it had spent a month telling people you’re open on Sundays. A tool that talks to customers needs an owner the way a till needs someone to cash it up.
of AI tools built with an outside partner reached deployment, against about a third of those built in-house
MIT found that partner-built tools reached deployment about twice as often as in-house ones, and the tools that lasted were the ones that learned from corrections. The report also found roughly half of generative AI budgets going to sales and marketing, while the better returns sit in back-office work like document handling and reporting. Smaller companies repeat the pattern in miniature: the customer-facing chatbot gets the attention while the invoice pile that eats ten hours a week goes untouched.
Source: MIT NANDAOur answer is a one-page agreement before anything is built: the one job the assistant does, what a good answer looks like, and who on your side owns it. Then a monthly look at the transcripts together, so what it got wrong last month is fixed this month. Prices change, policies change, the assistant changes with them. That loop is what separates a tool still running in a year from one switched off in week six.
What a first build with us looks like
Put the three fixes together and a second attempt looks less like a chatbot and more like a small piece of infrastructure. It starts with a short review of your operations to find the one job worth doing first, which is often behind the counter rather than in front of it. From there the build has four parts:
- One narrow job it can finish end to end. Rescheduling appointments, answering order-status questions, sorting the shared inbox. Not “customer service.”
- Read access to the systems that hold the answers. Booking tool, invoices, stock list, so it answers from your data instead of a text box you filled in once.
- A handoff that carries the full conversation to a named person, in a channel they already check.
- An owner and a monthly review, so it improves instead of drifting.
The model is the easy part; you can swap it in an afternoon. The connections, the escape hatch, and the person watching it are the system, and they’re what you own when we’re done. Build those and the chatbot mostly takes care of itself.
Sources
- 1.MIT NANDA — The GenAI Divide: State of AI in Business 2025 — Review of 300+ AI initiatives, 52 interviews and 153 surveyed leaders (Jan–Jun 2025): 95% of organizations see zero measurable return; the 5% that succeed use narrow, integrated, learning-capable tools; ~50% of GenAI budgets go to sales and marketing; partner-built tools reached deployment ~67% of the time vs ~33% for internal builds.
- 2.RAND — The Root Causes of Failure for Artificial Intelligence Projects (August 2024) — Interviews with 65 experienced AI engineers: the leading cause of failure is misunderstanding what problem the project should solve, followed by missing data, technology-chasing, missing infrastructure, and problems beyond AI's reach.
- 3.Gartner — Lack of AI-ready data puts AI projects at risk (February 2025) — Through 2026, organizations will abandon 60% of AI projects unsupported by AI-ready data; 63% of 248 surveyed data leaders lack, or are unsure of, the data practices AI needs.
- 4.Gartner — 87% of customers want access to a human agent (August 2026) — Survey of 3,566 customers, February–March 2026: 87% say companies using GenAI for service must provide access to a human; 50% say GenAI makes interactions easier.
- 5.Gartner — chatbot use in customer service (June 2023) — Survey of 497 customers: chatbots resolved only 17% of billing disputes, against far higher rates for simple requests.
- 6.Customer Experience Dive — Klarna reinvests in human customer service (May 2025) — Klarna's CEO told Bloomberg that cost had been too dominant a factor in its AI rollout and "what you end up having is lower quality"; the company kept AI for routine volume and resumed hiring human agents.
SmartBloc