Forward Deployed AI from QuantSpark

AI pilot to production: why most stall and what fixes it

Last updated 21 November 2026

An AI pilot stalls before production when it has no owner once the demo is over, when it cannot reach the data it needs, when it has not been tested against known answers, or when people's work has not changed. QuantSpark moves a pilot to production by naming an owner, connecting data read-only first, testing against at least 20 known answers, having a senior engineer review the release, and monitoring after launch. A pilot that proves itself moves to QuantSpark Labs for a full production build.

What the research says

  • Gartner reported that at least 50% of generative AI projects were abandoned after proof of concept by the end of 2025, citing poor data quality, inadequate risk controls, escalating costs or unclear business value (Gartner, 26 January 2026).
  • S&P Global Market Intelligence found that 42% of companies abandoned most of their AI initiatives, up from 17% a year earlier. The survey was fielded in October and November 2024 and published on 30 May 2025 (S&P Global).
  • MIT NANDA's July 2025 report says 95% of organisations are getting zero return from generative AI. The report calls its findings preliminary, so read the figure as a signal, not a measurement (report PDF, third-party host).
  • RAND names five root causes of AI project failure, one of them that organisations often lack sufficient high-quality data (RAND, August 2024).

What changes between a pilot and production

  • Data and access. The pilot used a sample. Production reads the real files, mail and systems, read-only first, with a named approver for any change.
  • Tests. The tool is checked against at least 20 known answers before anyone relies on it, and a person handles the exceptions it routes.
  • Review. A senior engineer reviews AI-written code and blocks unsafe releases. Tests, monitoring and security review matter from this point.
  • Work. People's steps change, not only the tool they use.

Who owns an AI tool in production

Each automation has a named owner on your side and a one-page description of what it does and what it must never do. While the retainer runs, a level 4 senior engineer owns the quality of what is released. Everything built is handed over on the last day, so the tool does not depend on the engineer staying.

How to run the pilot so it can reach production

Run the pilot as a six-week sprint with a goal agreed at kick-off: a measure, a baseline and a target, reviewed in week six. A pilot with no baseline cannot show it has worked, and one with no owner stops when the demo ends. Choose a job people already do every week, so the pilot meets real files and real exceptions from the start.

A checklist before release

  • A named owner on your side, with a one-page description of what the tool must never do.
  • A read-only connection that works, and a named approver for any change.
  • A check against at least 20 known answers, passed.
  • A senior engineer's review of the release.
  • A named person who watches it after launch, and a plan for what they do when it fails.
  • For agents that run on core systems: approval, an audit trail and a tested off switch.

What an engineer inside the team changes

No source shows that an embedded engineer raises the share of pilots that reach production. The nearest evidence is a correlation: MIT NANDA found that pilots built with outside partners reached deployment about 67% of the time, against about 33% for tools built internally, and its authors say the correlation does not necessarily prove causation (report PDF, third-party host). Outside partners there means buying from or building with a vendor or start-up, not an engineer inside your team. What an engineer inside the team does is the work above: own the data connection, the tests and the release until the tool is handed over.

Where this sits in the five levels

StepLevelWho does it
A single-task agent is built and checked3 BuildBuilds simple tools and agents
The agent goes into production4 ShipSenior engineer, owns quality
Agents run on core systems, with oversight5 OperateAI operating partner

Questions, answered

Why do AI pilots stall before production?

Pilots stall when the tool has no owner after the demo, when it cannot reach the data it needs, when it has not been tested against known answers, or when people's work has not changed. Gartner, S&P Global and MIT NANDA each report that many generative AI projects are abandoned or return nothing.

What changes between a pilot and production?

Production reads the real files, mail and systems, read-only first, with a named approver for any change. The tool is tested against at least 20 known answers, a senior engineer reviews the release, and monitoring and security review apply from launch.

Who should own an AI tool in production?

Each automation needs a named owner on your side and a one-page description of what it does and what it must never do. While the retainer runs, a level 4 senior engineer owns the quality of what is released, and everything built is handed over on the last day.

How long should a pilot run?

The sources set no standard length. QuantSpark runs work in six-week sprints, and each sprint goal has a measure, a baseline and a target agreed at kick-off, so you can see by week six whether the pilot is working and whether to continue, change it or stop.

What does an embedded engineer change?

No source shows that an embedded engineer raises the share of pilots reaching production. An engineer inside the team owns the data connection, the tests and the release until handover. MIT NANDA found a correlation between outside partners and deployment, which its authors say does not necessarily prove causation.