Messy Data Is Why Most AI Projects Fail (Fix This First)

Listen to this article

Messy Data Is Why Most AI Projects Fail (Fix This First)

0:00

Fixing your data before AI, in ten slides

Save it, share it, or send it to whoever is about to sign an AI contract.

Swipe to preview · 10 slidesDownload the PDF

Why buying AI first is a mistake

Many company leaders buy powerful AI models before they fix the data underneath them. This feels like progress. It is the opposite. An AI model is only as good as the data it can see. If that data is locked away in old, disconnected systems, the AI cannot find it, cannot understand it, and cannot use it safely.

Gartner forecasts that 60% of AI projects will be abandoned through 2026, and data fragmentation, meaning it is scattered across old databases that don’t talk to each other, is a leading reason why. When AI cannot see clean, connected data, it does one of three bad things. It gives wrong answers. It makes up facts that sound true but aren’t, a problem called hallucination. Or it accidentally shows sensitive customer information to people who should never see it.

The data backs this up from more than one angle. McKinsey & Company’s 2023 AI survey found that companies with unified data platforms, meaning all their data lives in one connected system, were far more likely to get real economic value from AI than companies with scattered data, the 2.5x figure shown above. IBM’s 2023 Cost of Data Breach Report put a number on the downside of getting this wrong: poor data rules and hidden data silos cost companies an average of $4.45 million every year, through security risks and wasted work, the same figure shown above.

Companies that build a shared data layer and automatic data checks before they turn on AI models cut their data preparation time in half within six months, compared to companies still relying on slow, manual data cleanup.

Three different studies, one same message: AI does not fail because the model is weak. It fails because the data underneath it was never made ready.

Three ways to fix scattered data

Once a company decides to fix its data before turning on AI, it has three main paths to choose from. Each one is a real technical approach, and each one trades speed against cost and complexity. Picking the wrong one for your situation can waste months of work.

There is no single “best” option. A Data Fabric is fastest to deploy but can slow down live AI searches. A Lakehouse gives the strongest analytics but costs the most and takes the longest. A Data Mesh scales well but needs real buy-in from every team.

Four steps to get data ready for AI

Whichever path a company picks, four steps have to happen before an AI model can be trusted with real company data.

Step 1: Find every data silo. A company cannot fix data it cannot see. Teams need to scan every database, every sales system, and every customer records system. This means automatically scanning for data tables nobody remembers exist, tagging any personal information so it follows privacy laws like GDPR, and tracking exactly where every piece of data came from and where it goes.

Step 2: Set clear rules for who owns what. When one central team controls all data decisions, approvals slow to a crawl. It works better to give each business team ownership of its own data, while everyone follows the same company-wide rules. That means naming a clear owner for every type of data, controlling who can access what through role-based permissions, and using the same naming rules and date formats company-wide, so “customer_id” doesn’t mean five different things in five different systems.

Step 3: Turn messy text into clean, AI-ready pieces. Modern AI tools that search company knowledge, a method called Retrieval-Augmented Generation or RAG, need documents like PDFs, emails, and support tickets converted into a mathematical format the AI can search. This means cleaning raw text into a simple, consistent format, breaking long documents into standard-sized chunks (around 512 words each, with a small overlap between chunks so no idea gets cut in half), and storing those chunks in a special search-friendly database built for this purpose.

Step 4: Check data quality automatically, all the time. Manually checking data by hand cannot keep up with AI running in real time. Companies need automatic checks that reject broken or incomplete records before they ever reach the AI, that watch for data slowly drifting out of date, and that instantly flag duplicate records or missing information.

Real companies that did this

JPMorgan Chase (Data Mesh migration). JPMorgan Chase moved to a Data Mesh architecture, giving individual business teams ownership of their own data while keeping shared company-wide rules, specifically to make data easier for teams to discover and trust. JPMorgan and AWS have documented the architecture publicly; neither has published a specific before-and-after time or headcount figure, so I have not invented one here.

Netflix (metadata catalog). Netflix faced fragmented data at a massive scale, more information than almost any other company handles. It built a shared metadata catalog so different engineering teams could find and use the same data without copying files or locking each other out, an approach Netflix has documented on its own engineering blog. Netflix has not published a specific percentage improvement tied to this change, so this post does not repeat one.

Three risks to watch for

Getting data ready for AI isn’t just a technical project. It also carries real business risk if done carelessly.

• Data leakage risk. Without strict access controls, training an internal AI model on combined company files can let low-level employees accidentally see restricted information, like executive pay or confidential strategy documents. This risk deserves its own deep dive: see The Data Leakage Risk Hiding Inside Your Fine-Tuned AI Model.
• Regulatory compliance risk. Rules like the European Union AI Act require companies to clearly document where their data came from and how their AI was trained. Data pipelines nobody has mapped out invite real fines.
• Cost versus performance risk. Moving every piece of old company data into cloud storage creates high storage and data-transfer costs. It’s smarter to use a lighter virtualisation approach for old, low-priority data that rarely gets used.

A simple six-month plan

Each phase depends on the one before it. Skipping discovery to rush into AI search just means the AI confidently retrieves data nobody checked for accuracy.

Fix the foundation, then build

None of this means a company should be afraid of AI. It means the order of operations matters. Buying a powerful AI model before fixing scattered, disconnected data is like installing a smart thermostat in a house with no wiring: the technology is good, but it has nothing solid to connect to. Fixing data first is not the slow path. It is the only path that reaches the destination, and JPMorgan and Netflix both proved that fixing data first gets a company to real AI results faster.

I’ve written more on what a clean data pipeline needs to include in Stop Tuning Prompts, Start Cleaning Data, and on the full technical setup for search-ready AI systems in How to Build Governed RAG 2.0 Systems.

The AI Governance & ROI Executive Programme walks your leadership team through fixing the data foundation first, with a scored starting point and a step-by-step plan as the result. Details are on the workshops page.