Executive Summary
Fixing scattered data is step one. But the moment that unified data trains or feeds an AI model, a second, quieter risk opens up: the access controls that used to separate finance from HR from strategy documents don’t automatically carry over. A model doesn’t check anyone’s job title before it answers. Unless permissions are rebuilt at the AI layer, the same project that finally connects your data can end up handing everyone in the company a search engine for everyone else’s restricted files.
Core conclusions
- Fine-tuning and retrieval leak data through two different mechanisms, and they need two different fixes. Retrieval leaks by serving the wrong document to the wrong person; fine-tuning leaks by memorising facts directly into the model’s weights, where they can’t be selectively deleted afterward.
- Never fine-tune on data you wouldn’t be comfortable handing to every employee with access to the model. Keep genuinely restricted material in a permission-aware retrieval layer instead, where access can still be revoked.
- Access control has to move from the document to the query. Every retrieval call needs to carry the requesting user’s clearance and filter results against it, before the AI ever sees the data, not after.
Access control for AI, in ten slides
Save it, share it, or send it to whoever owns the fine-tuning pipeline.
Messy Data Is Why Most AI Projects Fail covered the first risk: scattered data that never gets fixed. This one covers the risk that shows up right after you fix it. Unifying data is the whole point of a data fabric or a data mesh, but a single connected pool of data is also, by definition, a single connected pool of everything anyone ever restricted access to. The fix for one problem quietly creates the next one, unless access control gets rebuilt at the AI layer on purpose.
At a Glance
Two Leakage Mechanisms, Two Fixes
01 — Retrieval-Time Leakage
| What happens | Where it lives | Can it be reversed? |
|---|---|---|
| Search returns a document above the requester’s clearance | The vector database / document store | Yes — fix the query filter, the leak stops |
02 — Fine-Tuning-Time Leakage
| What happens | Where it lives | Can it be reversed? |
|---|---|---|
| The model memorises facts from training examples and can surface them under the right prompt | The model’s weights themselves | No — the only real fix is retraining without that data |
03 — The Rule of Thumb
If you wouldn’t be comfortable handing a document to every employee who has access to the model, don’t fine-tune on it. Keep it in a permission-aware retrieval layer instead, where access can still be revoked after the fact.
Why unifying your data creates a second problem
The whole reason a Centralised Lakehouse, a Data Mesh, or a Data Fabric earns its keep is that it stops treating finance data, HR data, and strategy documents as separate, disconnected islands. That’s exactly what makes AI on top of it useful, and exactly what makes it dangerous. Once an AI model can search or has been trained on all of it, the walls that used to sit between departments have to be rebuilt somewhere else, or they simply stop existing. A model doesn’t check anyone’s job title before it answers a question. If nobody rebuilt the wall, the model will happily explain the executive compensation structure to an intern who asked the right way.
This isn’t a hypothetical. It’s a direct, structural consequence of combining data for AI use, and it shows up through two distinct mechanisms that get treated as one problem when they actually need two different fixes.
The first mechanism: retrieval serves the wrong document
Most enterprise AI systems today answer questions using retrieval-augmented generation (RAG): a user asks something, the system searches a document store for relevant material, and hands the matched chunks to the model as context before it answers. The leak happens at the search step. If the retrieval system doesn’t check who’s asking before it decides what to return, it will fetch whatever chunk best matches the question, restricted or not, and the model will summarise it faithfully for whoever asked.
The good news: this kind of leak is reversible. The data itself never left the document store; it was only ever exposed through a bad query. Fix the query, and the leak is closed the moment the fix ships.
The second mechanism: fine-tuning memorises the fact itself
Fine-tuning is different in a way that matters a great deal. When a model is fine-tuned on a set of documents, it doesn’t keep a searchable index of them the way a RAG system does. It adjusts its own internal weights to better predict text like what it was trained on, which means specific facts, figures, and even verbatim phrases from the training data can get absorbed directly into the model. Research on training-data extraction has repeatedly shown that large language models can be prompted to reproduce specific snippets they were fine-tuned or trained on, sometimes word for word, even when nothing in the interface looks like a search result.
That has one uncomfortable implication: you cannot “delete” a fact from a fine-tuned model the way you can revoke access to a file. Once restricted data has been baked into a model’s weights, the only reliable fix is retraining the model without that data, not patching a permission somewhere. This is why the rule has to be applied before training starts, not after a leak is discovered.
A model doesn’t check anyone’s job title before it answers. If you wouldn’t hand a document to every employee who can use the model, don’t fine-tune on it — keep it in a permission-aware retrieval layer instead, where access can still be revoked.
Moving access control from the document to the query
The practical fix is to stop thinking of access control as something that lives on a document, and start treating it as something that has to travel with every single query.
- Classify before anything enters a pipeline. This is the same data-tagging step from fixing fragmented data, but it now has a second job: deciding which tier each document belongs to. “Safe to fine-tune on” data can be baked into a model. “Retrieval-only, permission-gated” data never should be.
- Gate retrieval at query time, not display time. Every search call needs to carry the requesting user’s role or clearance and filter the candidate results against it before anything reaches the model, using row-level security or namespace/tenant isolation in the vector database. Filtering after the model has already generated an answer is too late; the model has already read the restricted material.
- Redact at ingestion, not at the screen. Salary figures, personal identifiers, and other structured sensitive fields should be masked when data is first indexed, not stripped out of the response afterward. Masking at ingestion means even a misconfigured query can’t surface the raw value.
- Log every retrieval and generation with an identity trail. Record which user asked, which chunks were retrieved, and what the model returned. This isn’t just good practice — the EU AI Act specifically requires clear data lineage and training-set documentation for higher-risk systems, and an identity-tagged retrieval log is what makes that traceable after the fact, not just in theory.
Test it like an attacker would, before one does
None of the controls above are worth much until someone actively tries to break them. Treat your own fine-tuned or RAG-connected model the way a penetration tester treats a new system: try prompt variations designed to coax out restricted information, try asking indirectly instead of directly, and try asking from an account with low privileges to confirm the gate actually holds. Do this before launch and on a recurring schedule after, because retrieval configurations and fine-tuning datasets both change over time, and a control that held six months ago isn’t guaranteed to hold today.
The order still matters
None of this replaces the work of fixing fragmented data in the first place; it comes right after it. A company that unifies its data, skips this step, and rushes straight to fine-tuning has traded one failure mode for another; the AI project might technically work, and still be the reason a confidential document ends up in the wrong hands. Fix the data. Then, before anything gets trained or connected, decide what belongs in the model and what belongs behind a gate that can still be closed.
Sources: European Union AI Act, Regulation (EU) 2024/1689; academic research on training-data extraction and memorisation in large language models.
Free tool
Fine-Tune vs. Retrieval-Only Data Classifier
Score a dataset across seven risk factors to apply the rule of thumb from this piece: safe to bake into model weights, or keep behind permission-gated retrieval.
Free tool
AI Trust, Risk & Governance Dashboard
Operationalise the access-control and audit-logging layer described above, and score your RAG access-control posture.
The AI Governance & ROI Executive Programme walks your leadership team through exactly where this fits in your own AI roadmap. Details are on the workshops page.