Free Tool

Fine-tune it, or gate it?

Seven inputs covering sensitivity, revocability, regulation, and the value this data adds. A weighted score returns a clear call, with the risk flags a "yes" answer still needs.

Listen to this briefing

Should You Fine-Tune, or Use Retrieval?

0:00

Your Dataset

3

How restricted is access to this data today, before any AI touches it?

Fully publicExec / board-only
3

If this data needed to be deleted or revoked later, how important is that?

Doesn't matterMust stay revocable
3

Is this data subject to specific retention, deletion, or processing rules?

Not regulatedHeavily regulated
3

How much would training directly on this data improve the model's usefulness?

Marginal valueCore to the model
3

How much personal or identifiable information does this data contain?

NoneExtensive
3

How often does this data change, get corrected, or need updating?

Static, rarely changesFrequently updated
3

If this surfaced to the wrong person, how severe would the harm be?

Minor / no real harmSevere: legal, competitive, safety

Your Call

RETRIEVAL-ONLY

Keep It in a Permission-Gated Retrieval Layer

This dataset carries enough risk across sensitivity, revocability, or regulation that baking it into a model's weights is not worth it. Serve it through access-controlled retrieval instead, where a bad query can still be fixed.

Retrieval-OnlyRAG + Targeted Fine-TuneFine-Tune Safe
50/100 fine-tune suitability

Key factors driving this call

  • Balanced inputs across sensitivity and value keep this a close call either way

Recommended next steps

    The threshold for Fine-Tune Safe is set at 70/100, not 50. Fine-tuning is irreversible, so the bar to clear is deliberately higher than a coin flip. Scores from 50 to 69 land in a hybrid band: rather than force a binary call on a borderline dataset, this tool recommends keeping the fact-bearing core in gated retrieval and fine-tuning separately, on de-identified data, for stable behaviours only. Below 50, this tool defaults to gated retrieval alone.

    This is a self-assessment built from the inputs you enter, not a legal or compliance determination about this specific dataset. It is not legal advice, and the recommended steps above are a starting point for your own data governance review, not a substitute for it. Your inputs stay in your browser and are not sent to Terence Kok or reviewed by anyone.

    Opens a 5-page report with your inputs, recommended next steps, and calculated score. Print or save it as a PDF from there.

    Fine-Tune Safe

    When it applies

    The data adds real value to the model, and you would be comfortable handing it to every employee who has access to the model. Low sensitivity, no meaningful revocability need, and nothing that a regulator would ask you to trace or erase later.

    What to still do
    • De-identify any residual PII before training, even at low density
    • Document the training set for EU AI Act traceability requirements, and confirm the process satisfies GDPR's right-to-erasure obligations for any personal data involved
    • Re-check this classification if the dataset's contents change materially

    Retrieval-Only, Gated

    When it applies

    The data is sensitive, regulated, changes often, or you need the ability to revoke access later. A model can't selectively forget what it memorised during training, so this data stays in a document store where every query is filtered against the requester's clearance before anything reaches the model.

    What to still do
    • Gate at query time with row-level security, not after the model has already generated an answer
    • Redact structured sensitive fields at ingestion, not at the screen
    • Log every retrieval with an identity trail for audit and incident response

    RAG + Targeted Fine-Tune

    When it applies

    The score is borderline, not clearly safe to bake in, not clearly sensitive enough to rule out fine-tuning entirely. Current guidance treats this as a false binary rather than a coin flip: keep the fact-bearing or sensitive core in gated retrieval, and fine-tune separately, on de-identified or synthetic data, for stable behaviours like output format, tone, or refusal patterns.

    What to still do
    • Never fine-tune the behaviour layer on the raw sensitive or fact-bearing data, use a de-identified or synthetic set built for that purpose
    • Test the fine-tuned layer with extraction prompts before deployment to confirm it doesn't leak facts from the gated retrieval layer
    • Classify the retrieval corpus and the fine-tune set separately at the next review; they often land on different sides of the threshold

    If It's Still Not Comfortable

    When it applies

    A small category of data shouldn't touch an AI system at all, gated or not: material under legal hold, privileged legal communications, or data with an explicit contractual or regulatory bar on automated processing. This tool scores fine-tune vs. retrieval; it doesn't clear that third question.

    What to still do
    • Route anything scoring high on both Regulatory Obligation and Blast Radius to legal review before it touches any AI system
    • Treat "gated retrieval" and "not connected to AI at all" as different answers
    • Re-run this check after a legal or compliance review changes the facts
    Terence Kok
    Before You Go

    I built this after one too many meetings where someone asked, can we just fine-tune on this data, and nobody had a clean way to answer. The full fine-tune-safe threshold sits at 70 out of 100, not 50, because fine-tuning is permanent: once a fact is baked into a model's weights, you cannot selectively delete it later. Scores between 50 and 69 now get a third answer instead of a forced coin flip: keep the sensitive core in gated retrieval, and fine-tune separately, on de-identified data, for the parts of the job that are about behaviour instead of facts. If your dataset lands in that band, treat the discomfort as useful information, not a bug in the tool.

    Terence Kok