AI Trust, Risk & Governance Dashboard
Monitor three governance domains against industry-standard thresholds — bias and fairness metrics (80% rule, Equalized Odds), data lineage and privacy compliance (GDPR, MAS TRM), and OWASP LLM Top 10 security detection. Adjust the risk profile and regulatory framework to see how thresholds shift.
Bias & Fairness Metrics
Measured against the EEOC 80% Rule (Disparate Impact), NIST AI RMF fairness thresholds, and IBM AI Fairness 360 benchmarks. Thresholds tighten for High Risk and Critical tier applications.
Disparate Impact Ratio (80% Rule — EEOC/NIST)
Statistical Parity & Odds Differences (IBM AI Fairness 360)
Data Lineage & Privacy Compliance
Tracks consent coverage, PII classification, data residency, and retention adherence across the full training pipeline. Benchmarked against GDPR Article 5 requirements and MAS TRM data governance standards.
Regulatory Requirement Status
Security & Threat Monitoring
Mapped against the OWASP LLM Top 10 (2023) — the authoritative reference for AI-specific attack vectors. Each risk is scored by exploitability and detection coverage in your current deployment posture.
What each domain measures, why it is non-negotiable, and how to instrument it.
Bias & Fairness Metrics
The Disparate Impact Ratio (4/5 or 80% Rule) is the primary legal threshold: a protected group's selection rate below 80% of the majority group rate constitutes adverse impact under EEOC guidelines. The Statistical Parity Difference (|SPD| < 0.05 for low-risk; < 0.03 for high-risk) measures the absolute outcome rate gap. Equalized Odds requires both true positive and false positive rates to be consistent across groups — catching models that appear accurate overall but systematically fail one group.
- Enterprise demographic parity gaps average 8–14% in financial services before audit — well above the 5% acceptable threshold, yet invisible without measurement
- A model with 95% overall accuracy can simultaneously have a false negative rate 3× higher for one demographic group — accuracy alone is not a fairness proxy
- The EU AI Act (Article 10) and Singapore IMDA AIGF require documented fairness testing for high-risk applications. Non-documentation is a compliance finding independent of whether bias exists
- Fairness failures compound over time: a biased credit model denying loans creates economic conditions that make future loan applications from that group look even riskier to the model
- Define protected attributes pre-deployment for your jurisdiction — Singapore: race, gender, age, religion, disability; EU: adds nationality and genetic data
- Run IBM AI Fairness 360 or Aequitas on a holdout test set before go-live — target DIR ≥ 0.80 for all groups; ≥ 0.85 for high-risk applications
- Implement continuous monitoring on production outputs: sample 500+ predictions/group/week, compute DIR weekly, alert when it drops below 0.80
- Commission an independent bias audit annually — the auditor must not have been involved in model development (Singapore IMDA AIGF requirement for high-risk AI)
Data Lineage & Privacy
Data lineage documents the full journey of every training data point — from origin source through collection, processing, training, and inference — with consent status, PII classification, and retention expiry at each stage. The IAPP 2023 survey found GDPR-compliant enterprises average 76–88% consent coverage; best-in-class exceeds 95%. Only 61% of enterprises have complete PII classification across training datasets (Gartner 2023).
- GDPR Article 22 and Singapore PDPA require organisations to explain automated decisions — this requires traceable lineage from training data to inference output
- The right to erasure ("right to be forgotten") requires knowing exactly which training samples contain an individual's data — without lineage, PDPA/GDPR compliance is structurally impossible
- Undocumented data sources are the #1 finding in AI regulatory audits and carry the highest penalty risk under both GDPR and Singapore PDPA 2022 amendments
- Data poisoning attacks — deliberate corruption of training data — are undetectable without provenance documentation of every training sample's origin and handling chain
- Implement a data catalogue before training begins — retroactive lineage is always incomplete and is rejected by regulators under GDPR Article 30 (records of processing)
- Tag every source with: origin system, legal basis for collection, consent status, PII classification level (Public/Internal/Confidential/Restricted), data residency, retention expiry
- Run automated PII detection on training data using Microsoft Presidio or AWS Comprehend — manual review cannot scale beyond 10,000 records reliably
- Set a consent withdrawal monitor: alert when >5% of training records have expired consent or active withdrawal requests — this triggers a retraining decision under MAS TRM
OWASP LLM Security
The OWASP LLM Top 10 (2023) is the authoritative reference for AI-specific attack vectors. LLM01 Prompt Injection is ranked #1 — approximately 72% of tested LLM deployments show at least one exploitable injection vector (HiddenLayer 2024). LLM06 Sensitive Information Disclosure affects ~45% of enterprise RAG deployments. Only 23% of organisations have dedicated AI security monitoring (Gartner 2023).
- Prompt injection is the SQL injection of the AI era — successful injection can bypass safety controls, exfiltrate system prompts, or use the LLM as an attack relay against connected systems
- Model extraction attacks reconstruct your proprietary model through systematic output querying — representing intellectual property theft with no system breach and no standard security alert
- Traditional WAF, SIEM, and rate-limiting rules do not detect AI-specific attacks. A model extraction campaign can execute through traffic that appears completely normal to conventional security tools
- MAS TRM 2021 and Singapore Cybersecurity Act 2018 require documented threat monitoring for AI systems deployed in regulated financial services and critical information infrastructure
- Instrument every inference request with an AI-native security layer — keyword matching alone misses ~80% of adversarial prompt injection attempts; use semantic similarity against known attack embeddings
- Monitor output entropy and confidence distribution in real time: sudden high-confidence outputs on semantically unusual queries are the primary signal of active injection or extraction
- Implement query velocity analysis: flag sources exceeding 200 unique queries/hour for review — extraction attacks require high query volume that standard rate limits typically don't catch
- Run red team exercises quarterly using OWASP LLM01–LLM10 as the test plan — and separately test each integration point where the LLM connects to external systems or databases
Get the AI Trust & Governance Implementation Checklist
The 52-point governance checklist — bias audit protocol with DIR/SPD/EOD thresholds, data lineage documentation template, OWASP LLM security response playbook, and a Singapore IMDA AIGF / MAS TRM / GDPR compliance crosswalk.
- Bias audit protocol: protected attribute selection, threshold configuration, sampling requirements
- Data lineage template: 14-field mandatory documentation per training data source
- OWASP LLM01–LLM10 detection and response playbook for enterprise deployments
- IMDA AIGF · MAS TRM · GDPR · NIST AI RMF requirement crosswalk matrix
No spam. Unsubscribe any time.