Free Tool · Version 2.0

AI Vendor Evaluation Scorecard.

Stop evaluating vendors by demo quality. Rate your two shortlisted vendors across 9 due-diligence dimensions and see which one wins, and where each carries hidden risk.

Listen to this briefing

Vetting AI Vendors Beyond the Demo

0:00
VS

0 of 18 ratings entered

Dimension
Vendor A
Vendor B
Integration FitHow well does it integrate with your existing systems and workflows?1 = requires full rebuild · 5 = plug-and-play
Weight in your decision:
Vendor A
Vendor B
Data GovernanceVendor's data handling, compliance posture, and privacy protections.1 = poor / opaque · 5 = excellent / transparent
Weight in your decision:
Vendor A
Vendor B
Total Cost of OwnershipAll-in cost over 3 years: licensing, implementation, support, and maintenance.1 = very expensive · 5 = most affordable
Weight in your decision:
Vendor A
Vendor B
PortabilityEase of exiting the vendor and migrating data and workflows elsewhere.1 = deeply locked-in · 5 = fully portable
Weight in your decision:
Vendor A
Vendor B
Support QualityQuality of implementation support, SLAs, and ongoing customer success.1 = poor · 5 = excellent
Weight in your decision:
Vendor A
Vendor B
Time to DeployHow quickly can you realistically go live and start seeing value?1 = 12+ months · 5 = weeks
Weight in your decision:
Vendor A
Vendor B
Proof of ValueStrength of references, case studies, and documented results in similar contexts.1 = no references · 5 = strong peer evidence
Weight in your decision:
Vendor A
Vendor B
Tool-Ecosystem SecurityHow the vendor secures agent-to-tool connections (MCP servers, plugins, third-party integrations) it ships or supports.1 = unauthenticated / unaudited connectors · 5 = authenticated, scoped, and audited
Weight in your decision:
Vendor A
Vendor B
Pricing Model RiskHow predictable the vendor's pricing is at scale: token or credit consumption versus fixed or outcome-based pricing.1 = opaque token/credit burn · 5 = predictable, outcome-tied pricing
Weight in your decision:
Vendor A
Vendor B
Live Comparison

Vendor A

–

out of 100

VS

Vendor B

–

out of 100

This is a self-assessment built from the ratings you enter, not independent testing or verification of either vendor. It is not procurement, contractual, or legal advice, and the recommended steps above are a starting point for your own due diligence, not a substitute for it. Your ratings stay in your browser and are not sent to Terence Kok or reviewed by anyone.

Opens a 6-page report with your ratings, recommended next steps, and calculated score. Print or save it as a PDF from there.

Why These 9 Dimensions

The criteria that separate a good vendor choice from a costly mistake.

Integration, TCO & Speed

Why it matters
  • Integration Fit is the hidden multiplier on every other cost. A technically capable product that requires 12 months of integration work to deploy is not a low-cost option.
  • TCO is often understated in vendor proposals. As a directional pattern, not a measured statistic, licensing commonly runs 40–60% of all-in cost. The rest is implementation labour, training, maintenance, and governance overhead.
  • Time to Deploy matters because value not yet realised is value lost. A 6-month deployment difference has compounding ROI consequences.

Governance, Portability & Risk

Why it matters
  • Data Governance scores below 3 are a legal and reputational risk. PDPA, GDPR, and sector-specific regulations make vendor data practices a board-level concern.
  • Portability scores below 3 mean you are signing a long-term contract even if the paper says 12 months. Data lock-in, API dependencies, and proprietary formats all compound over time.

Support & Proof of Value

Why it matters
  • Support Quality is where most vendor relationships fail post-signature. The account executive who closed the deal is rarely the person handling your production incidents at 2am.
  • Proof of Value at score 1–2 means you are being asked to be a reference customer, not joining one. Ask for 3 references from organisations similar to yours before scoring above 3.

Tool-Ecosystem Security & Pricing Risk

Why it matters
  • Tool-Ecosystem Security covers how the vendor secures the MCP servers, plugins, and third-party connectors its agents call. Unauthenticated MCP implementations produced real, documented vulnerabilities in 2026: ask specifically how tool connections are authenticated and scoped.
  • Pricing Model Risk separates predictable pricing from token or credit-based models where the same usage pattern can produce wildly different invoices month to month. Ask for a worst-case cost scenario at your expected volume.
Terence Kok
Before You Go

Vendor demos are designed to hide exactly the things this scorecard forces into the open, total cost of ownership, portability, what support looks like once the sales rep is gone. Rate two vendors side by side across seven dimensions and you'll usually find the gap isn't where you expected. I added the risk flags after seeing too many contracts signed on a strong demo and a weak governance answer nobody pushed back on. Use it before the meeting where you're expected to have a recommendation.

Terence Kok