AI Vendor Security Score: What Founders Miss

SOC 2 certification does not tell you whether a vendor trains on your customer prompts. ISO 27001 does not specify whether a model retains conversation data across sessions. Founders who rely on certification logos to clear their security obligations are exposed in ways the logos were never designed to cover.

The Cloud-Based AI Services privacy survey names the specific failure modes that standard cloud checklists miss: training data leakage and inference attacks. These are not theoretical. They are the categories where AI-specific systems behave differently from the cloud infrastructure those certifications were built to audit. A vendor with full AES-256 encryption and a clean SOC 2 audit can still feed your customer support conversations into a shared model. The certification does not prohibit it.

What the checklist you're using was built for

General security questionnaires were designed for systems that store and transmit data. AI tools do something different: they train on data, infer from it, and in some configurations retain it across users. NIST's Generative AI Profile, published in July 2024, identifies LLM-specific control expectations that go beyond confidentiality and availability into model lifecycle and training data governance. No standard cloud checklist covers those.

GDPR adds a second layer. The iGDPR compliance guidance is direct: when you send a prompt containing personal data to an AI tool, the legal obligation to know the processing basis sits with you, not the vendor. Your vendor's privacy policy does not transfer that obligation. You remain the data controller.

The diagnostic, scored across ten points

Score your vendor from 0 to 10 across three areas.

Data handling covers four points. Ask whether the vendor trains on your data by default, whether you hold deletion rights, whether data is stored in a jurisdiction compatible with your obligations, and whether the vendor provides a Data Processing Agreement. A vendor who trains on customer prompts without an opt-out scores zero on the first point. No DPA scores zero on the fourth. The HSCC Third-Party AI Risk Guide maps training data poisoning and model inversion to specific contractual controls — these must be confirmed in writing, not assumed from a privacy page.

Access controls cover three points. Ask whether the vendor enforces role-based permissions at the model layer, whether unstructured data is covered by those controls, and whether the vendor offers a "Do Not Train" flag at the account level. Transcend's AI governance analysis identifies real-time permission enforcement and "Do Not Train" controls as the capabilities that separate serious vendors from those who bolt on a compliance page after the fact.

Incident response covers three points. Ask for the vendor's breach notification timeline in writing, ask who holds the obligation to notify affected users, and ask whether the vendor has disclosed a prior incident and how they handled it. IBM's 2024 Cost of a Data Breach Report puts the average breach cost at $4.88 million across 604 organizations. A vendor who cannot produce a written notification timeline before you sign is telling you something about how they handle the aftermath.

When a SOC 2 badge doesn't cover the breach that actually happens

The strongest objection to this diagnostic is practical. DigitalOcean's 2023 cybersecurity survey of 554 founders found that limited time and no dedicated privacy staff are the binding constraints on security work. An evaluation requiring ML-specific interrogation of vendors will not happen if it takes two weeks. The objection is accurate about the constraint.

It does not follow that SOC 2 coverage is adequate. The constraint describes current behavior, not safe behavior. The iGDPR guidance confirms that GDPR obligations attach to the data controller at the moment of processing, not at the moment of audit. Founders who skip the data handling questions in this diagnostic remain exposed regardless of what the vendor certified.

The diagnostic above takes roughly forty minutes to run. Four questions on data handling, three on access, three on incident response. A vendor who refuses to answer any of them in writing has answered the most important question already.