AI vendor due diligence: a checklist for vendors that use AI on your data

An AI vendor is any vendor that puts your data through a model. That covers the companies selling AI products and, more often, the tools you already use that have added an assistant, a summary button or a smarter search. Your existing due diligence still applies to them. What changes is where your data goes, how long it stays there, and who is responsible when the output is wrong.

This guide covers the questions that are specific to AI: training on customer data, retention and logging, the model providers behind the vendor, access controls, output accuracy, intellectual property, security testing and incident handling. It ends with a checklist you can paste into your questionnaire.

What is different about AI vendors

A conventional SaaS vendor stores and processes your data to deliver a defined function. An AI feature adds three things that a standard security review does not always catch:

• A new data flow. Prompts, uploaded files and retrieved documents are often sent to a model provider the vendor does not operate. That provider is a fourth party to you.
• A new kind of record. Prompts and outputs may be logged for debugging, abuse monitoring or product improvement, sometimes under a different retention period from the rest of your data.
• A new failure mode. Output can be confidently wrong, can leak content from another source, or can be steered by instructions hidden in the content it reads.

Tier the vendor first, as you would any other (see Vendor risk tiering: how to classify third parties and set a review cadence). An AI feature that drafts marketing copy from public information is not the same risk as one that reads customer contracts or helps decide who gets a loan.

Training on your data, and how to opt out

The first question is whether your data, or anything derived from it, is used to train or improve a model. Ask it about both the vendor and each model provider, because the answers can differ. Get specifics:

• Is customer data used for training, fine-tuning, evaluation or human review?
• Is that off by default, on by default with an opt-out, or not configurable? Who at your company can change the setting, and is the change logged?
• Does the commitment sit in the contract or data processing agreement, or only in a help center article the vendor can edit?
• If you opt out later, what happens to data already used? A model that was trained on it cannot usually be made to forget it, so the default matters more than the switch.

Retention, logging and model sub-processors

Ask the vendor to describe the full path of a prompt: where it is sent, what is stored, where, and for how long. The answer should name every model provider involved and treat each one as a sub-processor, with the same notice and objection rights you have for the vendor’s hosting provider.

Location matters too. If personal data is in scope, ask where each provider processes it and which transfer mechanism applies. The same processor questions from GDPR processor due diligence: vetting vendors that handle personal data apply to model providers.

Access controls around AI features

An assistant that can search everything in a workspace can also surface everything in a workspace. Ask whether the AI feature respects the permissions of the user asking, or runs with broader service access. Ask whether admins can turn the feature off per team or per data type, and whether the vendor’s own staff can see prompts and outputs (and under what approval and logging). If the feature connects to other systems on your behalf, ask what scopes it requests and whether they can be narrowed.

Output accuracy and human review

No vendor can promise correct output, and a vendor that does should worry you. What you can ask is how they measure and limit error, and how they expect you to use the result.

• How do they test output quality before release and after model changes?
• Do outputs cite their sources so a reviewer can check them?
• How much notice do customers get when the underlying model changes?
• Where the output feeds a decision about a person (hiring, credit, housing, insurance), what testing for bias has been done, and what documentation will they share?

On your side, decide which outputs a person must review before they are used. That is a control you own, and it belongs in your records next to the vendor assessment.

Intellectual property and indemnity

Read the contract for three points. Who owns the outputs, and does the vendor claim any license to your inputs beyond delivering the service? Does the vendor offer an indemnity for third-party claims that the output infringes someone’s rights, and what conditions attach to it (often that you used the product as documented and kept its filters on)? And does the limitation of liability quietly cap that indemnity at a figure too small to matter? These terms vary widely, so have counsel read them for any vendor whose output you will publish or ship.

Security testing: prompt injection and friends

Prompt injection is the AI-specific attack to ask about first: instructions hidden in an email, web page or document that the model reads and then follows. It can make an assistant leak data or take an action the user never asked for. The OWASP Top 10 for LLM Applications is a reasonable reference list for the rest.

Ask whether the vendor’s penetration tests or red-team exercises covered the AI features specifically, and ask for the summary. Ask what the model is allowed to do without a human confirming it (send, delete, pay, share), and how untrusted content is separated from instructions. A vendor that has thought about this will answer in specifics; a vendor that has not will talk about the model provider’s safety features.

Incident handling

Your standard incident clause should cover AI events explicitly. Ask whether the vendor treats these as incidents that trigger notice to you: exposure of prompts or outputs, a model provider breach affecting your data, a successful prompt injection, or output that disclosed another customer’s information. Ask how quickly they can disable the AI feature for your account if you need it off, and whether they will tell you when a provider incident affects them.

Frameworks and laws worth knowing

• NIST AI RMF 1.0 (January 2023) is voluntary guidance built on four functions: Govern, Map, Measure and Manage. NIST has also published a Generative AI Profile (NIST AI 600-1). A vendor citing it tells you their vocabulary, not that anyone has checked their work.
• ISO/IEC 42001:2023 specifies an AI management system and can be certified by an accredited body. As with ISO/IEC 27001, ask for the certificate and check that the scope includes the product you are buying (see ISO 27001 supplier relationships: meeting Annex A 5.19–5.23 with evidence).
• State and local laws are developing. Colorado has passed a law aimed at high-risk AI systems used in consequential decisions, though its effective date has moved and further changes are possible, and New York City regulates certain automated hiring tools. If a vendor’s AI contributes to decisions about people, check the current rules with counsel rather than relying on the vendor’s summary.

AI vendor due diligence checklist: questions to send

Add these to the questionnaire for any vendor whose service puts your data through a model. Ask for evidence (a contract clause, a policy excerpt, a test summary) wherever a yes is possible.

• Which AI features in the service process our data, and are any enabled by default?
• Is our data, or data derived from it, used to train, fine-tune or evaluate any model, by you or by a provider? Where is that commitment in the contract?
• If training use is configurable, what is the default, who can change it, and is the change logged?
• List every model provider involved, what data each receives, and where each processes it. Are they on your sub-processor list with advance notice of changes?
• How long are prompts, outputs, logs and embeddings retained by you and by each provider, and are they deleted at contract end?
• Can humans at your company or at a provider read our prompts or outputs? Under what approval, and is access logged?
• Does the AI feature respect each user's existing permissions? Can admins disable it per team or data type?
• How do you test output quality, and how much notice do we get before the underlying model changes?
• If the output informs decisions about individuals, what bias testing has been done, and what can you share?
• Who owns outputs, and do you offer an IP indemnity for them? What conditions and liability caps apply?
• Did your last penetration test or red-team exercise cover the AI features, including prompt injection? Attach the summary.
• What actions can the AI take without a user confirming them?
• Do you treat exposure of prompts or outputs, provider breaches and successful prompt injection as notifiable incidents? Within what time?
• Do you align with NIST AI RMF or hold ISO/IEC 42001 certification? If certified, attach the certificate and scope.

Score the answers the same way as the rest of the questionnaire, with weights fixed in advance. The method in Vendor risk assessment questionnaire: what to ask, and how to score the answers works unchanged; training use by default and no incident commitment are sensible knock-outs for a vendor handling confidential data.

Where to go next

If you want to run this without spreadsheets, VendorVett lets you send the questionnaire to the vendor through a private link and scores the answers deterministically against weights you set. AI can summarize the evidence a vendor uploads, a person reviews it, and the score is never set by a model. For the wider process, start with Vendor due diligence: a practical guide for small and mid-sized businesses, and for what to collect when a new vendor signs, see Vendor onboarding checklist: the checks to finish before the contract is signed.