- Standard vendor forms were written for software that stores your data. AI tools also read it, learn from it and act on it, which are three questions the form never asks.
- Turn due diligence into an evidence request. For each question, name the document that answers it rather than accepting a yes.
- GDPR Article 28 already gives you audit rights, sub-processor notice and deletion at the end of service. Most buyers never use them.
- A certificate proves a scope, not a behaviour. Read what the scope statement covers before treating it as an answer.
- The failure nobody drafts for is a silent model swap. Pin the version, or accept that your tested output can change on a Tuesday.
The contract is on the second screen and the trial ends Friday. Somebody has sent you a security questionnaire, forty rows of it, and the vendor has filled it in without hesitating: data hosted in Frankfurt, encrypted at rest, single sign on available, penetration test done in March. Every answer is true. None of them tell you whether the tool you are about to feed a decade of customer emails into will use them to improve a model that other people also pay for.
That gap is not the vendor being cagey. It is a form built for a different category of product. Software that stores your data raises storage questions. Software that reads your data, generates text on the strength of it and increasingly acts on your behalf raises questions the form has no rows for.
Why does a normal security questionnaire miss this?
Because it was written to assess a database, and you are buying a judgement. A conventional questionnaire establishes where information sits, who can reach it and what happens when somebody breaks in. Those remain necessary. They stop short of the three things that decide whether an AI purchase goes wrong.
The first is learning. Your text may improve a model that serves other customers, and the default position differs by vendor, by plan tier and sometimes by whether you clicked something during onboarding. The second is provenance. The tool you are buying is often a thin layer over somebody else's model, so the company you signed with is not the company holding your prompts. The third is autonomy. A tool that drafts a reply is a different risk from a tool that sends it.
The nine questions, and the document that answers each
A question a vendor can answer with a word is a question you have not asked, and the one most buyers never think to ask at all is which model sits underneath and what happens at its published model deprecation date. Each row below names the artefact you should be sent, because an artefact is checkable and a reassurance is not.
| # | Question | Ask for this document | Who inside the vendor owns it |
|---|---|---|---|
| 1 | Do you train on my content, ever, on any plan? | The clause in the terms, quoted with its section number | Legal |
| 2 | Whose model runs underneath, and where? | The sub-processor list with regions | Legal or security |
| 3 | How long are prompts and outputs kept? | The retention schedule, per data type | Security |
| 4 | What did you measure before selling this? | Evaluation results with the date and the test set | Product or research |
| 5 | What is the tool documented not to do? | A model card or system card with limitations | Product |
| 6 | Can it act without a human, and where is that switch? | The permissions matrix and default settings | Product |
| 7 | Who is told when it goes wrong, and how fast? | The incident notification term with hours in it | Legal |
| 8 | Can the model change under me without notice? | The version and deprecation policy | Product |
| 9 | What do I take with me when I leave? | Export format plus the deletion commitment | Legal |
Nine is not a magic number. It is the count at which the questions stop being about this vendor and start being about every vendor, which is the point where a small team can reuse the list rather than rebuild it each time.
What does GDPR already give you before you negotiate?
More than most buyers use. If the tool touches personal data and you decide why it is processed, you are the controller and the vendor is your processor, which triggers the contract requirements set out in Article 28 of the GDPR. Three of them are worth reading before your next call.
The processor may not bring in another processor without your written authorisation, and where you granted a general authorisation it has to tell you about additions or replacements so you can object. That is your sub-processor question, already answered by law. The processor must make available all information necessary to demonstrate compliance and submit to audits, including inspections, conducted by you. That is your evidence request, already a right. At the end of the service the processor deletes or returns the personal data, at your choice. That is your exit question, already covered.
The practical move is to stop asking whether the vendor is compliant, which invites a yes, and start exercising the specific right. Ask for the current sub-processor list and the notice mechanism for changes. A vendor whose list is a page on their site with a subscribe link is in better shape than one who has to go and write it for you.
Article 28 has a sting for the vendor too. A processor that decides the purposes or means of processing becomes a controller in its own right. If a tool quietly reuses your customer text for its own product development, that is not a footnote in a policy, it is a change of legal role.
Who is the provider here, and who is the deployer?
The vendor is usually the provider and you are usually the deployer, and the duties are not symmetrical. Under Article 26 of the EU AI Act, deployers of high risk systems carry obligations no supplier contract removes. You have to use the system according to the instructions that came with it. You have to assign human oversight to somebody with the competence and the authority to intervene, not merely somebody whose name is on a rota.
Two further duties surprise buyers. Where you control the input data, you must ensure it is relevant and sufficiently representative for the intended purpose, so feeding a screening tool five years of skewed history is your problem rather than the vendor's. And automatically generated logs have to be retained for a period appropriate to the purpose, of at least six months unless other law says otherwise. If the vendor's default log retention is thirty days on your plan, you have a compliance gap that no amount of vendor assurance covers.
Most tools a small shop buys are not high risk in the Act's sense. Recruitment and credit related uses can be, which is why we treated AI resume screening as a compliance purchase rather than a time saver. The reason to read Article 26 anyway is that it tells you what a careful buyer is expected to be able to show, in any jurisdiction, when something goes wrong.
What does the vendor's certificate actually prove?
That a defined scope was assessed, on defined dates, against defined criteria. Nothing outside that scope is covered, and the model is very often outside it.
Read the scope statement, which is one paragraph and usually the only part of a certificate worth your time. A security certification covering the hosting platform and the customer portal is genuine and useful, and it says nothing about how the model behaves, what it was trained on or whether its outputs were evaluated. An information security certificate answers the question your old questionnaire was asking. It does not answer questions one, four, five or eight in the table above.
The same caution applies to framework names dropped into sales decks. The NIST AI RMF Playbook states plainly that the framework and the Playbook are intended for voluntary use, with organisations borrowing as many or as few of the suggested actions as apply to them. A vendor saying it aligns to the framework is telling you it chose some suggestions. That is not a lie and it is not a certification. What the framework is genuinely good for is vocabulary: its four functions give two companies a shared way to describe governance, mapping, measurement and management, so that three vendor answers become comparable rather than three sales narratives.
How do you ask about model quality without becoming a researcher?
Ask what was measured, on what, and when. The format for that answer has existed since 2018. The model cards paper by Mitchell and colleagues proposed short documents accompanying a trained model that report benchmarked evaluation across different conditions and groups, disclose the context the model is intended for, and describe the evaluation procedure. A vendor that can hand you one has done the work. A vendor that cannot may still be fine, in which case the fallback question is narrower and harder to dodge: show me the evaluation you ran on the task I am buying this for, the examples it failed, and the date.
The date matters more than the score. An evaluation from before the current model version is a historical document. If the answer to question eight is that models can change without notice, then every evaluation you were shown has an unstated expiry, and you should treat the numbers as marketing rather than evidence.
The layer under the layer
Most tools a small business buys are an interface, a prompt and a database sitting on top of a foundation model somebody else operates. Good ai due diligence follows that stack down, because three of your nine answers live at the bottom of it rather than with the company sending the invoice.
Training on customer data is the first. The ai vendor may promise it does not train on your content while the model provider underneath applies its own default, and those defaults live in a model licence rather than in your contract, which is why open weights and open source AI are worth telling apart. The two positions only line up if the vendor bought the right plan and configured it. Ask which, and ask for the clause from both layers. Prompt logging is the second: many providers retain inputs for a short abuse monitoring window even when training is off, so the honest answer to your retention period question is often two numbers rather than one, the vendor's and the provider's, and the published provider windows run from 30 to 55 days before any exception applies.
The third is your data processing agreement, which has to reach all the way down or it protects nothing. If the model provider is not named as a sub-processor, either it is not being used or the paperwork is incomplete, and both answers are worth having before you sign.
Write the exit plan while you are still keen on the product. Export format, whether the export includes the conversation history rather than only the records, who deletes what, and how you would prove deletion happened. Six months into a migration nobody negotiates this well.
What happens when the model changes under you?
Your tested behaviour changes with it, and nothing in a standard contract stops that. This is the failure mode that gets least attention and produces the most support tickets. A tool that has classified your enquiries correctly for four months starts routing refund requests into the wrong queue, nobody deployed anything, and the vendor's status page is green because the service is up. What changed was the model behind it.
Two clauses fix most of the exposure. The first pins a version: you are served a named model version, and a change requires notice with a window in which you can test. The second defines deprecation: how long an old version stays available after a new one ships. Neither is exotic, and a vendor building for business customers will usually have both written down already. The ones who do not will tell you the change is an improvement, which may be true and is not the point. You bought a behaviour.
Keep a small regression set of your own for this. Twenty real inputs with the answers you consider correct, run monthly, kept in a spreadsheet. It costs an hour to build and it is the only instrument you will own that detects a silent change on your own workload rather than on a public benchmark.
What does a refusal to answer tell you?
Usually not malice. Usually that nobody at the vendor owns the answer, which is its own risk signal, and a fairly precise one.
A vendor who cannot produce a retention schedule is telling you retention is not managed. A vendor who cannot name the model provider is telling you the contract chain is longer than they have mapped. A vendor who answers question one with a link to a general privacy policy rather than a quoted clause is telling you the clause does not say what the salesperson said it says. Ask for the section number and read it yourself. This takes four minutes and settles more arguments than any questionnaire round.
Where the answers come back thin and you still want the tool, buy it smaller. Run it on a bounded dataset, with logging you control, on a contract term short enough that being wrong costs a quarter rather than three years. That is the same logic that makes an inventory the first governance artefact rather than the last, which we argued in the case for building an inventory before a policy.
Turning this into something a small team runs in an afternoon
Copy the nine rows into a document. Add three columns: the answer, the artefact received, the date. Send it before the demo rather than after, because a vendor answering in writing before they have your enthusiasm answers differently.
Set a bar in advance for what a pass looks like. A workable bar for a small business: questions one, two, three and nine answered with documents, question seven answered with a number of hours, and the rest answered honestly even where the honest answer is no. Vendors who fail on the first four are selling you a category of risk you cannot price. Vendors who fail on the others are usually just young.
Then write down what you decided and why. Not for an auditor, for the version of you in eighteen months who has to remember whether the training question was answered by the vendor or assumed by a colleague. Pair it with the classification you already use for internal data, the one that decides what staff may paste where, which we set out in a one page AI policy built on data classes. The purchase decision and the usage rule are the same decision seen from two ends, and keeping them in one place is how a small team avoids owning two contradictory documents.
None of this makes a bad tool good. It makes the bad tool visible before the migration, the training and the customer emails, which is the only point at which the information is worth anything. If you want the shape of the controls that sit behind these answers on our own side, we publish them on our security page.
The one question to ask if you only ask one
Where does my text go, and who else can learn from it? Every other question on the list is a refinement of that one. It is also the question a vendor either answers in a sentence with a clause number attached, or does not answer at all, and the difference between those two responses tells you more about the company than the next forty rows of any form.