An AI Ethics RFP That Isn’t Just a Checkbox

An AI ethics RFP that isn’t just a checkbox.

“Do you have an AI ethics policy?” is a question every vendor answers yes to, because it costs them nothing and proves nothing. A question that actually tests something is harder to write and much harder for a vendor to fake.

Most AI procurement questionnaires share a structural flaw: they ask a vendor to confirm a fact about itself, with no independent way to check the answer, and no consequence attached to answering generously. “Do you test for bias?” “Do you have a responsible AI framework?” “Is your model explainable?” Every reasonable vendor says yes to all three, the questionnaire gets filed as compliance evidence, and nothing about the actual risk of the system has been established either way.

What a self-report question can never do

The problem is not that vendors lie, most do not need to; the questions themselves rarely ask for anything a lie would be required to cover. “Do you test for bias” is satisfiable by having run any bias test at all, on any dataset, at any point, regardless of whether it covered the specific population the buying organisation actually serves. The question was written to be answerable, not to be informative, and a vendor answering honestly and a vendor answering evasively produce the identical word on the form.

What a question that actually tests something looks like

The shift that makes a question useful is asking for evidence rather than assurance, and asking for it specific to the buyer’s own context rather than the vendor’s general practice.  

“Do you test for bias” becomes “provide the most recent bias evaluation you ran, including which protected characteristics were tested and against what population.”   “Is your model explainable” becomes “describe, with an example, how a person affected by an adverse decision from this system could contest it, and who at your organisation is accountable for that appeal.”   “Do you have a responsible AI framework” becomes “name the person who can pause a deployment on ethics grounds, and describe the last time that authority was actually used.”  

A vendor with genuine practice behind these questions can answer them specifically, with names, dates, and documents. A vendor without it produces something noticeably vaguer, without ever having to say no to anything, which is exactly the signal a checkbox question can never surface! The same logic applies to contractual follow-through, not just the questionnaire. A specific answer at RFP stage is worth little if the contract itself does not obligate the vendor to notify the buyer of material changes to the model, provide incident data on request, or support an audit if evidence of harm emerges later. The questionnaire establishes what was promised; the contract is what determines whether that promise survives contact with a difficult situation eighteen months in.

Who actually has to do this

Writing procurement questions this specifically is not, on its own, a technical skill. It is a governance skill: knowing what evidence would actually change the buying decision, and being willing to ask for it even when it slows the buying process down. That is squarely inside the ethicist’s job, alongside procurement and legal.

Written by us at Ethics Directive. If anything here needs correcting, we will say so in the open, dated.