Most public bodies do not build AI themselves — they buy it. Procurement is therefore one of the most powerful levers for responsible public-sector AI. A well-written contract can secure transparency, auditability and accountability; a poor one can lock an agency into an opaque system it cannot explain to citizens.

Step 1 — Start with the problem

Define the public need before choosing technology:

  • What problem are you solving, for whom?
  • What does success look like, and how is it measured today?
  • Would a simpler, non-AI solution work?
  • What decisions will the system influence, and how significant are they for people?

Step 2 — Assess the risk

Classify the use by impact. A chatbot answering opening-hours questions is low risk; a system scoring benefit applications is high risk. Consider:

  • effects on rights, benefits, liberty or access to services;
  • vulnerable groups affected;
  • legal requirements (data protection, equality law, and in the EU, the AI Act’s high-risk rules);
  • reversibility of errors.

Frameworks such as the NIST AI Risk Management Framework provide a structured approach.

Step 3 — Engage stakeholders

Involve frontline staff, legal and data protection officers, and — for significant uses — representatives of the people affected. They identify risks that technical teams miss.

Step 4 — Write requirements and contract clauses

Key clauses for AI contracts:

  • Performance and testing: agreed metrics, including by subgroup, and acceptance testing on local data.
  • Documentation: model purpose, training data sources, known limitations.
  • Audit rights: access for the agency and independent auditors to test the system.
  • Data: ownership, location, retention, and prohibition on using public data to train vendor models without permission.
  • Transparency: enough information to explain decisions to citizens and publish in algorithm registers.
  • Change control: notification and re-testing when models are updated.
  • Incident reporting: obligations to report errors and security issues.
  • Exit: data portability and transition support to avoid lock-in.

Step 5 — Pilot and test

Run pilots with real but protected data, ideally in a “shadow mode” where the system’s outputs are compared with human decisions before affecting anyone.

Step 6 — Be transparent

Publish information about the system. The UK’s Algorithmic Transparency Recording Standard, for example, gives public bodies a template to describe algorithmic tools, their purpose, how they work and the human oversight involved.

Step 7 — Monitor for the whole lifecycle

Assign an accountable owner, track performance and complaints, review fairness regularly and be ready to suspend the system if problems appear.

Buying generative AI tools for staff

Many public bodies are now buying generative AI assistants for civil servants rather than specialized decision systems. Specific questions for these purchases:

  • Data handling: are prompts and documents excluded from model training? Where is data processed and stored?
  • Classification: which categories of information may staff enter, and how is that enforced?
  • Accuracy safeguards: does the tool cite sources from internal documents, and are staff trained to verify outputs?
  • Records management: are AI-generated drafts subject to freedom of information and archiving rules?
  • Accessibility: does the tool meet accessibility standards for all staff?
  • Usage policy: clear guidance on acceptable uses — drafting, summarizing, translation — and prohibited ones, such as making decisions about individuals without review.

Common procurement mistakes

  1. Buying a product before defining the problem.
  2. Accepting vendor accuracy claims without local testing.
  3. Signing contracts without audit rights or data protection clauses.
  4. Ignoring the total cost, including integration, training and monitoring.
  5. Forgetting an exit plan, leaving the agency locked in.

What to watch next

  • Standard AI contract clauses developed by governments.
  • Mandatory algorithm registers for public bodies.
  • Public sector requirements under the EU AI Act.
  • Shared evaluation and testing services for government AI.

Key takeaways

  • Procurement is a key lever for responsible public-sector AI.
  • Start with the public problem and consider non-AI options.
  • Classify risk by impact on people’s rights and services.
  • Contracts should secure testing, documentation, audits, data protection, transparency and exit.
  • Pilot in shadow mode, publish information and monitor for the system’s lifetime.

Frequently asked questions

Should public bodies avoid AI for high-stakes decisions?

Not necessarily, but high-stakes uses need a clear legal basis, rigorous testing, human oversight and contestability.

What is shadow mode?

Running a system alongside existing processes without affecting decisions, to compare its outputs with human decisions.

Can vendors keep models secret from governments?

Contracts can require enough documentation and audit access to verify performance and explain decisions, even if some details stay confidential.