Guide AI

AI Vendor Selection, Risk Assessment and Pilots

How to choose the technology and vendor for an AI project, assess its risks, confirm the legal requirements and run a controlled pilot.

10 November 2026
6 min read

This page covers steps 4 to 7 of our AI implementation guide. It assumes you have a defined objective and a prioritised use case, which steps 1 to 3 in choosing AI use cases and checking data readiness set up. Testing, rollout and monitoring follow in steps 8 to 12.

Step 4: Select the technology, architecture, and vendor

Most UK SMEs implementing their first AI use case are choosing between an off-the-shelf AI feature in a platform they already use, a specialist vendor product, and a custom build on top of a foundation model API. Each has a different cost, speed, and control profile.

⚡
Buy: platform-native AI feature
Fastest to deploy, lowest build cost, least flexible. Right choice when the use case matches what the feature was built for and the platform already holds the relevant data.
🏗️
Buy: specialist AI vendor product
Purpose-built for the use case, faster than a custom build, but introduces a new vendor relationship and integration work. Worth it when the vendor has a proven track record in your specific process and industry.
🔧
Build: custom application on a foundation model API
Most flexible, most control over data handling and behaviour, highest build and maintenance cost. Right choice when the use case is core to your competitive position or no existing product fits it well.

A note on architecture

Whichever route you take, decide early how the system fits together: where the data lives relative to the model, whether the AI component calls out to other internal systems or external APIs on its own, and where the human checkpoint from Step 11 sits in that flow. A platform-native feature usually has this decided for you. A specialist vendor product or custom build does not, and an architecture that lets the AI component read or write more systems than the use case actually needs is the single easiest way to turn a contained pilot into an unbounded one. Keep the integration surface as narrow as the objective requires, and widen it later if a specific, justified need arises.

Whichever route you take, the vendor due diligence questions in our AI Governance guide still apply in full: where data is processed, what happens to it on contract termination, and what security certifications the vendor holds. Add two implementation-specific questions: how well does the vendor's product perform on a sample of your actual data, not their demo data, and what does the exit path look like if you need to migrate off the tool once a process depends on it.

Step 5: Assess the risks

Treat this as a proportionate assessment against the use case's actual risk tier, not a checklist to complete for every project regardless of size. A low-risk internal use case needs a lighter pass than one that touches customers or individuals' rights.

Accuracy and reliability

How often does the system need to be right for the use case to be worth deploying, and what happens when it is wrong? A drafting assistant that is wrong 5% of the time is fine because a human reviews the output. An automated decision system that is wrong 5% of the time at scale is a different problem entirely.

Security

Does the implementation introduce new attack surface: an API integration, a chatbot that processes untrusted input, a system connected to internal data it did not previously have access to? The security controls in our AI Governance guide's security section apply directly here.

Privacy

Does the use case process personal data in a new way? If so, a DPIA is likely required before deployment, not after. See Step 6 below.

Bias and fairness

Where the system's output affects individuals differently based on who they are, test for disparate outcomes across relevant groups before deployment, using a representative sample large enough to detect a meaningful gap.

Transparency

Can you explain, in terms a non-technical stakeholder or a regulator would accept, why the system produced a given output? If the answer is "we do not fully know," that materially limits which use cases the system is suitable for.

Autonomy and agency

If the system can take an action on its own, sending an email, updating a record, calling another system, rather than only producing text or a recommendation for a human to act on, treat it as a materially higher-risk implementation regardless of how well it performs. Define exactly which actions it is permitted to take unsupervised, which actions always require human approval first, and what happens if it is given a task that requires a tool or a level of access nobody explicitly granted it. An agentic system that quietly acquires more reach than the use case intended is a bigger risk than one that occasionally gets an answer wrong.

Confirm this before you build, not after a working prototype exists and legal raises an objection nobody budgeted time to address. Two questions cover most implementation projects: does UK GDPR require a DPIA, and does the EU AI Act apply.

If the use case processes personal data and involves profiling, automated decision-making with significant effects, or large-scale monitoring, a DPIA is required under UK GDPR Article 35 before deployment. If your organisation has EU customers, EU operations, or supplies an AI system into the EU, check the use case against the EU AI Act's risk tiers, particularly if it touches recruitment, credit, or another high-risk domain listed in Annex III. Our AI Governance guide covers both the UK GDPR obligations and the EU AI Act applicability in full detail, including the current implementation timeline.

Step 7: Run a controlled pilot

A pilot is not a soft launch with a different name. It needs a defined scope, a success threshold agreed before it starts, and a rollback plan agreed before it starts, not improvised if things go wrong.

1
Scope
A limited slice of the process: one team, one region, one ticket category, one document type. Small enough that a failure is contained, large enough to produce a meaningful volume of real cases.
2
Success criteria
The specific, numeric threshold that determines whether the pilot moves to wider rollout, agreed with the business sponsor before the pilot starts. Avoid "see how it goes."
3
Timeline
Long enough to gather a representative volume of cases, short enough to reach a decision point before the pilot becomes a permanent, ungoverned fixture. Four to twelve weeks covers most use cases.
4
Rollback plan
The specific steps to revert to the previous process if the pilot underperforms or produces an unacceptable failure, and who has the authority to trigger it.

Common questions

How long does an AI implementation pilot typically take?

Most well-scoped pilots run for four to twelve weeks, long enough to gather a meaningful volume of real-world cases without letting the pilot drift into a permanent, ungoverned deployment. The right length depends on how often the process being changed actually runs: a pilot covering a monthly reporting task needs several cycles to produce useful data, while a pilot on a high-volume daily process can reach a decision point in a few weeks.

Does the EU AI Act apply to AI systems we build ourselves, not just vendor tools?

Yes. The EU AI Act applies based on what the AI system does and who it affects, not on whether you bought it or built it. An internally built system used in recruitment, credit decisions, or another high-risk domain is assessed against the same risk tiers as a commercial product, and if you supply that system to others you take on provider obligations rather than the lighter deployer obligations that apply to organisations using a vendor's tool. Confirm your regulatory position before you build, not after. Our AI governance guide covers the risk tier structure in detail.

Sources

AI Implementation Assessment

Not sure where to start with AI in your business?

We help you turn a vague "we should do something with AI" into a scoped use case, a tested pilot, and a measured result. Deliverable: a prioritised use case shortlist and a pilot plan ready to run.

  • ✓ Which use cases fit your business objective and your data
  • ✓ Whether to buy, use a specialist vendor, or build custom
  • ✓ What the legal and regulatory position is before you build
  • ✓ A pilot scope, success criteria, and rollback plan
  • ✓ How oversight and monitoring should be built into the process
  • ✓ A 90-day roadmap from decision to measured result

Disclaimer: This article is for general informational purposes only and does not constitute legal, regulatory, or professional advice. Cyvra makes no warranty as to the accuracy or completeness of this content, which may not reflect the most current regulatory developments. Readers should seek independent legal and regulatory advice appropriate to their specific circumstances. Cyvra accepts no liability for any loss arising from reliance on this content.