How to choose an agentic AI implementation partner: questions to ask before you sign
The short answer
The questions to ask AI vendors before you sign a build cover ten areas: agent claims, ownership, the model swap, production by day 90, fees tied to results, the EU AI Act provider role, data and logs, the team, exit, and a run where the system was wrong. A good answer names your company, a contract clause, a named person or a record you can inspect; an answer that describes a policy is the red flag.
Many published lists are written for buying a product: features, uptime, single sign-on. Commissioning a system from an implementation partner flips three defaults: who owns what was built, whether your company becomes the provider under the AI Act, and what is left when the partner goes. Nucleo sits on the partner’s side of this table, so these are questions we have to answer too. Ask them of us.
Key takeaways
- Score answers, not questions. Ask for the document behind each answer before the second meeting.
- Test the agent claim first. “AI agent” has no legal definition in the EU. Ask for a recorded run, and for a run where the system was wrong.
- You may be the provider. On a plain reading of Article 3(3), a company that has a system developed and puts it into service under its own name is its provider. Confirm it with counsel before the build.
- Day 90 is your own test. By then one workflow should run in production against one number, with a named owner on your side.
- Exit is decided before signature. What you hold on the last day is whatever the contract names.
What questions should you ask AI vendors before you sign?
Ten, one per area, sent before the first meeting and answered in writing before the second. Score each row on the test above; for what you own, the ownership checklist accepts only your company. A partner that answers in the room and declines in writing has given you its answer.
| Question | A good answer sounds like | Red flag | Read more |
|---|---|---|---|
| Agent claims. Show me a recorded run on data like ours. Which steps ran without a person prompting? | A run log with the autonomous steps counted, the approval points named, and the tests behind each claim | “Fully autonomous”, with no run to show | Agent washing |
| IP, context layer and exhaust. Who owns the code, the context engine, the evaluation sets, the prompts and the traces? | A clause assigning each to your company, held in your repositories and cloud accounts | “You own your data” | The ownership checklist |
| Model swap. If we changed model provider next quarter, what would you rebuild? | One adapter and some prompt tuning, the evaluation set rerun before cut-over, notice of model retirement in the contract | Prompts and pipelines that live in one vendor’s console | An exit-clause checklist |
| Production at 90 days. What runs in production on day 90, against which number, owned by whom? | One workflow, one number with a measured baseline, a production date, a named owner | “A successful proof of concept” | Before you fund the next pilot |
| Fees. Which part of your fee depends on a result we can attribute? | Named fee parts, any outcome-tied part set against a baseline and window agreed in advance | A success fee on a number nobody measured before the build | What an outcome-based contract must specify |
| AI Act role. Are we the provider of what you build and, if the use is high-risk, what goes in the Article 25(4) agreement? | A written split of provider tasks, documentation, known limitations and technical access, reviewed by counsel | “You are just the deployer”, without reading Article 3(3) | Provider vs deployer |
| Data, logs and security. Which company signs for our data, what can the system’s service account touch, and can you rebuild one decision from our records? | A named entity and region, every subprocessor listed, permissions scoped per task, logs in an account you control | “The data stays in the EU”, with logs in the partner’s tenant | Four questions a European buyer should add |
| Team. Who writes the code, what share of their week is ours, and who directs them? | Engineers named in the statement of work, replaced only with your approval, directed by the partner’s own lead | Roles in the proposal, names after signature | What to put on the scorecard |
| Exit. If we ended the contract in month six, what would we hold, and who on our payroll could change it? | Each artefact named, the repository or account it sits in, and the person on your payroll who maintains it | “You can export your data” | What a team should leave behind |
| When it was wrong. Show me a run where the system was wrong: the trace, the case it became, the fix. | A trace from records you hold, the evaluation case it became, the change that fixed it | An accuracy figure with no failed run to show | One question for the next vendor |
Treat ownership, exit and, where the use could be high-risk, the AI Act role as stops: each is cheap to settle before signature and expensive after it, when the partner has no reason to renegotiate. The same ground, written as contract clauses, is in what to settle before you sign.
How do you choose an agentic AI implementation partner?
Decide the delivery model first, then score the partner on what it will put in writing. An agentic AI implementation partner builds systems that carry multi-step work inside your stack, on your data, which is a different purchase from licensing a product.
The delivery model is its own decision: a consultancy for a question with no system to build, an integrator or the vendor’s own team for a known packaged target, a forward-deployed team for a workflow no package covers (the decision rule). For a forward-deployed shortlist, five checks settle most of it, from the production date to what stays when the team leaves.
What is agent washing in AI?
Agent washing is calling a product an AI agent when it is not one, overstating what an agent does, or failing to disclose its risks. The clearest definition comes from the law firm Debevoise & Plimpton, in a post on the Harvard Law School Forum on Corporate Governance based on its memo on securities disclosure risk: “Agent washing refers to situations in which companies call an AI tool ‘agentic’ when it is really just conventional automation, or only has limited generative AI functionality, as well as when companies overstate the degree of autonomy, reliability, or business impact of an AI agent, or fail to adequately disclose the risks and uncertainties associated with deploying agents in consequential AI workflows.” Its standard of proof works just as well for a buyer.
The best-known figure is an analyst estimate. As ITPro reported in June 2025, Gartner said just 130 real agentic AI products exist, despite claims from thousands of vendors. Read it as a mid-2025 snapshot. Gartner’s analyst Anushree Verma added: “Many use cases positioned as agentic today don’t require agentic implementations.” For a buyer, that is the useful half. An honest partner will sometimes tell you the workflow needs rules and a queue, not an agent.
In the EU the label carries no legal weight of its own. The Commission’s AI Act FAQ says “the term AI agent is not legally defined and is used colloquially for different kinds of artefacts”; the Act reaches agents through its definitions of an AI system and a general-purpose AI model. Article 3(1) defines an AI system as one “designed to operate with varying levels of autonomy”, and Recital 12 says the definition “should not cover systems that are based on the rules defined solely by natural persons to automatically execute operations”. On Recital 12’s reading, rule-based automation sold as an agent is not an AI system at all.
Regulators will not run the test for you. The US Federal Trade Commission launched Operation AI Comply on 25 September 2024, on the principle that “there is no AI exemption from the laws on the books”. The five cases announced at launch targeted deceptive AI claims and schemes, and in December 2025 the FTC set aside the order in one of them, against a tool that generated reviews, saying the complaint “failed to satisfy the legal requirements of the FTC Act”. None of the five examined how an agent behaved inside a customer’s workflow, and none gives a European buyer a remedy. Three questions do most of the work before signature.
- Does it carry a multi-step workflow without being re-prompted at each step? Ask for a recorded run on data like yours, and count the steps the system took on its own against the points where a person prompted, approved or corrected it. Debevoise names what an agent-washed product can be underneath: “scripted automation, retrieval plus summarization, or a human-in-the-loop workflow that is presented as fully autonomous”. Approval points are oversight; the test is whether they are labelled honestly: “If the agent requires approval before external communication, that functionality should not be marketed as autonomous execution.”
- Can the vendor show a run where it was wrong? Debevoise writes that claims about what an agent can do, how reliably and with how little supervision “should be supported by testing, deployment records, scope limitations, and defined human-review requirements”. Ask for those four, for the task you are buying.
- Who holds the trace? If the record of each run sits only in the vendor’s console, you cannot check any claim after signature. The oversight side of the test is in what decision-makers should require before approving an agent.
Who owns the IP, the context layer and the intelligence exhaust?
Your company should, by name in the contract, with a line for each, because they are different things. The IP is the code, configuration and documentation the partner writes. The context layer is the reconciled, governed knowledge the system reads from, which we build as a context engine. The intelligence exhaust is what accumulates as the system runs: expert corrections, evaluation sets, agent traces, prompts and retrieval indexes. The six-point audit shows where each usually sits today, and it is rarely with the buyer.
The UK government’s 2020 Guidelines for AI procurement, written with the World Economic Forum, tell buyers: “Consider whether you or the supplier should own any new intellectual property and who can best exploit the IP generated.” The EU’s nearest template leaves it to you: the model contractual AI clauses first published through the Commission-hosted Public Buyers Community “do not contain any conditions concerning intellectual property, acceptance, payment, delivery times, applicable law or liability”. Ownership is your main contract’s job.
A good answer separates two kinds of IP. What the partner built for you is assigned to your company. What it brought with it, its libraries and internal tools, is licensed to you without a time limit, so the system keeps running if the partner goes. Then it says where each artefact lives.
The red flag is “you own your data.” It is true and beside the point. The question is who owns what was learned from it.
What happens when the underlying AI model changes?
Ask what the partner would rebuild if you changed model provider next quarter, and expect a short answer. In a model-agnostic system the model sits behind one adapter and the prompts are versioned in your repository. Prompts may need tuning for the new model, which is what rerunning the evaluation set before cut-over is for; the context engine, the integrations and the workflow logic do not move. If the answer involves rebuilding pipelines inside a vendor’s platform, the vendor owns your exit timetable, whatever the proposal says.
The voluntary UK Code of Practice for the Cyber Security of AI says operators “shall re-run evaluations on released models that they intend on using” and shall tell end-users before a model is updated. The contract should carry the commercial version: notice before a production model is retired or replaced, and the right to substitute another provider’s model, as drafted in the exit-clause checklist.
The AI Act adds a reason to plan the swap from the start. Under Article 3(23), a substantial modification is a change “which is not foreseen or planned in the initial conformity assessment carried out by the provider” and that affects the system’s compliance or changes its intended purpose, and for a high-risk system it can move the provider role (provider vs deployer). How a given swap is classified is for counsel; the engineering answer is to document planned model changes before the first model goes live.
The red flag is a lab partnership offered as the answer. Partnerships are fine. A system that cannot leave one is not yours.
What will be in production at 90 days, and what part of the fee depends on it?
Day 90 is your own window, and a good answer fills it with four things: one workflow, one number with a baseline measured before the build, a production date, and a named owner on your side. Production means real data, behind your access rules, used by the people who do the work. Providers publish their own sequences (how an engagement works compares several), and none is a norm. Ninety days is a test you set, not a benchmark: how long production takes depends on the systems the workflow touches and the approvals it needs, so a partner that needs longer should name them and commit to a production date. What fails the test is a plan with no production date at all.
In Germany, where the company has a works council, a system that records data on how employees work needs the council’s agreement or a conciliation board ruling (what that means in practice), and the Federal Labour Court reads that test broadly: a system qualifies if it records behaviour or performance data that can be tied to a person, whatever the employer intends (1 ABN 36/18). The run traces this post asks for are likely to qualify. Ask whether the 90-day plan includes the agreement, because it can set the production date more than the engineering does.
The red flag is a proof of concept with a demo date. Debevoise describes the market pressure on companies to claim they have deployed agents, which creates incentives to “market pilots as scaled production systems”. A proposal can carry the same stretch. Write the definition of production into the contract, with the date.
Fees follow from the same question: which part depends on a result you can attribute, and how the partner is paid if the work turns out smaller than scoped (the three questions to ask before you sign). An outcome-tied fee needs a baseline and a window agreed in advance (before a headline ROI number goes into your case). Ours has three parts, including a share tied to the outcome where it is a discrete event that can be attributed; how they are structured is in the outcome-pricing post, and there is no rate card, as the services page says.
EU AI Act provider vs deployer: which one are you when a partner builds the system?
Possibly both. A mid-cap that commissions a custom system and runs it under its own name can fit the provider definition as well as the deployer one. The duties that follow are heavy only for high-risk systems, but the contract should settle the roles before the build starts.
The definitions are in Article 3, quoted from the Commission’s AI Act Service Desk, which shows the text as consolidated at 27 July 2026, after the Digital Omnibus on AI. A provider is a body “that develops an AI system … or that has an AI system … developed and places it on the market or puts the AI system into service under its own name or trademark”. A deployer is a body “using an AI system under its authority”. Putting into service includes supply “for own use in the Union” (Article 3(11)), so a system used only inside your company is still put into service. Read plainly, the second limb of the provider definition describes a commissioned build, with the partner who wrote the code as your supplier. The self-build case and what deployers owe in their own right are in what human oversight means in law. As of October 2026 we have found no Commission guideline on the split; Article 96 requires guidelines on Article 25 and on substantial modification, so check before you sign. The open point for counsel is whether an internal system with no brand runs “under its own name or trademark”.
For most systems the provider role is light, but not empty: Article 4 asks providers and deployers of every AI system to support AI literacy, and Article 50 sets transparency duties for systems that interact with people or generate content. The weight lands on the high-risk uses in Annex III (what the EU AI Act demands, and from when), where the provider takes on Article 16, including a quality management system and a conformity assessment before the system is put into service.
Article 25 then says when a deployer becomes a provider. A deployer “shall be considered to be a provider of a high-risk AI system” if it (a) puts its name or trademark on one, (b) makes a substantial modification to one, or (c) changes the intended purpose of a system, including a general-purpose one, so that it becomes high-risk. Case (a) applies “without prejudice to contractual arrangements stipulating that the obligations are otherwise allocated”, so the contract can move tasks between the parties.
| How the system reaches you | Who is the provider, on a plain reading | What to put in writing |
|---|---|---|
| You license a product and use it as sold | The vendor | Instructions for use, and logs you control |
| A partner builds a system you put into service under your name | Your company, as the body that had it developed (Article 3(3)) | Who prepares the technical documentation and, if the use is high-risk, the Article 25(4) agreement |
| You put your name on a vendor’s high-risk system | Your company (Article 25(1)(a)) | The allocation clause, if the vendor keeps provider tasks |
| Someone makes a substantial modification to a high-risk system | Whoever makes it (Article 25(1)(b)) | Planned changes, model swaps included, in the initial documentation |
| A new use makes the system high-risk | Whoever changes the purpose (Article 25(1)(c)) | A written intended purpose, under change control |
The fourth row matters after go-live. If the partner keeps changing a high-risk system under a fee to run and evolve it, each kind of change should be planned in the initial documentation, or the provider role can move. Our recurring fee is exactly that arrangement, so ask us too.
The Digital Omnibus on AI widened the contract term and gave it a fine. Since the original 2024 text, Article 25(4) has required a written agreement between the provider of a high-risk system and the third party that supplies an AI system, tools, services, components or processes used in it, specifying “the necessary information, capabilities, technical access and other assistance” the provider needs to comply. The amendment adds “AI model” to that list. And Article 99(4) now lists these obligations, and the handover duties of Article 25(2), alongside the provider duties of Article 16, with fines of up to EUR 15 000 000 or 3 % of total worldwide annual turnover, whichever is higher, once the high-risk rules apply (from 2 December 2027 for Annex III uses, per the Commission’s FAQ). For SMEs and small mid-cap companies (SMCs), the cap is whichever is lower (Article 99(6) and (6a)).
So ask the partner for its Article 25(4) agreement before you need it. The AI Office may recommend voluntary model terms for these contracts; as of October 2026 we have not found a published set. The updated EU model contractual AI clauses are written for a public buyer and its supplier, but their full version for high-risk AI is a usable starting schedule. Ask, too, how the model provider’s documentation reaches you.
Where will the data and the logs live?
Ask which company signs for your data, in which country, and whose servers handle each request. A good answer names the contracting entity, the region and every subprocessor, on paper.
Then ask about the logs. The buyer’s test: can the partner rebuild one decision end to end from records your company holds? What to capture, field by field lists what that takes.
What security questions should you ask AI vendors?
The ones a product questionnaire misses are about a system that acts in your other systems. Ask which service account it acts through, and whether its permissions are limited to what each task needs; the UK code asks that permissions granted to an AI system on other systems be “only provided as required for functionality and are risk assessed” (provision 2.6). Ask how it was tested against indirect prompt injection, instructions planted in a document, email or ticket it reads, which the code names among the risks distinct to AI. Ask how quickly you are told about an incident, and to see the incident management and recovery plans the code asks for (provision 6.5). Notice before a model update is covered above (provision 7.4). A good answer is a permission list per task, a test record and a notification time in the contract.
The red flag is “the data stays in the EU”, with no entity named, the logs in the partner’s tenant and one service account with broad rights across your systems.
Who is actually on the team, and who directs them?
Ask for the names of the people who will write the code, the share of their week that goes to your build, and who directs them day to day. A good answer puts the engineers’ names in the statement of work, with a clause that they are replaced only with your approval, names a lead from the partner who directs them against an agreed result, and expects an owner on your side from the first week. Who is on a forward deployed engineering team sets out both sides.
Then ask which vendor relationship funds the pod (what to put on the scorecard instead). Direction is a legal question too: if your own managers direct the partner’s engineers day to day, the engagement can look like labour leasing under German or Italian law (who carries the delivery risk).
The red flag is roles in the proposal and names after signature.
What happens when the contract ends?
Ask what you would hold on the last day if you ended the contract in month six, and who on your payroll could change the system. A good answer names each artefact, where it sits and who maintains it. The full inventory is in what a forward deployed engineering team should leave behind, and the exit-clause checklist drafts the terms: an export of the learning record, a deletion certificate, termination assistance and notice before a model is retired.
Two public documents add what buyers forget. The UK procurement guidelines ask buyers to evaluate “the thoroughness and logic of the knowledge transfer plan”, and the UK cyber security code says that at decommissioning the operator shall “securely delete applicable data and configuration details”. Put both in the statement of work: a knowledge transfer plan with names in it, and a deletion process that ends in a certificate.
The red flag is “you can export your data.” That is how you leave a product. Leaving a build should hand you a running system and the people to run it.
How will the partner show what the system did when it was wrong?
Ask to see a run where the system was wrong: the trace, the evaluation case it became, and the change that fixed it. It is the last question because it tests the others. A partner who can show it keeps traces somewhere you can read them, has an evaluation set, and has run a system long enough to see it fail. A real answer describes a mechanism, not a policy (one question for the next vendor conversation).
When the provider role passes to someone else, the Act asks the initial provider to tell the new one about “known limitations and failure modes” (Article 25(2)). Ask your partner for that list as a deliverable from the first month, updated each time a run goes wrong.
The red flag is an accuracy figure with no failed run behind it, or an answer about who reviews the work rather than what the record shows.
When is Nucleo the wrong partner?
When the work is a packaged platform rollout with a known target, a fixed deadline and a need for a large certified team, an integrator or the vendor’s own team will beat us on it (when a consultancy or a system integrator is the right choice). A strategy question with no system attached is consulting work. And a workflow that starts and ends inside one platform is usually best served by that platform’s own agents and people, who have roadmap access we do not.
Everywhere else, hold us to the same ten rows. Our published answers are on the services page: the context engine, the systems built on it and the IP belong to your company; the build is model-agnostic; it runs on EU-hosted infrastructure, in your own cloud account or on your premises; and the engagement starts with one workflow and one number agreed before the build. Ask us for the clause behind each, the same as anyone else.
Two rows we cannot answer from a web page. We publish no client runs, so ask to see our trace format and the first evaluation cases in the opening weeks, and hold the contract to them. And our page says where the system runs, not which entity and subprocessors sit in the path: ask us for that in writing.
FAQ
What questions should you ask an AI vendor before signing? Ten, one per area: agent claims, ownership of the IP and context layer, the model swap, production at day 90, fees tied to results, the EU AI Act provider role, data and security, the team, exit, and a run where the system was wrong. An answer passes only when it names your company, a contract clause, a named person or a record you can inspect.
What security questions should you ask AI vendors? Four specific to a system that acts: which service account it uses and whether its permissions are limited to each task, how it was tested against indirect prompt injection, how fast you hear of an incident, and whether you get notice before a model update. Then the data questions: which company signs for your data, every subprocessor in the path, and whether one decision can be rebuilt from records you hold.
What is agent washing in AI? Calling a product an AI agent when it is conventional automation or limited generative AI, overstating an agent’s autonomy, reliability or business impact, or failing to disclose the risks of deploying it in consequential workflows, as Debevoise & Plimpton define it. As ITPro reported in June 2025, Gartner said just 130 real agentic AI products exist, despite claims from thousands of vendors. The EU AI Act does not define an AI agent, so the buyer has to test the claim: a recorded run, a run where the system was wrong, and who holds the trace.
What is the difference between a provider and a deployer under the EU AI Act? A provider develops an AI system, or has one developed, and puts it on the market or into service under its own name or trademark. A deployer uses an AI system under its authority. Putting into service includes supply for the company’s own use, so on a plain reading of Article 3, a company that commissions a custom system and runs it under its own name can be both. The heavy obligations attach to high-risk systems. Take the specifics to counsel.
When does a deployer become a provider under the EU AI Act? Under Article 25(1), when it puts its name or trademark on a high-risk AI system, makes a substantial modification to one, or changes a system’s intended purpose so that it becomes high-risk. Article 25(4) requires the provider of a high-risk system and its suppliers to set out information, capabilities and technical access in a written agreement; since the Digital Omnibus on AI it names the supplier of the AI model, and breaches can be fined under Article 99(4) once the high-risk rules apply, from 2 December 2027 for Annex III uses.
How do you choose an agentic AI implementation partner? Decide the delivery model first: a consultancy when there is nothing to build, an integrator or the vendor’s own team for a known packaged target, a forward-deployed team for a workflow no package covers. Then score every shortlisted partner on the same ten questions, and ask for the document behind each answer before the second meeting.
What should be in production after 90 days? One workflow running on real data, behind your access rules, measured against one number with a baseline taken before the build, and owned by a named person on your side. Ninety days is a test you set, not an industry standard, and a proof of concept with a demo date does not pass it. In Germany, if the company has a works council and the system records data on how employees work, plan for the council’s agreement or a conciliation board ruling.
What should an AI implementation contract say about exit? It should name what you hold on the last day, so settle it before signature: ask what you would hold if you ended it in month six, and who on your payroll could change the system. The terms behind the answer are an export of the learning record, a deletion certificate, termination assistance, notice before a model is retired, and a knowledge transfer plan with names in it.
Related reading
- Forward-deployed engineer vs consultant vs system integrator: a decision rule for buyers
- What is a forward-deployed engineer (and why enterprise AI needs one)
- Own your intelligence: what it actually takes
- Nadella’s reverse information paradox, read by a European CIO
- Keeping humans in the loop: human oversight for enterprise AI agents
- Audit trails for AI agents: what regulated enterprises actually need
Nucleo is an AI-native transformation company for European mid-caps. Our forward-deployed engineers build the context engine your company runs on, in your estate and model-agnostic, and put systems into production on top of it. Ask us the ten questions above. See how an engagement works, or write to us.
Sources: Debevoise & Plimpton, Agent washing, Harvard Law School Forum on Corporate Governance; ITPro, Gartner on agent washing; FTC, Operation AI Comply and the Rytr order set aside; AI Act Service Desk, FAQ; AI Act, Article 3; Recital 12; Article 4; Article 16; Article 25 and its original 2024 text; Article 50; Article 96; Article 99; Public Buyers Community, EU model contractual AI clauses and updated version; UK government, Guidelines for AI procurement; UK government, Code of Practice for the Cyber Security of AI; Betriebsverfassungsgesetz, section 87; Federal Labour Court, 1 ABN 36/18.