Most companies get burned by AI agencies because they evaluate on portfolio and price, not process and ownership.
Hiring an AI development agency is not like hiring a design studio or a marketing firm. The output is infrastructure. It runs in production, handles real data, and either works when you need it or it does not. Getting that choice wrong is expensive, disruptive, and often hard to reverse. We have spoken to dozens of businesses who arrived at us after a previous AI agency delivered something that could not be maintained, used proprietary tools that created lock-in, or simply stopped responding once the invoice was paid.
The agency evaluation process is broken. Most buyers look at case study slides, check a few LinkedIn profiles, and compare day rates. None of those signals tell you whether the agency writes production-ready code, who owns the IP when the engagement ends, or how they handle the messy realities of a live AI system six months post-launch.
This guide gives you the exact questions to ask, the red flags to walk away from, and a direct look at how Axonari answers each one.
7 Questions to Ask Before You Sign
Ask every agency on your shortlist these seven questions. Their answers, and how quickly and confidently they give them, will tell you more than any case study.
1. Who writes the code, and where are they based? Some agencies sell through a UK or US front-of-house and offshore delivery without disclosing it. That is not inherently a problem, but it needs to be disclosed. You are entitled to know the seniority level and location of the people doing the work.
2. Who owns the IP when the engagement ends? The default in many agency contracts is that the agency retains copyright until full payment is received, and some retain rights to reuse components across other client projects. You want a contract where all custom work transfers to you on completion and the agency has no ongoing rights to your codebase.
3. How do you handle post-launch issues? AI systems degrade in ways that websites do not. A model that was accurate at launch can drift as the data it operates on changes. Ask how the agency defines their post-launch support obligations and what the commercial arrangement is for maintenance versus bug fixing.
4. Can you show me a system you built that is still running in production two years later? Case studies are easy to write. A production system running reliably two years after handover is evidence of code quality, documentation, and a client who could actually operate what was handed to them.
5. What happens if a key person on the project leaves your team mid-engagement? Small agencies are exposed here. If the lead engineer on your project leaves, is there someone who can pick up the work without a knowledge transfer period that delays your delivery?
6. What does the handover look like? A good handover includes a working system, documented architecture, a runbook for common operational tasks, and a codebase your team or a future agency can extend. An agency that cannot describe their handover process in specific terms is an agency that does not do good handovers.
7. What have you built that failed, and what did you do about it? Every honest agency has had a project go wrong. An agency that cannot answer this question either has not built enough, is not being honest, or has never taken responsibility for a failure. The quality of the answer tells you more about how they operate than any success story.
Red Flags to Walk Away From
Some signals are clear enough that they should end the conversation before it progresses further.
Guaranteed results before scoping is complete. An agency that quotes outcomes, conversion lifts, cost savings, or accuracy percentages before doing a technical discovery of your data and systems is telling you what you want to hear, not what is realistic. AI system performance depends entirely on your data quality, your integration environment, and what you are actually optimising for. No honest agency can promise a specific outcome without understanding those factors first.
No clear ownership of code or data on exit. If the proposal does not clearly state that you own the code, the models, and the data pipelines at the end of the engagement, that ambiguity will be exploited. Get the IP terms in writing before any work starts.
Generalist portfolio dressed as AI specialisation. A web agency that added an AI chatbot to three projects is not an AI development agency. Look for evidence of LLM integrations, autonomous agent architecture, production RAG systems, or AI automation workflows, not AI-adjacent feature work on otherwise standard web projects.
No mention of failure modes or limitations. An AI agency that presents only upside in their initial conversations has not done this at production scale. Real AI deployments involve edge cases, model limitations, data quality problems, and monitoring requirements. If none of that comes up in the first conversation, the agency either lacks experience or is hiding problems you will encounter later.
Understanding Pricing Models
How an agency prices work shapes the incentives on both sides. There are three main models and each fits a different project type.
Fixed price works when scope is genuinely well-defined. You get a specific deliverable for a specific number. The agency bears the risk of underestimating. The problem is that AI projects rarely have scope that is well-defined before a discovery phase. Fixed price on an AI project with vague requirements usually means the agency has padded heavily or will deliver the minimum viable interpretation of the brief.
Time and materials works when scope evolves as you learn. You pay for actual hours at an agreed rate. The agency bears less risk, so they can be more honest about complexity and discovery findings. The problem is that without strong project management on your side, costs can expand. Good T&M engagements have weekly reporting on hours and clear milestones that gate continued spend.
Retainer works when the relationship is proven and you need ongoing capability rather than a defined deliverable. Monthly access to a team for a fixed number of days. This model suits post-launch support, continuous improvement programmes, and ongoing AI model maintenance. Do not start a new engagement on retainer. Earn the retainer by delivering something first.
How to Evaluate a Proposal
A good proposal demonstrates that the agency understood your problem, thought through the architecture, and has a plan for handing over something you can actually operate. Look for these five elements.
A specific problem statement, not a restatement of your brief. If the proposal opens with a summary of what you told them, it is not a proposal. A good proposal opens with the agency's diagnosis: what the actual problem is, what the constraints are, and what success looks like in measurable terms.
An architecture section that is specific to your environment. Proposals that describe generic AI capabilities without referencing your specific data sources, existing systems, or integration requirements have not been thought through for your situation. The architecture section should name the components, explain the data flow, and identify the integration points that will require work.
A clear handover plan. What documentation will be produced? What does the runbook contain? How will the team be trained to operate and maintain the system? This section should be as specific as the build section.
Named risks with mitigations. A proposal that identifies the likely problems and explains how they will be managed is a proposal from an agency that has done this before. A proposal that presents only a linear build plan with no risk identification is a proposal from an agency that will be surprised by problems you could have anticipated.
Post-launch terms written down. What is included in the post-launch support period? What falls outside it? What are the commercial terms for ongoing maintenance? These answers should be in the proposal, not something to negotiate after you have already signed.
How Axonari Answers These Questions
We hold ourselves to the same standard we ask clients to apply to every agency they evaluate. On IP: all custom code transfers to you on completion. We retain no rights to reuse or reproduce your codebase. On staffing: the engineers named in the proposal are the engineers who do the work. We do not subcontract. On post-launch: every engagement includes a 30-day support period with defined response times, and we offer maintenance retainers for clients who want ongoing coverage. On handover: every project closes with a working system, a documented architecture, and a runbook. On failure: we have had projects that ran over time and one that required a significant rebuild six months post-launch because the client's data environment was different from what discovery had revealed. We absorbed the cost of the rebuild.
Key Takeaways
The right questions to ask an AI agency are about ownership, production history, team composition, and how they handle problems, not about their portfolio or their day rate. The signals that end the conversation before it starts are guaranteed outcomes before scoping, unclear IP terms, and a portfolio that does not include systems still running in production. The pricing model matters less than whether it aligns incentives for your specific project type. A good proposal diagnoses the problem, names the architecture, and includes a specific handover plan.
Ready to Ask Us These Questions?
Book a 30-minute call. Bring your shortlist questions, your requirements, and your scepticism. We will answer all of it and give you an honest assessment of whether we are the right fit.