Red Flags When Hiring an AI Agency

TL;DR: When hiring an AI agency, the biggest red flags are no working software to show, no named tech, a quote with no breakdown, one person doing everything, and a contract that won't say you own the code. If you see two or more, walk away before you sign.

Some AI agencies are consultants with a ChatGPT subscription and a good pitch deck. Others build. Knowing which one you're talking to saves you months and real money.

The stakes are real. RAND says that by some estimates more than 80% of AI projects fail, twice the rate of IT projects without AI. That is an estimate RAND cites, not something it measured. Its interviews with 65 experienced data scientists and engineers point to a top cause: people misunderstand or miscommunicate the problem the AI needs to solve. A weak agency makes that worse. A good one catches it early.

Here are the red flags, with the question to ask for each.

Can they show you AI they built?

This is the first question. Not case studies. Not slides. Working software.

Ask to see a live product, a demo, or a client you can speak to. A real agency has shipped work. We do AI software development at Devwiz, and we've built 200+ apps since 2015. Clients include NSW Government, Briometrix, Vivid and Huskee.

If you get "we've done similar projects" or a PDF full of frameworks and process diagrams, that's your answer. They talk about AI. They don't build it.

Ask straight: what did you ship, who for, and what can I see?

Do they talk about tech or just outcomes?

Consultants talk about outcomes. Builders talk about tech.

A real agency can tell you quickly what they'd use for your project. Would they call an existing model through an API, fine-tune one, or build an agent that takes actions? Do they have a view on Claude, GPT or Gemini, and why?

Some "it depends" is fine early on. But if they can't give you a rough technical direction after a 30-minute call, they're working it out as they go. You want a team with opinions. Opinions come from shipping.

Are they being straight about scope and cost?

AI projects have a reputation for blowing budgets. Some of that is real complexity. Some of it is agencies quoting low to win the work, then adding scope halfway through. Gartner forecast that at least 30% of generative AI projects would be abandoned after proof of concept by the end of 2025. Its reasons: poor data quality, weak risk controls, rising costs and unclear business value. That is a forecast, not a count. But the reasons ring true.

What a project costs depends on scope, and scoping comes first. So watch for:

• Quotes with no line-item breakdown
• Vague milestones like "Phase 1: Discovery" with no deliverables attached
• Contracts that define success as hours billed, not what gets delivered
• No clear ownership of the code at the end

A good agency gives you an honest number, explains what drives it, and tells you what they'd cut if you needed to shrink the scope. If someone quotes far below everyone else, ask why. Either something important is missing, or change requests are coming.

Is it one person or a team?

Some "AI agencies" are solo operators or small freelance groups. That's not automatically bad. But it matters for your project.

Ask who will do the work. Is there someone on product, someone on the back end, someone on the front end? Or does one person do it all? If it's a team, ask to meet them.

One person can hit a ceiling fast on a complex build. They're also a risk. If your key person gets sick or takes another job, your build stalls. For businesses building AI platforms and programs, you want real depth across the stack, not one person who knows a bit of everything.

Are they learning on your budget?

This one is harder to spot. Ask how many AI projects they shipped in the last 12 months. Ask what went wrong on one of them and how they fixed it. Ask what AI they use in their own business.

If they can't answer honestly, they're probably still learning AI. You'd be paying for their education.

A current team will tell you what changed in their approach in the last six months and which tools they dropped. The tools move fast, and an agency that was sharp 18 months ago may not be now. Google's DORA research, a survey of nearly 5,000 technology professionals, found AI works as an amplifier. It magnifies a team's strengths and its weaknesses. A weak process with AI is still a weak process.

Are they building something custom or stitching tools together?

There's nothing wrong with using existing AI tools. Most good builds do. But you need to know what you're paying for.

If an agency charges custom-build prices to connect a few APIs and wrap them in a screen, that's integration work. It may still be worth doing. The price should match the work.

A good AI app development partner is clear about what's custom and what's off the shelf, and explains the trade-offs. So ask: what are you building from scratch, what are you plugging in, and why?

Are the contracts one-sided?

Agency contracts often protect the agency. That's normal. Some are plainly unfair. Check these:

• Who owns the code? You should own what you paid to build.
• What happens if you leave mid-project? Is there a clear handover process?
• Are there consequences if they miss milestones?
• What do support and fixes look like after launch?

If an agency resists plain questions about ownership, or goes vague on what happens when the relationship ends, that's a flag.

Where will your data go?

If your customers' details will pass through the agency's team or tools, ask where the work happens. If personal information goes overseas, the Australian Privacy Principles say you must take reasonable steps to make sure the recipient doesn't breach them. You can be held accountable for what they do with it. The OAIC's APP 8 guidance explains it. That is general information, not legal advice. An agency that shrugs at this question is a flag. For more on the trade-offs, see our post on offshore development risks.

What should you do if you spot two or more of these?

Walk away, or slow down and ask the hard questions in writing. Good agencies are happy to answer them. Our guide on how to choose an AI development company covers what to look for instead.

Your next step: on your next call, ask "what did you ship, who for, and what can I see?" Then judge the answer.

Frequently asked questions

How do I know if an AI agency builds or just consults?

Ask to see working software they've shipped. A real agency can show you live products or put you in touch with clients. Slides and frameworks mean they're consultants. Ask: what did you build, who for, and can I see it?

What questions should I ask when hiring an AI agency?

Start with these. What tech would you use for this project and why? What have you shipped in the last 12 months? Who will do the work? Who owns the code at the end? A real builder answers these clearly. Vague answers to technical questions are a warning sign.

Why do AI agency projects go over budget?

Usually vague scoping up front, a low quote to win the work followed by added scope, or underestimating data preparation and integration. Ask for a line-item breakdown before you sign. Agree what success looks like in clear terms, not just hours billed.

Is it okay to hire a small AI agency or solo operator?

Sometimes. A solo operator can be quick on a contained build. The risk is capacity and continuity. If AI is core to your product, you want a team across product, engineering and data. Ask who does what, and what happens if your key person is unavailable.

How do I check if an AI agency is up to date?

Ask what changed in their approach in the last six months and which tools they dropped. Ask what AI they use inside their own business. A current team has strong opinions and recent examples. A team coasting on old knowledge gives generic answers.

Should I own the code an AI agency builds for me?

Yes. You should own what you paid to build, and the contract should say so in plain words. Also check what happens to the code if you part ways mid-project. If an agency won't put ownership in writing, treat it as a red flag.

How many red flags should make me walk away?

One flag is a reason to ask more questions. Two or more, such as no shipped work and a quote with no breakdown, is a reason to walk away. A good agency answers hard questions without getting defensive.