TL;DR: The best questions to ask AI developers cover five areas: the people, the build, the AI itself, the contract and life after launch. Ask all 15 below before you sign. Clear, specific answers are a good sign. Vague answers are your warning.
If you have a proposal from AI developers on your desk, the slides will look good. The gaps show up later. These 15 questions close them first. Each one has a reason and a sample of what a good answer sounds like.
The five questions in the third group are about the AI itself. Don't skip them.
This post is not a list of warning signs. For that, read our red flags when hiring an AI agency post, which also has a free 10-question checklist. Still comparing firms? Start with how to choose an AI development company.
Price depends on scope, and scoping comes first. So cost is not on this list.
What questions should you ask AI developers about the people?
1. Who will actually build my software?
Why it matters. The person who sells you the project is often not the person who writes it. You need to know who does the work.
A good answer sounds like: Names, roles and hours for each person. You meet the lead before you sign. If they use contractors, they say so up front.
2. Have you built something like this before?
Why it matters. A first-time build is a learning job, and you pay for the learning. Past work on a similar problem cuts your risk.
A good answer sounds like: Working products you can try, not slides. A plain story of what went wrong last time and what they changed. A client you can call. For context, Devwiz has built 200+ apps since 2015. That is the kind of track record to ask for.
3. How do you use AI to write the code, and who checks it?
Why it matters. Most developers use AI tools now. In the Stack Overflow 2025 Developer Survey, 84% of respondents use or plan to use them. But more developers distrust the accuracy of AI tools (46%) than trust them (33%).
A good answer sounds like: "We use AI to move faster. A person reads every change. Tests run before anything ships."
What questions should you ask about the build?
4. How will you scope the work before you quote?
Why it matters. A RAND report looked at why AI projects fail. It interviewed 65 experienced data scientists and engineers. The top root cause was simple. People misunderstand or miscommunicate the problem the AI needs to solve.
A good answer sounds like: A written plan, a clickable prototype and a clear first-version scope before any quote. They also tell you what is not in it. Our discovery phase explainer shows what that looks like.
5. How do you test the work?
Why it matters. Google's DORA research found AI works as an amplifier. It magnifies a team's strengths and weaknesses. So ask about habits, not just tools.
A good answer sounds like: Automated tests, a staging site, and a step where you try it yourself before launch.
6. How will I see progress?
Why it matters. If you only see the work at the end, you only find the problems at the end.
A good answer sounds like: A working demo every week or two. A shared task board. One named person you can message.
What questions should you ask AI developers about the AI itself?
7. Which AI model will you use, and why?
Why it matters. Claude, GPT and Gemini differ in quality, speed and price per use. Picking one with no reason is a guess.
A good answer sounds like: They name the model. They say what they tested against your task. They say why it suits you, and where a cheaper model can handle the simple jobs.
8. What happens when the model changes?
Why it matters. AI providers update and retire models. Your product's answers can shift when that happens, and nobody warns your customers.
A good answer sounds like: The model sits behind a layer that can be swapped. Tests re-run on any new model before you switch. You get notice and a say.
9. How do you test the AI's output?
Why it matters. AI can give a different answer each time. "It worked in the demo" proves nothing.
A good answer sounds like: A set of real test questions with expected answers, run on every release. Limits on what the AI may say. A person checks risky answers. A plan for when it gets one wrong.
10. Who pays the model usage bill?
Why it matters. AI runs on usage fees. Every time a user acts, someone pays. A surprise bill hurts, and a hidden markup hurts more.
A good answer sounds like: The account sits in your name, so you see the real bill. They estimate usage from your expected volume. Spending caps and a monthly report come standard.
11. Where does my data go?
Why it matters. The OAIC's APP 8 guidance covers sending personal information overseas. You must take reasonable steps to make sure the overseas recipient does not breach the privacy principles. You can be held accountable for what they do with it. This is general information, not legal advice.
A good answer sounds like: A list of every service that touches your data. Where each one is hosted. Whether the AI provider trains on your data. See our guide to data residency in Australia.
What should the contract say?
12. Who owns the code, the prompts and the data?
Why it matters. Prompts are the instructions that shape the AI. They are part of what you paid for. Check the contract doesn't let the firm keep them.
A good answer sounds like: You own all three, in writing, once you have paid. Any shared tools or open-source parts are named in a list.
13. What do I get if we part ways?
Why it matters. You may change firms one day. Your software should not be stuck.
A good answer sounds like: The code in a repository you own. Hosting and AI accounts in your name. Written notes and a handover session. Read our AI software handover process for the full list.
14. How do you handle changes and delays?
Why it matters. Scope always moves. The question is who pays for the move, and when you find out.
A good answer sounds like: A written change step. You see the effect on time and scope before any work starts. They say what happens if a milestone slips.
What happens after launch?
15. Who fixes it when it breaks, and how fast?
Why it matters. Software breaks. AI products break in new ways, because the model underneath keeps moving.
A good answer sounds like: A named support contact. Response times in writing. Monitoring that spots problems before you do. A clear line between free fixes and paid support. For example, the Devwiz 30-Day Prototype Launch includes 30 days of fixes. Ask every firm what your window is.
What do you do with the answers?
Send the questions in writing and ask for written answers. Then compare them side by side.
Specific answers with names, steps and tests are solid. Vague answers like "we handle that" are not. Treat a refusal to answer as a no.
Your next step: pick the five AI questions and send them to every firm on your shortlist today. Then compare what comes back.
Frequently asked questions
How many questions should I ask AI developers?
Ask all 15 if you can. If time is short, start with the five AI questions. They cover the model, model changes, output testing, the usage bill and your data.
What is the most important question to ask AI developers?
Ask who owns the code, the prompts and the data. It is the hardest thing to fix after you sign. Get the answer in writing and make sure the contract says the same thing.
Is it a red flag if AI developers use AI to write code?
No. Most developers do. The question is who checks the work. Ask how they review AI-written code and what tests run before anything ships.
Do I need to understand the tech to ask these questions?
No. Each question is plain English. You are judging how clear the answer is, not how technical it is. Good developers explain things so you can follow.
What if the developers will not answer a question?
Treat it as a no. A firm that is sure of its work can explain it. If they dodge the data or ownership questions, move to the next firm on your list.
Who should own the AI model account?
You should. When the account is in your name, you see the real usage bill and keep control if you change firms. Ask for that in the contract.
Should a lawyer read the contract?
Yes. These questions help you spot gaps, but they are general information, not legal advice. Have a lawyer check ownership, data and liability terms before you sign.
About James Killick
10+ years building digital products · 200+ apps shipped since 2015
James is a co-founder of Devwiz and an AI product specialist. Since 2015 he has helped ship 200+ apps for founders, businesses and government, including work for NSW Government, Briometrix and Huskee. He builds AI-first platforms and writes about turning a proven program into software. He also hosts the Up in the AI podcast.
More articles by James · James's personal site · LinkedIn · AI Orchestrators
Tags: Hiring


