AI feasibility assessment: go, iterate or stop

TL;DR: An AI feasibility assessment tests whether an AI idea can be built, has the data to back it, and will pay for itself. It ends in a viability statement: build, iterate or stop, with a confidence level. Check technical fit, data, model score, operations and returns. Test at three points: before building, after, and after launch.

You have an AI idea. The team likes it. Someone wants to start building on Monday.

Should you?

An AI feasibility assessment answers that before you spend the engineering budget. It tests three things: can it be built, does the data support it, and do the numbers work. The output is a short viability statement. It says build, iterate or stop, and how sure you are.

This is not the same as a discovery phase. Discovery finds and ranks the problems worth solving. Feasibility takes one idea and asks if it will pay. Our discovery phase guide covers the first job. This post covers the second.

What a good assessment hands you

Four things come out the other end:

• A viability statement. Build, iterate or stop, with the confidence behind it.
• A sensitivity table. How returns shift if cost, adoption or accuracy come in worse than hoped.
• Acceptance criteria. What "good enough" means for the model before it goes live.
• A short risk register. Data, technical and commercial risks, each with an owner.

Leaders use it to fund a pilot. Investors use it to check a pitch. Engineers use the acceptance criteria to know when to ship.

The five checks

Any one of these can sink an AI project on its own.

Technical. Can the architecture meet your speed and integration needs? A model that works in a notebook is a different job when it must answer in real time against a legacy system.

Data. Is there enough clean, labelled data? Do you know where it came from? Can you legally use it for this purpose? Check privacy now, not after training.

Model performance. Pick the target metric first. Set a minimum score below which the project is dead. Use a test set big enough to trust, so a lucky split can't fool you.

Operations. Who watches the model once it's live? You need drift monitoring, a retraining plan, an incident plan and a real maintenance budget.

Commercial. List the benefits, the build cost and the running cost. Work out the payback time. Test it under a good, a normal and a bad scenario.

Set the minimum model score before you see any results. It's far easier to agree a bar in the abstract than after a model has let you down.

Test viability at three points, not one

A 2026 paper in Business & Information Systems Engineering offers a clean way to do this. The authors built a Viability Assessment Framework. It checks a project at three stages:

1. Pre-development. Work out the minimum model performance the project needs to be worth building.
2. Post-development. Measure the real model against that bar.
3. Post-deployment. Check again on a schedule, because live data drifts.

The paper's key measure is profit-per-decision. It ties a model's accuracy to the money each decision makes or loses. So a model can look great on accuracy and still be a poor buy.

The authors tested the framework with expert interviews and scenario tests across eleven companies. Read the full paper on Springer. Note that the metric is defined for binary classification (yes or no calls). A chatbot or an agent needs a different yardstick, so use the framework as a way of thinking, not a drop-in formula.

Treat the result as a range, not a single number. The spread tells you how much to trust it.

Fold governance in from day one

Governance is cheaper in the assessment than after launch. It changes both your cost estimate and your acceptance criteria.

Australia's guidance has moved. The Voluntary AI Safety Standard set out 10 guardrails in September 2024. In October 2025 the National AI Centre released the Guidance for AI Adoption, which sets out six essential practices. It's voluntary, and it doesn't replace legal advice. Our post on AI regulation in Australia shows where the hard law sits.

In practice:

• Run the risk check alongside the technical check, not after it.
• Name who is affected by the model's decisions. Put their concerns into the acceptance criteria.
• Put testing and monitoring in the running budget as a line item.

The Federal Government's National AI Plan points the same way: adopt AI, and do it responsibly.

Not sure your business is ready for any of this? Start with our AI readiness checklist. The Orchestrators have a good read on the five levels of AI maturity, and a piece on AI data governance.

Run it in weeks, not months

Keep the team small: a technical lead, a data person and someone who owns the commercial case. Time-box the work.

1. Start: Define the one decision the AI will support. Agree the minimum score. Check the data for volume, labels and source.
2. Middle: Build a light proof of concept, or benchmark against what you already do. Draft the sensitivity table.
3. End: Write the risk register, line it up with the governance guidance, and write the viability statement.

Use existing baselines and off-the-shelf benchmarks. You want evidence here, not new tooling. Our guide to LLM evaluation helps if the build involves a language model.

Staff who need governance skills quickly can look at accredited AI training programs for government agencies.

If the data check fails at the start, stop. Fix the data before you spend a dollar on modelling.

Go, iterate or stop

Score the five checks. The answer falls out.

• Go. Technical, data and performance checks pass, and the numbers hold in a normal scenario. Move to a pilot, then a roadmap.
• Iterate. One part falls short and can be fixed, usually data or scope. Narrow the use case or fix the data, then retest.
• Stop. The minimum score can't be reached with the data you have, or the numbers fail even in the good scenario. Write down what you found and file it.

Show the viability statement with the confidence level attached. Then track the same measures through the pilot: performance against the bar, cost against budget, and adoption.

Planning the cost side? See what it costs to build an AI app from AI-Led, and our post on FinOps for AI.

Where Devwiz fits

We've shipped 200+ apps, including work for NSW Government (Justice and Corrective Services), Briometrix, Vivid and Huskee. AI now runs through how we build and what we build.

A good result is a starting point, not a finished product. Running the assessment in-house works for a narrow pilot. A build partner pays off once you need production infrastructure and someone to own it after launch.

We build AI apps and turn programs into AI platforms. CTOs scoping their own needs can start with our guidance for CTOs. Before launch, run the AI product launch checklist.

Got a viability statement in hand and want to know what building it properly looks like? Worth a chat?

Frequently asked questions

What is an AI feasibility assessment?

It is a short, structured check done before major build spend. It tests if the AI can be built, if your data supports it, and if the returns justify the cost. It ends in a viability statement: build, iterate or stop.

What are the main checks in an AI feasibility assessment?

Five: technical fit, data readiness, model performance against a minimum bar, operational readiness (monitoring and retraining), and the commercial case. Any one of them can sink a project on its own.

How is feasibility different from a discovery phase?

Discovery finds and ranks the problems worth solving. Feasibility takes one chosen idea and tests if it will work and pay. Most teams do discovery first, then run feasibility on the top pick.

What is profit-per-decision?

It is a metric from a 2026 research paper that links a model's accuracy to the money each decision makes or loses. The paper defines it for binary classification, so treat it as a way of thinking for other kinds of AI.

When should you stop an AI project?

Stop if the minimum score can't be reached with the data you have, or if the numbers fail even in the good scenario. Write down what you found and file it. A fast, well-documented stop saves far more than a slow failure.