AI

The AI Development Lifecycle: Stages, Tests and Upkeep

By James KillickNovember 18, 2026

TL;DR: The AI development lifecycle takes an AI system from a defined business problem to a monitored product and, later, to retirement. Unlike normal software, the work continues after launch because model behaviour changes with data and usage. Budget for testing, monitoring and retraining from day one.

The AI development lifecycle is the set of stages that takes an AI system from a business problem to a monitored, maintained product, and later to retirement. It differs from a normal software build in one way that matters to founders: the work does not stop at launch. Model behaviour shifts as data and usage shift. Plan the budget, the team and the contract around that, and the build is far less likely to stall at demo stage.

We build AI software for Australian businesses. This is the lifecycle we plan around.

What the lifecycle covers

It runs from "should we use AI here?" to switching the system off. The Digital Transformation Agency's AI technical standard names three stages:

  • Discover: design, data, train and evaluate.
  • Operate: integrate, deploy and monitor.
  • Retire: decommission.

It is written for government agencies, but the shape suits any business. The government's pilot AI assurance framework is more detailed. It adapts the OECD lifecycle, starts at early experimentation and ends at retirement. It also says the phases are often iterative, not strictly in order. Retirement can be called at any point while the system is running.

Adding one AI feature to an existing app? You still run this lifecycle, just at feature size. Building an AI-first platform? It is the spine of the whole project.

How it differs from a normal software build

Traditional software is predictable. The same input gives the same output, so a test passes or fails. Generative AI is not like that. A LaunchDarkly engineering post explains that these systems give non-deterministic results, and that testing now covers softer measures such as tone, helpfulness and scope of knowledge.

For a founder or CTO, that changes four things:

  1. Requirements include data. You say what the system must do and what information it learns from or reads.
  2. Testing is statistical. You measure how often it is right, safe and useful. One passing case proves little.
  3. Launch is the start of learning. Real users find behaviour no test set predicted.
  4. Maintenance never stops. Models, prompts and data all change, and each change needs tracking. Our guide to AI model versioning in SaaS covers that habit.

The stages, one by one

1. Problem framing

Define the business problem, who uses the output and what success looks like in numbers. Think time saved per task, resolution rate and error tolerance.

The National AI Centre's implementation guidance asks you to document acceptance criteria and test results for each system. Write them down now, not after the demo.

This is also where you decide to build, buy or wait. Our AI feasibility assessment guide helps you make that call, and our AI product roadmap post shows how to turn the answer into a plan.

2. Data collection and preparation

Data work is usually the longest stage. A Microsoft engineering playbook starts with the problem: agree success criteria, find data sources and check they are available. It then builds a baseline model before anything fancier.

The AI technical standard's data statement adds the controls that protect you later. Measure how representative the dataset is. Document how data is used, so you have an audit trail if bias shows up.

For most business software, the real question is where the information lives and whether the system can reach it reliably. That is a pipeline job, and it sits under every AI feature. We cover the rules in data governance for AI, and the AI Orchestrators team has a good read on AI data governance.

3. Choosing the approach

There is rarely one right answer. The National AI Centre's guidance on choosing a solution says the aim is to choose the simplest option that meets your needs, and add complexity only where the value justifies it. The technical standard's model statement agrees: start with simple, small architectures and add complexity step by step.

For a product feature, the options run like this:

  • A hosted model with good prompts.
  • Retrieval-augmented generation (RAG), where the model answers from your own documents.
  • A custom-trained model.

Each step up adds cost, data needs and running work.

4. Design and architecture

Here the model becomes part of a product: the screens, the data flow, the integrations and the fallback when the model is unsure. Our guides to AI platform architecture and UI patterns for AI cover the layers and how trust shows up on screen.

Security belongs here too, not at the end.

5. Development and integration

Build the feature, connect it to your systems and set up the pipelines that feed it. Integration is where many AI projects slow down. Read our guide to integrating AI platforms before the build starts. Our AI program and platform work uses AI throughout the build, from AI-assisted coding to the AI features inside the product.

6. Testing and validation

This stage proves the system is accurate, reliable and safe enough to release. The Australian Signals Directorate's cyber centre has guidelines for secure AI system development. They say to release models only after appropriate security evaluation, such as benchmarking and red teaming. Red teaming means deliberately trying to make the system misbehave. They also say to be clear with users about known limits and failure modes.

A workable test plan has:

  • A fixed set of real cases with agreed correct answers.
  • Pass marks set before testing, not after.
  • Hostile cases: prompt injection, off-topic requests and attempts to pull out private data.
  • Human review of a sample for tone and helpfulness.
  • Normal app testing and user acceptance testing around the AI feature.

For more on the automated side, see CI/CD for AI and AI agent testing from the AI Orchestrators team.

7. Deployment and launch

Release in a controlled way. Start with a small group of users, switch logging on and keep a fast route back. Version the model, the prompts and the retrieval data together. Then you can tell which change caused which result.

8. Monitoring, improvement and retirement

The technical standard's monitoring statement asks agencies to monitor performance and AI drift against pre-defined metrics. It also covers system health, safety and security. Drift means accuracy slips as real-world data moves away from what the model was built on. We explain it in AI model drift.

Plan the end as well. Retirement means decommissioning the model, archiving or deleting data and telling users. Our notes on post-launch AI platform support show what good upkeep looks like.

A worked example

Take a SaaS founder who wants an AI assistant that answers customer questions from the company's help content.

  • Frame: resolve a set share of tickets without a person, with handover for the rest. The head of support owns the result.
  • Data: help articles, past tickets and product docs are cleaned, de-duplicated and stripped of personal details.
  • Approach: a hosted model with RAG over the help content, not a custom-trained model.
  • Build and test: wire it into the app. Test it on real past questions, plus attempts to pull out other customers' data.
  • Launch: 5% of users first, with thumbs-up and thumbs-down logging.
  • Operate: review failed answers weekly, update documents when the product changes and re-index when accuracy falls below the agreed mark.

Launch sits in the middle of that list. Most of the value, and most of the cost, comes after it.

Australian rules to build in

These are not extra stages. They sit inside the ones above.

  • Privacy. If your system handles personal information, the Privacy Act 1988 and the Australian Privacy Principles apply. That covers what you feed the model and what it produces.
  • Voluntary AI Safety Standard. The government describes it as guidance to help businesses adopt AI safely. It has also put forward mandatory guardrails for high-risk AI as a proposal. Building those habits now avoids rework. For a plain-language view, see this overview of Australia's voluntary standard.
  • AI management standard. AS ISO/IEC 42001:2023 is the Australian-adopted standard for running an AI management system. It sets requirements for risk, data and ongoing performance.

If your product handles health, financial or other sensitive information, talk to a lawyer or privacy adviser about your own obligations before launch.

Plan the whole lifecycle at the start

Founders usually ask how long the build will take. The better question is who owns the system in month six.

We have shipped more than 200 apps. A demo shows a model can do something once. A platform has to do it every day for real customers. That gap is the lifecycle. Data pipelines, test sets, versioning and monitoring turn a prototype into something a business can run on.

So scope the whole lifecycle at the start, even if you fund it in phases. It keeps the budget honest and the roadmap real.

If you are working out what an AI build involves, we take you from idea validation and product strategy through design, development, testing, launch and support. See our AI app development work, which includes projects for NSW Government, Briometrix, Vivid and Huskee, or talk to us about your project.

Frequently asked questions

How many stages are in the AI development lifecycle?

It depends on who draws the model. The Australian Government's AI technical standard uses three: Discover, Operate and Retire. Working teams often split those into six to eight phases. The phases overlap and repeat.

Is the AI software development lifecycle the same as the SDLC?

It builds on it. You still plan, design, build, test, release and maintain. You add data preparation, statistical testing, drift monitoring and retraining. Because model outputs vary, testing and upkeep never really stop.

Where do most AI projects go wrong?

Usually before any model is chosen: an unclear success measure, or data that is hard to reach, unrepresentative or messy. Start with the simplest approach and set acceptance criteria early.

Do I need to follow Australian rules when building AI?

If your system handles personal information, the Privacy Act 1988 and the Australian Privacy Principles apply. The Voluntary AI Safety Standard is guidance, and mandatory guardrails for high-risk uses have been proposed. For your own product, check with a privacy adviser.

About James Killick

10+ years building digital products · 200+ apps shipped since 2015

James is a co-founder of Devwiz and an AI product specialist. Since 2015 he has helped ship 200+ apps for founders, businesses and government, including work for NSW Government, Briometrix and Huskee. He builds AI-first platforms and writes about turning a proven program into software. He also hosts the Up in the AI podcast.

More articles by James · James's personal site · LinkedIn · AI Orchestrators

Tags: AI Development Lifecycle, AI Platforms, Testing, Australia

Browse all Devwiz articles·See our case studies