Software testing explained: what it is and why it matters

TL;DR: Software testing checks that software works before customers find out it doesn't. It runs across four levels (unit, integration, system, acceptance), splits into functional and non-functional types, and works best when it starts at requirements instead of the week before launch. AI features break the old rules, because the same input can give you a different answer twice.

Software testing is how you check that software does what it should before your customers find out it doesn't. It covers two things: running the code to see what happens, and reviewing the design and requirements to catch problems before anyone writes a line. ASTQB sets the industry definition, and it stretches from reading a requirements doc for gaps right through to a full regression run the night before release.

Two words get mixed up all the time. Verification asks: did we build it right? Validation asks: did we build the right thing? You need both. A payment form can match the spec perfectly and still charge in the wrong currency.

Testing does three jobs:

• Cuts risk. Bugs found before production cost far less than bugs found after.
• Finds defects. Structured testing catches failures that code review alone misses.
• Helps you decide. It tells you whether to ship, hold, or fix.

What software testing actually covers

There are four terms people use as if they mean the same thing. They don't.

Static vs dynamic. Static testing doesn't run the code. Peer reviews, walkthroughs, and tools like SonarQube or ESLint. A developer reading a pull request for logic errors is doing static testing. Dynamic testing runs the software and watches what it does. Pushing a login flow through Selenium is dynamic.

Testing vs QA. QA is about process. It sets standards and stops defects getting into the build. Testing is about execution. It finds defects in what got built. You can have great QA and still need hard testing.

Testing vs debugging. Testing finds the failure. Debugging finds the cause and fixes it. A tester files the crash report. A developer traces it to a null pointer.

The SWEBOK guide makes one point worth repeating: test planning should start when you write requirements, not when development finishes.

Why testing matters to your business

A bug caught in development is cheap. The same bug in production touches more people, corrupts more data, and reaches more customers before anyone fixes it.

Then there's your reputation. A broken payment gateway, a data leak, or an app that crashes on the most common phone in Australia all turn into bad reviews and support tickets. Sometimes worse.

For Australian businesses there's a legal side too. The Privacy Act 1988 and Australian Consumer Law both set rules on how you handle data and whether your product is fit for purpose. Security and data testing is how you show you took that seriously.

Different people use test results for different calls:

• Product managers prioritise the backlog by defect count and severity.
• Release managers approve or block a deployment on pass rates.
• CTOs read coverage to judge technical risk before scaling.
• Business owners sign off on acceptance results.

The Ministry of Testing puts it well. Testing is an investigation that gives people the information to make a call. It's not a pass/fail stamp at the end.

The four levels of testing

Testing levels stack up from one function to the whole product in a user's hands.

Unit tests are the base. Call one function with known inputs, check the output. If your discount calculation is wrong, this catches it before anything else sees it.

Integration tests check the joins. An API endpoint can handle its own logic fine and still break when it calls a payment service.

System tests treat the app as a black box and run it against the requirements. A tester walks the whole checkout, including expired cards and dodgy addresses.

Acceptance tests are the last gate. Real stakeholders using the real product. UAT sessions surface usability problems no automated test would ever think to look for.

Here's the thing. Each level catches a different kind of bug. Unit tests run in milliseconds and give developers fast feedback. System tests take longer but prove the whole thing works. Skip a level and you get a blind spot.

The types you will meet

Testing splits into two families: functional and non-functional.

Functional testing checks what the software does against the requirements.

• Smoke testing runs a thin slice of critical paths after every build. Can you log in? Does the homepage load? If smoke fails, stop and fix.
• Regression testing re-runs old tests after a change to confirm you didn't break something that used to work.
• Sanity testing is a quick focused check on one fix, rather than the full suite.
• User acceptance testing puts real users on the real flows before launch.

Non-functional testing checks how well it does it.

Performance testing measures response times and stability under load. Practical Australian example: you want to know the platform holds 10,000 shoppers before the Black Friday sale, not during it.

Security testing covers penetration tests, vulnerability scans, and auth checks. For apps holding personal data this feeds straight into your Privacy Act obligations. OWASP ZAP and Burp Suite are the usual tools.

Usability testing watches real people try to finish a task. It's qualitative, and it finds friction no tool will flag.

Accessibility testing checks standards like WCAG 2.1. That matters for any Australian government or public sector work under the Digital Service Standard. Axe and Lighthouse automate part of it.

Compatibility testing confirms it works across browsers, devices, and operating systems. An app that looks great in Chrome and breaks in Safari on iOS loses you a big slice of Australian mobile users.

Manual or automated?

You need both. They solve different problems.

Manual testing wins when human judgement is the point. A tester poking at a new feature without a script finds edge cases nobody scripted. It's also right for one-off checks, tests that rarely run, and anything where "good" is subjective.

Automated testing wins on speed and repetition. A regression suite that takes a person two days runs in minutes in a pipeline.

A simple rule. If a test runs more than once a week, the answer is yes or no, and scripting it takes under four hours, automate it. If it runs monthly, needs judgement, or sits on a UI that changes weekly, keep it manual.

Automate these first:

• Smoke tests on critical paths: login, checkout, core API endpoints.
• Regression tests on stable, high-traffic features.
• API endpoints with a settled contract.
• Data validation checks that run every build.

If you build on infrastructure as code, IaC testing strategies add cover at the environment level and catch config drift before production does.

The seven steps of the testing lifecycle

The testing lifecycle runs seven phases. Each one has an output and a condition for moving on.

1. Requirement analysis. Testers read the requirements and flag anything unclear or untestable. Output: a list of testable requirements plus questions. You move on once the gaps are answered.
2. Test planning. Decide scope, approach, tools, people, and timeline. Output: a test plan and effort estimate. You move on when stakeholders sign it off.
3. Test case design. Write the cases and prepare the data. Output: a test case repository mapped back to requirements. You move on once cases are peer reviewed.
4. Test environment setup. Stand up the environment, data, and access. Output: a working environment with a smoke test passing. You move on when the environment is stable.
5. Test execution. Run the cases and log what happens. Output: execution results and defect reports. You move on when the planned cases have run.
6. Defect reporting and retest. Developers fix, testers retest, then run regression around the fix. Output: verified fixes and an updated defect log. You move on when no critical defects are open.
7. Test closure.** Review the metrics, write down what you learned, archive the artefacts. Output: a completion report and sign-off.

How to pick what to test

You can never test everything. These techniques decide what gets your time.

Black box treats the software as a sealed unit. You know the inputs and what should come out, not the code inside. Submitting valid and rubbish email formats to check the validation rules.

White box uses knowledge of the code to hit specific branches and paths. A developer writing unit tests that cover every branch in a pricing rule.

Grey box sits between them. Enough internal knowledge to aim better, without full code access. Common on API and integration work.

Boundary value analysis tests the edges, because that's where bugs live. A field that accepts ages 18 to 65 gets tested at 17, 18, 65 and 66. Off-by-one errors hide right there.

Equivalence partitioning groups inputs that should behave the same, then tests one from each group. Same age field: valid, too low, too high. Three tests instead of fifty.

Exploratory testing has no script. The tester designs and runs tests at the same time, learning the system as they go. It's very good at finding the weird interactions nobody planned for.

Pro tip: run exploratory sessions in 60 to 90 minute blocks with a clear brief, like "poke the checkout for edge cases around discount codes". Write down everything you see, not just the failures. Those notes turn into your next set of scripted cases.

Tools teams actually use

Most Australian teams mix open source and paid, depending on support needs and budget.

Unit tests: Mocha (JavaScript), JUnit (Java), pytest (Python), NUnit (.NET). Fast, and they plug into any pipeline.

API testing: Postman for manual and automated work, with a free tier that suits small teams. REST Assured for writing API tests in Java. Karate DSL if you want testing and mocking in one.

UI automation: Selenium WebDriver is the long-standing open source option. Cypress is the modern JavaScript pick with a fast feedback loop. Playwright is getting popular here for cross-browser work.

Mobile: Appium covers iOS and Android and reuses Selenium know-how. XCUITest and Espresso are the native options from Apple and Google when you need deeper platform access.

Performance: Apache JMeter for load and stress. k6 if you want it developer-first and pipeline-friendly. Gatling for high concurrency and better reporting.

Security: OWASP ZAP is free and the one most referenced in Australian compliance work. Burp Suite is the commercial standard, with a free Community Edition.

The open source options in every category are production grade. Paid tools mostly add support, dashboards, and enterprise integrations. For most Australian startups, open source tools with a proper test strategy beat paid tools without one.

Habits worth the effort

Shift left. This is the big one. Get testers involved at requirements, not after the build. When a tester reviews requirements early they catch the ambiguity before a developer builds the wrong thing. It's the same logic behind a proper discovery phase: decide what you're building before you build it.

Put tests in the pipeline. Every commit triggers a test run. Teams doing this catch regressions minutes after the commit that caused them, not days later. GitHub Actions, GitLab CI and Buildkite all handle it.

Manage your test data. This one gets skipped and it bites. Tests running on production data create a privacy problem under the Privacy Act. Use synthetic data, anonymised sets, and dedicated test environments. For anything holding health or financial records that isn't optional.

Tier your regression suite. Not every test needs to run every commit. A fast smoke suite under five minutes on every commit. The full regression nightly or before release. Fast feedback without losing coverage.

Pro tip: tag your test cases by risk and feature area from day one. When a release is tight you can run just the high-risk set and defend the decision, instead of picking tests at random.

If your team builds on Australian cloud infrastructure, keep test environments inside Australian data residency boundaries wherever the production data falls under the Australian Privacy Principles.

Testing AI features is a different job

Normal testing assumes: give it X, always get Y. AI doesn't work that way. The same input can produce different output on different runs, so exact-match assertions stop working. You need range checks, similarity scoring, and human review instead.

What changes with AI:

• Data quality. Your training and inference pipelines need testing as much as your application code. Bad data in, unreliable output out.
• Model regression. Every retrain or version bump needs tests confirming the answers that used to be right still are.
• Non-deterministic output. For chatbots, summarising, and RAG systems, you set acceptable output ranges or score responses with an evaluation model.
• Prompt hardening. Feed it hostile prompts and confirm it handles them safely and stays on brand.
• Drift monitoring. After launch, watch whether performance drops as real-world data shifts away from what the model saw in training.

A working checklist:

• Validate input data pipelines before you test the model.
• Run model regression after every retrain.
• Test adversarial and edge-case prompts for safety.
• Monitor output quality and latency in production.
• Write down your evaluation criteria before testing starts, not after.

This is where a lot of AI products come unstuck. A chatbot that tested fine degrades in production as real user inputs drift. A RAG system that answers well today returns stale answers next month if nobody maintains the knowledge base. It's the same failure pattern behind why product-led growth stalls for vibe-coded products: the demo works, the product doesn't hold. Worth reading alongside the honest take on whether you can really build a SaaS without writing code.

Enterprise builds add their own weight. Our CARED case study is a national NDIS allied health platform on the Microsoft stack: four mobile apps, three web platforms, and integrations into healthcare and government systems. When that many moving parts have to agree with each other, testing across the integration surface stops being optional.

Where teams get it wrong

Most teams understand testing in theory. The gap is how early and how consistently they do it. The usual pattern treats testing as a phase after development, so defects pile up and surface late, when they're expensive and messy.

Teams that get it right treat testing as something that runs continuously, not a gate before launch. They build monitoring into the deployment. They run model regression on every retrain. They define what "good output" means before testing starts. The same discipline shows up in how strong teams run delivery day to day.

For founders and CTOs building on AI, testing isn't a cost centre. It's what makes the platform trustworthy enough for customers to bet on.

How Devwiz handles testing

We build testing into delivery rather than bolting it on at the end. Automated and manual testing across web, mobile and API layers. For AI features that means model regression, prompt hardening checks, and output evaluation.

We've shipped 200 plus apps for Australian businesses, including work for the NSW Government in Justice and Corrective Services, Briometrix, Vivid and Huskee. If you want software that's tested properly from the start, have a look at our AI app development or custom software development work. If you're the one carrying the technical risk, our page for CTOs covers how we work.

Testing a product before launch is one question. Testing whether the market wants it is another, and we've written separately on AI product market fit testing and on running a proper beta test.

Frequently asked questions

What is software testing in simple words?

It is checking that software works properly and does what users need. You do that by running the software and by reviewing its design, so you find the problems before your customers do.

What are the seven steps of software testing?

Requirement analysis, test planning, test case design, test environment setup, test execution, defect reporting and retesting, then test closure. Each step has its own output, like a test plan, a set of test cases, or a completion report.

Can you learn software testing in two months?

You can learn the fundamentals in two months: core concepts, writing manual test cases, and finding your way around the common tools. Getting good at automation usually takes another 6 to 12 months of steady practice.

Is software testing a difficult job?

The barrier to entry is lower than development, but the job needs sharp analytical thinking and clear writing. You have to understand a system well enough to find its weak points. Automation testing adds scripting on top, which lifts the technical bar.

What is the difference between QA and software testing?

QA is about process. It sets the standards that stop defects getting in. Testing is about execution. It finds the defects in what already got built. They work together but they are not the same job.

How is testing AI features different?

Normal tests assume the same input always gives the same output. AI does not work that way. Instead of exact matches you use acceptable output ranges, similarity scoring, or an evaluation model, plus regression tests after every retrain and drift monitoring once it is live.