Mobile AI features: what to build and where to run it

TL;DR: The AI features that earn their place in a mobile app fall into five groups: language, vision, personalisation, speech and agents. Run private and instant work on the phone. Send big generative jobs to the cloud. Most good products use both. In Australia, the Privacy Act and ACSC guidance apply from the planning stage, not after launch. Start with one feature tied to a task you can measure, then add more.

Which AI features are worth adding to your app? And should they run on the phone or in the cloud? Those two questions decide most of the cost, the speed and the risk. This guide answers both, then covers the Australian privacy and security rules you need to plan for.

What we mean by mobile AI features

Search for this term and you get two kinds of pages. Most are for people buying a phone. A few are for people building an app. This one is for builders.

In an app, an AI feature is anything that depends on a trained model. The model might sit inside your app, come built into the phone, or sit behind an API. Your user doesn't care which. They care that search gets what they meant, that a photo is read correctly, or that a long thread becomes three lines.

Phone makers have raised the bar. People who use Apple Intelligence or Galaxy AI now expect your app to help in the same way.

Five kinds of feature worth considering

There are dozens of ideas. Most fit one of five groups.

Language: summarise, rewrite, chat and search

This is where most teams start. Summarise a long thread. Draft a reply. Answer questions from your own help content. Let people search in plain sentences. A retrieval setup (RAG) pulls answers from your documents, not the model's memory. That keeps answers tied to your content and cuts made-up ones.

Vision: read what the camera sees

Scan a document. Pull fields off an invoice. Check a photo is usable before upload. Vision suits field service, inspections and logistics, because it kills manual data entry.

Personalisation and recommendations

Rank content. Suggest the next action. These lean on your own behaviour data, so the data pipeline matters as much as the model. Our guide to the main types of AI personalisation in mobile apps goes deeper.

Speech and voice

Transcription, voice commands and dictation. Handy when hands are busy, like on a worksite or in a ute.

Agents that finish the job

An agent does more than answer. It books the appointment, updates the record, raises the ticket. That's the highest-value group. It also needs the most care, because the agent acts with the user's authority. Our guide on how to build an AI agent covers the steps. For the risks, see this piece on AI agent security from our sister brand AI Orchestrators.

On-device or cloud: the call that shapes everything

Every feature runs somewhere. Where it runs changes cost, speed, privacy, offline use and how often you can update.

The phone makers use both. Apple describes on-device processing for personal context, with Private Cloud Compute taking the heavier requests. You can copy that pattern.

What the platforms give developers now

Apple's Foundation Models framework opens up the on-device model behind Apple Intelligence. On Android, Google's ML Kit GenAI APIs bring Gemini Nano to jobs like summarising, proofreading and rewriting.

The upshot: a small, tight feature, like summarising a note, may not need a cloud model or a metered bill any more. Save the cloud for work that needs a bigger model or your own server data. Availability depends on the phone and OS version, so check how many of your users can run the feature and plan a fallback for the rest. Our explainer on on-device AI covers the build side.

A simple rule for splitting the work

1. Does it touch sensitive personal data? Lean on the device.
2. Must it work offline or feel instant? Lean on the device.
3. Does it need a big model or data from your backend? Use the cloud.
4. Will usage be high and each request cheap? Cost it out before you pick the cloud.
5. Both? Run a small model on the phone to prep the request, then send only what's needed.

Australian privacy, security and paperwork

This is where AI features stall late in a project. Build the rules in at the start. It costs far less than bolting them on.

Privacy

The OAIC has guidance for developers of generative AI models and systems. It explains how the Privacy Act applies to collecting, using and disclosing personal information. Its guidance on commercially available AI products matters if your feature calls a third-party model API. There's also an OAIC checklist for choosing an AI product.

If your feature makes decisions about people, read the OAIC's APP 1 guidelines and make sure your privacy policy matches.

For third-party APIs, treat the contract as part of the feature. Get it in writing whether the provider keeps your users' prompts or trains on them. For anything that affects people in a big way, do a privacy impact assessment early. Not legal advice, so talk to a privacy lawyer about your own app.

Security

AI doesn't replace normal mobile security. It adds new risks. The Australian Cyber Security Centre's AI guidance is the place to start, and its guidelines for secure AI system development apply whether you build from scratch or on someone else's service. Four points for a mobile team:

• Prompt injection. Text on a web page, email or document can carry orders aimed at your model. Limit what an agent can do without a tap to confirm.
• Model files. A bundled or downloaded model is a component with supply-chain risk. Know where it came from.
• Logging. Record requests so you can audit misuse, within the privacy limits above.
• Drift. Data and behaviour change. Watch output quality after launch, not only before.

Paperwork

The National AI Centre's guidance for AI adoption stresses governance, risk management and human oversight. Write down what data the feature uses, who owns the IP, which models and vendors are involved, and how it connects to your other systems. A habit of logging each architecture choice with its trade-offs, like a one-paragraph decision log, makes audits and handovers far easier. Our summary of the topic is in AI software compliance explained.

Pick your first feature

A short list done well beats a long one done badly. Use these filters.

1. Tie it to a task you can measure. Time to finish a form. Tickets raised. Searches that end with no result.
2. Check you have the data. Recommendations without clean behaviour data will disappoint.
3. Know what wrong looks like. A bad summary is a nuisance. A bad medical or money answer is a risk. Start at the low-risk end.
4. Plan the fallback. What happens on an older phone, offline, or when the model service is down?
5. Plan adoption. People ignore features they can't find. See our guide to AI feature adoption and user engagement.

A worked example

Take a field-inspection app. Release one adds photo capture with on-device checks that the image is in focus and the right asset is in frame. Personal and site data stay on the phone. Release two sends the confirmed photo and notes to a cloud model that drafts the inspection summary from your templates. The inspector approves it before it saves.

Step one is cheap, fast and private. Step two uses the bigger model only where it earns its cost, and the human sign-off limits the damage from a mistake.

Adding AI to an app you already have

If your product is live, you rarely need a rebuild. The usual path is a thin AI layer: one service that handles model calls, prompts, retrieval and logging, and talks to the app through your existing API. That keeps model changes out of your app store release cycle. Our article on adding AI to an existing app covers the steps. our AI platform integration guide covers the connection points, and AI platform architecture shows how the layers fit. For the integration side from another angle, see this AI API integration guide from AI-Led.

Off-the-shelf tools and no-code builders are fine for prototypes. They run out of road when you need your own data model, your own permissions or ownership of the IP. We weigh that up in custom AI software vs off-the-shelf.

Our take

We've shipped more than 200 apps, and AI now runs through how we build them, from AI-assisted coding to AI features inside the product. The pattern we see most isn't a lack of ideas. It's a demo that wowed a meeting and was never built for real users, real data and real cost.

So before we pick a model, we ask four things. What's the one user outcome this improves? Where does the data live, and who's allowed to use it? What does it cost per thousand uses at the volume you're hoping for? Who owns it after launch? Those answers settle device versus cloud far more than benchmark scores do.

We also keep the first release small. One feature that's fast, private and measurable gives you real usage data. That data tells you what to build next better than any roadmap workshop.

Build mobile AI features with Devwiz

Want a feature scoped, designed and shipped inside a production app? Our team covers discovery through to launch and ongoing support. See our AI app development service, or iOS and Android app development if the mobile product itself is still to be built. Tell us your use case and we'll tell you where we'd start.

Frequently asked questions

What is an AI smartphone?

It's a phone with chips and software built to run machine learning jobs like translation, photo editing and writing help, often on the device. Apple calls its version Apple Intelligence. Samsung calls its version Galaxy AI. For app builders, these features are the baseline users compare your app to.

Can a mobile app use AI without sending data to the cloud?

Yes, for jobs small enough for a local model. Apple's Foundation Models framework and Google's ML Kit GenAI APIs both give developers on-device models. Bigger jobs usually still need a cloud model, so many apps use both.

Do Australian privacy laws apply to AI features in my app?

If your organisation is covered by the Privacy Act, the Australian Privacy Principles apply to personal information your AI feature collects, uses or discloses. The OAIC publishes guidance for developers and for organisations using third-party AI products. For your own app, talk to a privacy lawyer.

Which AI feature should I build first?

Pick one tied to a task you can measure, with clean data on hand and a low cost of error, like summarising or document capture. Ship it small, watch real usage, then add more. Don't start with an agent that takes actions that can't be undone.