Adding AI Features to a Mobile App: On-Device vs Cloud

"Add AI" now appears in a lot of product briefs, usually without much detail. The result is often a chat screen bolted onto an app that didn't need one. The AI features in mobile apps that people actually use tend to be smaller and more specific: a summary of a long thread, a form filled from a photo, search that understands a sentence, a draft reply ready to edit.

Once you know the job, the big architectural question is where the model runs: on the phone, or in the cloud. That choice decides privacy, cost, latency, offline behaviour and which devices get the feature at all. This guide covers how to pick the job, the on-device and cloud options in 2026, a safe architecture, the store rules that apply, cost per user and how to check quality before launch.

Key takeaways

• Start from a job users already do, not from a model. Summarise, extract, classify, search and draft are strong mobile patterns.
• On-device models are private, free per request and work offline, but run only on recent devices and are less capable.
• Cloud models are more capable and reach every device, but cost money per request, need a network and send data to a third party.
• Never put model API keys in the app. Call cloud models through your backend or a service with app attestation.
• Disclose and get permission before sending personal data to third-party AI, which Apple's guideline 5.1.2(i) now names explicitly.
• Evaluate with real inputs before launch, and keep measuring after.

Start from a job, not a model

A good AI feature has a clear input, an output users can check at a glance, and a graceful fallback when the model gets it wrong.

On-device vs cloud

Many apps end up hybrid: on-device for private, quick tasks where it's available, cloud for heavier work and for devices without on-device models.

On-device options in 2026

• Apple's Foundation Models framework (iOS 26 and later) gives apps access to the on-device model behind Apple Intelligence, with guided generation into typed Swift structures and tool calling. It only works on Apple Intelligence-capable devices with the feature enabled, so always check availability and provide a fallback.
• Android's ML Kit GenAI APIs run Gemini Nano through AICore, Android's system service for on-device models, with APIs for summarisation, proofreading, rewriting, image description and custom prompts on supported devices from Google, Samsung and other manufacturers.
• Open models in your app through runtimes such as ExecuTorch or llama.cpp, at the cost of a much larger download and more memory. React Native ExecuTorch, from Software Mansion, wraps Meta's ExecuTorch runtime in React hooks for running language, vision and speech models on the device; it suits features that must work offline or keep data private, on phones with enough memory.
• Classic machine learning is often enough. Text recognition, barcode scanning, image labelling and face detection from Apple's Vision framework and ML Kit are mature, fast and free.

In React Native, you reach the platform models through native modules: a community library or a small Expo module of your own.

Cloud: a safe architecture

The most common mistake is calling a model API straight from the app with an embedded key. Anyone can extract that key from the app bundle and run up your bill. Route requests through your backend, where you authenticate the user, limit usage and keep the key:

import Anthropic from "@anthropic-ai/sdk";

const anthropic = new Anthropic(); // reads ANTHROPIC_API_KEY on the server

// requireUser and limitPerUser: your auth and per-user rate limiting
app.post("/ai/summarise", requireUser, limitPerUser, async (req, res) => {
  const text = String(req.body.text ?? "").slice(0, 8000);
  const message = await anthropic.messages.create({
    model: "claude-haiku-5-5",
    max_tokens: 300,
    system: "Summarise the user's notes in three short bullet points. Plain text only.",
    messages: [{ role: "user", content: text }],
  });
  const block = message.content[0];
  res.json({ summary: block.type === "text" ? block.text : "" });
});

If you don't run a backend, Firebase AI Logic offers client SDKs for Gemini models protected by Firebase App Check. Either way:

• Use the smallest model that does the job well, and route only hard cases to larger ones.
• Set timeouts and fallbacks, so a slow provider degrades the feature, not the app.
• Stream long responses so users see progress; in Expo apps, expo/fetch supports streaming response bodies.
• Cache results where inputs repeat, such as summaries of the same document.

Store rules and safety

• Apple's guideline 5.1.2(i): "You must clearly disclose where personal data will be shared with third parties, including with third-party AI, and obtain explicit permission before doing so." Ask clearly, at the point of use, and respect a "no".
• Google Play's AI-Generated Content policy states that developers "are responsible for ensuring that their generative AI apps do not generate offensive content." Filter outputs, and give users a way to report bad results inside the app.
• Label AI output so users know to check it, and keep a human decision in the loop for anything that matters: money, health, legal or safety.
• Update your privacy declarations (Apple's privacy details, Google Play's Data safety form) to cover what you send to AI providers.

Cost per user

Cloud AI costs scale with usage, so model them per user before launch:

Monthly cost per user ≈ requests per user × (input tokens × input price + output tokens × output price)

Then make it predictable: cap requests per user per day, trim inputs to what the model needs, cache repeated work, prefer small models, and tie heavier features to paid plans. Watch the real numbers weekly after launch, because usage patterns rarely match the spreadsheet.

Evaluate before you ship

AI features fail differently from ordinary code: they mostly work, then fail in surprising ways. Before launch:

1. Collect 50 to 200 real inputs, including messy and adversarial ones.
2. Define what "good" means for this feature, and score outputs against it, with people for subjective tasks.
3. Re-run the set whenever you change the prompt or model, like a regression test.
4. Add feedback in the app (thumbs up or down, "report a problem") and review it weekly.

From the field: One of the products I've built is an AI trading assistant on Next.js, Python and PostgreSQL, designed with guardrails and no auto-execution: the model suggests, a person decides. That principle carries over to mobile features. Let AI draft, summarise and suggest, and keep consequential actions behind a human tap.

FAQ

Should AI features run on-device or in the cloud?

Run on-device when privacy, offline use or zero per-request cost matter and the task is focused, such as summarising or rewriting short text, on devices that support it. Use the cloud for more capable models and full device coverage. Many apps combine both, with a cloud fallback for unsupported devices.

Can I call OpenAI or Claude directly from my mobile app?

You shouldn't with an embedded API key, because keys can be extracted from the app. Send requests through your own backend, which authenticates users, limits usage and holds the key, or use a service designed for client access with app attestation, such as Firebase AI Logic for Gemini.

Do I need to tell users about AI features?

Yes. Apple's guideline 5.1.2(i) requires clear disclosure and explicit permission before sharing personal data with third-party AI, and both stores require accurate privacy declarations. Labelling AI-generated output also helps users judge it.

How much do AI features cost to run?

On-device models cost nothing per request. Cloud models charge per token, so cost depends on requests per user and the size of inputs and outputs. Estimate cost per user before launch, then cap usage, trim inputs, cache results and use small models where they're good enough.

Next steps

If you're planning an AI feature for your app and want it useful, safe and affordable, I can help you choose the job, the model and the architecture. Get in touch, or see my services.

Sources

• Foundation Models, Apple
• ML Kit GenAI APIs, Google
• Firebase AI Logic, Firebase
• App Review Guidelines (5.1.2), Apple
• AI-Generated Content policy, Google Play Console Help
• Expo (expo/fetch), Expo documentation