The quickest way to prototype an LLM feature in a mobile app is to paste the provider's API key into the client and call the API directly. It works in an afternoon. It is also the one shortcut you have to undo before release, because anything in an app binary can be extracted, and a leaked key is a bill someone else runs up in your name.

The fix is a small server that sits between your app and the provider. This post covers what that proxy has to do, what it should refuse to do, and how to keep it small enough that one person can run it.

Why the key cannot live in the app

Obfuscation, splitting the key into pieces, or fetching it from remote config at launch all fail in the same way: the key ends up in memory on a device you do not control, and it goes out over a connection the device owner can inspect. Once someone has it, they can call any model the key allows, with any prompt, at any volume.

You also lose control in less dramatic ways. With the key in the client:

  • Rotating the key requires an app update, and old versions break.
  • Prompts are baked into shipped binaries, so fixing a bad prompt takes a release.
  • You cannot limit one user without limiting everyone.
  • You have no single place to see what the feature costs per user.

The five jobs of the proxy

A useful proxy is not a general-purpose gateway. It does five things.

1. Authenticate the caller

Every request needs an identity you can count against. If your app has accounts, use the session token you already issue. If it does not, issue an anonymous token per install: the app registers once, your server returns a signed token, and the app stores it in the keychain or keystore.

Anonymous tokens can be farmed by a script that registers repeatedly. Platform attestation raises the cost of that: App Attest on iOS and the Play Integrity API on Android let your server check that a registration came from a genuine copy of your app. Treat it as one signal among several, and gate registration with it, not every request.

2. Expose tasks, not a raw model endpoint

This is the decision that matters most. If your proxy forwards whatever JSON the client sends, you have built a free LLM API for anyone with a valid token. Instead, the client sends a task name and the user's input, and the server builds the provider request: system prompt, model, output token cap and all.

const TASKS = {
  summarize_note: {
    maxInputChars: 8000,
    buildRequest: (input) => ({
      model: SUMMARY_MODEL,
      max_tokens: 300,
      system: SUMMARY_PROMPT,
      messages: [{ role: 'user', content: input }]
    })
  }
};

The client cannot choose the model, raise the token cap or replace the system prompt. A stolen token is worth only as much as your narrowest feature. As a bonus, prompts now live on the server, so you can fix them without an app release. If you version them the way we described for treating prompts like code, a prompt change becomes a normal deploy.

3. Enforce limits before spending money

Check everything cheap before calling the provider:

  • Input size. Reject oversized input with a clear error. The client should enforce the same limit in the UI so real users never see it.
  • Per-user quota. A counter per user per day is enough to start. Store it wherever is easiest: a key-value store with expiring keys, or a database row.
  • Short-window rate limit. A few requests per minute per user stops loops and retry storms, including the ones caused by your own bugs.
  • Global ceiling. A daily cap across all users, set in config, that switches the feature to a polite "try again later" state. It should never fire, and you will be glad of it the day it does.

Pick the numbers by measuring. Log requests per user for a week of beta, look at what your heaviest legitimate users do, and set the quota comfortably above that. Our post on keeping LLM feature costs predictable goes further into budgets and tiering.

4. Forward the request and pass the response through

The proxy adds the provider key from a server-side secret, calls the provider and returns the result. If the feature streams, pipe the response body through instead of buffering it; most serverless and edge runtimes let you return the upstream body directly.

export default {
  async fetch(req, env) {
    const user = await verifyToken(req.headers.get('Authorization'), env);
    if (!user) return new Response('unauthorized', { status: 401 });

    const { task, input } = await req.json();
    const spec = TASKS[task];
    if (!spec) return new Response('unknown task', { status: 400 });
    if (typeof input !== 'string' || input.length > spec.maxInputChars) {
      return new Response('input too large', { status: 413 });
    }
    if (!(await underQuota(user.id, env))) {
      return new Response('quota exceeded', { status: 429 });
    }

    const upstream = await fetch(env.LLM_URL, {
      method: 'POST',
      headers: providerHeaders(env.LLM_API_KEY),
      body: JSON.stringify(spec.buildRequest(input))
    });

    return new Response(upstream.body, {
      status: upstream.ok ? 200 : 502,
      headers: { 'Content-Type': upstream.headers.get('Content-Type') ?? 'application/json' }
    });
  }
};

This is a sketch, not a drop-in file: verifyToken, underQuota and providerHeaders are yours to write, and the request shape depends on your provider. The order of checks is the part to copy: identity, task, size, quota, and only then the paid call.

Do not relay provider errors to the client verbatim. Map them to a small set of your own codes, such as busy, quota, too_large and unavailable, so the app can show a sensible message for each and you can change providers without changing the client.

5. Record what happened, minus the content

For each request, log the user ID, task name, model, input and output token counts as reported by the provider, latency and outcome. That is enough to answer the questions you will have: what each task costs, who the heavy users are, and how often requests fail.

Leave the prompt and response text out of the logs by default. User content in logs is a privacy liability and rarely helps with operational questions. If you need samples to debug quality, make that a separate, deliberate, opt-in path with a short retention period.

Where to run it

The proxy is stateless apart from the counters, so it fits most hosting options. Choose based on what you already operate:

  • Edge or serverless functions. No server to patch, and streaming pass-through is usually simple. Check the platform's limits on request duration, because long generations can exceed the defaults.
  • Your existing backend. If you already run an API with auth, add one route. You get your current user model and database with no new moving parts.
  • A small VPS. Fine if you already run one, but you take on uptime and TLS yourself.

Whichever you pick, keep the provider key in the platform's secret store, not in the repository, and use a key dedicated to this app so you can rotate it without touching anything else.

Mistakes that undo the work

  • Accepting a model or system field from the client "for flexibility". This reopens the hole the proxy was meant to close.
  • Retrying on the server and on the client. Two layers of retries multiply each other. Retry in one place, with a limit.
  • Counting quota after the call. Concurrent requests will all pass the check. Increment first, and refund on failure if you want to be generous.
  • No kill switch. Keep a config flag per task that disables it without a deploy, and make sure the app handles the resulting error gracefully.
  • Treating the proxy as the place for business logic. Once it starts post-processing, caching and personalising, it is no longer thin. Add those when you have a measured reason.

A pre-release check

  1. Search the app binary and the repository for the provider key. It should appear in neither.
  2. Call the proxy with no token, an expired token and an unknown task name. Each should fail without reaching the provider.
  3. Send input over the size limit and confirm it is rejected.
  4. Exhaust a test user's quota and check what the app shows.
  5. Flip the kill switch and check the app again.
  6. Rotate the provider key and confirm that nothing but a server secret changes.

Conclusion

A proxy for an LLM feature is a small amount of code with a large effect: the key stays on your server, the client can only ask for the tasks you defined, every request is tied to a user and a limit, and you have one place to change prompts, models and providers. Build it before the first public build, keep it narrow, and resist adding anything that is not one of the five jobs until your logs show that you need it.