Most AI features start the same way. Someone wires a text box to a model API, the demo works, and the feature ships. Weeks later someone asks a simple question: where does the text users type actually go? Nobody on the team can answer it with confidence.
A small team can answer that question properly. You don't need a compliance department. You need to know where the data goes, send less of it, and tell users the truth about it. This post is the checklist we use before an LLM-backed feature goes out.
Step 1: Draw the data flow before you write the prompt
Get a sheet of paper or a text file and follow one user request from start to finish. For each hop, write down what data is present and who can read it. A typical cloud-backed feature looks something like this:
- User types or speaks input on the device.
- The app adds context: the user's name, previous messages, document contents, location, settings.
- The request goes to your backend, or straight to the model provider.
- Your backend logs the request (often by accident, through a framework's default request logging).
- The provider processes it and may keep it for some period under its own terms.
- The response comes back, gets shown, and may be cached or stored in your database.
- Error reporting and analytics tools may capture the request or response when something fails.
Steps 4 and 7 are where surprises usually turn up. Crash reporters and request loggers are good at capturing whatever was in memory, and that now includes user prompts. If you followed our advice on crash reporting and triage, check that your breadcrumbs and attached payloads leave out prompt text.
Step 2: Read your provider's data terms, then write down the answers
Provider terms vary, and they change. Don't rely on what you remember. Look up the current terms for the exact API and account type you use, and record the answers in your repo next to the integration code:
- Retention: Are inputs and outputs stored? For how long? Can you request shorter retention or zero retention, and does that need a specific plan or agreement?
- Training: Is API data used to train models by default? Is there an opt-out, and is it on for your account?
- Region: Where is the data processed? This matters if you have users in places with data residency expectations.
- Sub-processors: Does the provider pass data on to other parties?
- Abuse monitoring: Even with zero-retention options, some data may be kept for safety review. Find out the conditions.
A short PRIVACY.md in the module that calls the model works well. When someone changes providers or models, the file reminds them to check the answers again.
Step 3: Send less
Minimization protects users more than anything else on this list, and it often makes requests cheaper and faster too. For every field in your prompt context, ask whether the model actually needs it to do this task.
- Names and identifiers: A summarizer rarely needs to know who the user is. Leave out account IDs, emails, and display names unless the output really depends on them.
- History: Send the last few relevant turns, not the whole conversation. Smaller context also helps with keeping LLM feature costs predictable.
- Attachments: If the feature works on a selected paragraph, send that paragraph, not the whole document.
- Metadata: Device model, precise location, and timestamps often slip in through generic "context" objects. Build the prompt payload explicitly instead of serializing a whole app state object.
That last point is a code habit. Make the prompt builder take only the fields it needs as named parameters:
// Avoid: easy to leak fields added later
func buildPrompt(state: AppState) -> String
// Prefer: the signature lists exactly what leaves the device
func buildSummaryPrompt(selectedText: String, tone: Tone) -> StringStep 4: Redact what you can before sending
Users paste things. Email addresses, phone numbers, card numbers, and API keys end up in text boxes all the time. A simple redaction pass before the request goes out catches the obvious cases. It won't catch everything, so don't present it as a guarantee, but it cuts down what ends up in third-party logs.
import re
PATTERNS = [
(re.compile(r"[\w.+-]+@[\w-]+\.[\w.-]+"), "[EMAIL]"),
(re.compile(r"\b(?:\d[ -]?){13,19}\b"), "[NUMBER]"),
(re.compile(r"\+?\d[\d ()-]{7,}\d"), "[PHONE]"),
(re.compile(r"\b(sk|pk|api|key)[-_][A-Za-z0-9]{16,}\b"), "[SECRET]"),
]
def redact(text: str) -> str:
for pattern, label in PATTERNS:
text = pattern.sub(label, text)
return textThe trade-off: redaction can make outputs worse when the redacted value actually matters. A feature that drafts a reply to an email may need the sender's address. Decide per feature, and run your eval cases with redaction switched on so you can see the effect. If you don't have eval cases yet, building a small eval set is a good first step, and you can add a few inputs full of personal data while you're at it.
Step 5: Control your own logs
A lot of the time, the provider isn't the biggest privacy risk. Your own logging is. Check these spots:
- Request logging middleware: Turn off body logging on the routes that proxy model calls, or mask the prompt field.
- Debug logging: Lines like
print(prompt)that were useful during development tend to survive into production. Search for them before release. - Stored responses: If you keep outputs so users can see their history, apply the same retention and deletion rules as for any other user content. When a user deletes their account, their AI history goes too.
- Quality review: If you want real prompts to improve the feature, make it opt-in, keep a sample instead of everything, set an expiry, and limit who can read it.
A useful rule: log the shape of a request, not its contents. Token counts, latency, model name, feature name, and error codes are enough to debug most problems and to keep an eye on cost.
{
"feature": "summarize",
"model": "provider-model-name",
"input_tokens": 812,
"output_tokens": 140,
"latency_ms": 1930,
"redactions": 2,
"status": "ok"
}Step 6: Consider on-device for the sensitive parts
On-device models have become a realistic choice for some tasks: classification, short rewrites, extraction, and simple summaries. They aren't a cure-all. They're smaller, quality is lower on harder tasks, and availability depends on the device and OS version. They do change the privacy situation, though, because the text never leaves the phone.
One middle path is to split the work. Do the sensitive step on the device (for example, pull structured fields out of a private note) and send only the reduced, less sensitive result to a cloud model for the harder step. Measure quality for both approaches with the same eval set before you commit. And plan a fallback for devices that can't run the local model: either turn the feature off there or ask the user before falling back to the cloud.
Step 7: Tell users plainly, at the point of use
A privacy policy update is necessary, but on its own it won't do. Users decide whether to trust a feature at the moment they use it. A good disclosure is short and specific, and it sits next to the feature:
Text you send here is processed by our AI provider to generate a summary. We don't store it after the summary is shown. Avoid including passwords or payment details.
Only write what is true, and check it against the answers you recorded in Step 2. If you store history, say so. If the feature is optional, make it easy to turn off. If it's on by default for text that's already in the app, such as notes or messages, think hard about making it opt-in instead.
Also check your app store privacy declarations. Sending user content to a third-party processor may change what you have to declare. Review those forms every time you add or change an AI feature.
The pre-ship checklist
- Data flow written down, including logging and error reporting hops.
- Provider retention, training, and region terms checked and recorded in the repo.
- Prompt builder takes explicit fields only, with no serialized app state.
- Redaction pass in place, and evals run with it on.
- Prompt bodies excluded from server logs, crash reports, and analytics.
- Stored AI outputs covered by account deletion.
- On-device option considered for the most sensitive step.
- In-context disclosure written, accurate, and visible.
- Store privacy declarations reviewed.
Wrapping up
Privacy work on AI features comes down to knowing where the data goes and deliberately sending less. Most of the checklist takes an afternoon: tracing one request, tightening a function signature, switching off a logger, and writing two honest sentences in the UI. Do it before launch, while the integration is still small. Then repeat it whenever you change models or providers, because every change can affect the answers you wrote down.