A chat-style LLM feature can get away with returning prose. Most of the features small apps actually ship can't. A "suggest tags" button needs a list of tags. A "turn this note into a task" feature needs a title, maybe a due date, and a priority. A receipt scanner needs line items and a total. In each case your code, not a person, reads the model's reply first, and your code is much less forgiving than a person.

This post is about that handoff: getting structured output from a model and turning it into data your app can trust. The core idea is simple. Treat model output the way you treat any untrusted input from the network. Define what valid looks like, check it every time, and know what you'll do when it fails.

Design the schema before the prompt

Start by writing the type your app wants to receive, then work backwards to the prompt. A schema that's easy for a model to fill in is usually also easy for you to validate.

  • Keep it small and flat. Every extra field is one more thing that can go wrong. If the UI only shows three fields, ask for three.
  • Use enums wherever you can. "priority": "low" | "normal" | "high" is far easier to validate than a free-text priority.
  • Make "I don't know" a legal answer. If a due date might not be in the text, make the field nullable. Otherwise the model will invent a date to fill the slot.
  • Don't ask for self-reported confidence scores. A number the model gives for its own certainty isn't a calibrated probability. If you need a review signal, use an explicit enum such as "needs_review": true and back it with your own checks.
  • Leave formatting to your code. Ask for an ISO 8601 date, not "next Tuesday" or a localized string. Your app already knows how to display dates.

Here is a schema for a note-to-task feature:

{
  "type": "object",
  "properties": {
    "title":    { "type": "string", "maxLength": 120 },
    "due_date": { "type": ["string", "null"], "description": "YYYY-MM-DD or null if not stated" },
    "priority": { "type": "string", "enum": ["low", "normal", "high"] },
    "tags":     { "type": "array", "items": { "type": "string" }, "maxItems": 3 }
  },
  "required": ["title", "due_date", "priority", "tags"],
  "additionalProperties": false
}

Ask for structure the strongest way available

There are roughly three ways to get JSON out of a model. From strongest to weakest:

  1. Schema-constrained output. Many hosted APIs and some on-device runtimes let you pass a JSON schema and constrain generation to match it. Use this if your provider supports it. It removes most syntax failures, though not semantic ones.
  2. Tool or function calling. You describe a "function" whose arguments are your schema, and the model "calls" it. Even if you never execute anything, the arguments arrive as structured data. If you already use tool calling, see Tool Use for App Developers for the safety side.
  3. Instructions in the prompt. Put the schema and one short example in the prompt and ask for JSON only. This works more often than not, but you have to expect stray prose, markdown fences, or trailing commas.

Whichever one you use, the next steps stay the same. Constrained output narrows the failure modes, but it doesn't eliminate them, and you may switch providers or models later.

Parse in stages: extract, decode, validate, normalize

Split parsing into stages so a failure tells you where it broke. That makes debugging much easier when you look at logs later.

struct TaskSuggestion: Decodable {
    enum Priority: String, Decodable { case low, normal, high }
    let title: String
    let dueDate: String?
    let priority: Priority
    let tags: [String]

    enum CodingKeys: String, CodingKey {
        case title, priority, tags
        case dueDate = "due_date"
    }
}

enum ParseFailure: Error {
    case noJSON
    case decode(String)
    case invalid(String)
}

func parseSuggestion(_ raw: String, allowedTags: Set<String>) -> Result<TaskSuggestion, ParseFailure> {
    // 1. Extract: tolerate prose or code fences around the object
    guard let start = raw.firstIndex(of: "{"),
          let end = raw.lastIndex(of: "}"), start < end else {
        return .failure(.noJSON)
    }
    let data = Data(raw[start...end].utf8)

    // 2. Decode: types and enums
    let s: TaskSuggestion
    do {
        s = try JSONDecoder().decode(TaskSuggestion.self, from: data)
    } catch {
        return .failure(.decode(error.localizedDescription))
    }

    // 3. Validate: rules the type system can't express
    let title = s.title.trimmingCharacters(in: .whitespacesAndNewlines)
    guard !title.isEmpty, title.count <= 120 else {
        return .failure(.invalid("title_length"))
    }
    if let d = s.dueDate, !isPlausibleDate(d) {
        return .failure(.invalid("due_date"))
    }

    // 4. Normalize: drop what you can safely drop instead of failing
    let tags = s.tags.map { $0.lowercased() }.filter { allowedTags.contains($0) }

    return .success(TaskSuggestion(title: title, dueDate: s.dueDate,
                                   priority: s.priority, tags: Array(tags.prefix(3))))
}

The same structure works in Kotlin with kotlinx.serialization, or in TypeScript with a schema library. The language doesn't matter much. What matters is keeping the stages separate.

Validation is where the real bugs hide

A reply can be valid JSON with correct types and still be wrong for your app. Write explicit checks for:

  • Ranges. Is the date a real calendar date? Is it within a sensible window of today? Is a quantity positive?
  • References. If the model picks from the user's existing projects, contacts, or tags, check that the item actually exists. Never use a model-supplied ID to look something up without checking it against the current user's data.
  • Consistency. On a receipt, do the line items add up to roughly the total? Is the end time after the start time?
  • Length and content. Cap string lengths. Strip control characters. If the text ends up in a URL, a file name, or a query, escape it the same way you would escape user input.

Decide for each rule whether to fail or normalize. An unknown tag can be dropped quietly. A total that doesn't match the line items probably means the whole extraction can't be trusted.

Retry once, then fall back

When parsing fails, you have three options: repair, retry, or fall back.

  • Repair covers mechanical problems: stripping fences, trimming surrounding text. The extraction step above does this already. Don't build a homemade JSON fixer on top of it, because you'll end up with data you can't reason about.
  • Retry works well for decode and validation failures if you tell the model what went wrong. Send the original request again with a short note, such as Your previous reply failed validation: due_date must be YYYY-MM-DD or null. Return only the corrected JSON. Cap it at one retry. Each retry costs latency and money, and a second failure usually means the input is genuinely ambiguous.
  • Fall back to a UI that doesn't need the model. Open the task editor with the note text in the title field. Show the receipt image with empty fields. The user should still get somewhere useful.

A failed parse should look to the user like a feature that helped less than usual, not like an error. Designing LLM Features That Fail Gracefully covers that side in more depth.

Version the contract

The schema is a contract between your prompt and your app, and both will change. A few habits help:

  • Give each schema and prompt pair a version string, and log it with every request.
  • Make decoders tolerant of extra fields the model shouldn't send but sometimes does, unless your schema tooling already rejects them on the provider side.
  • If the prompt lives on a server and the parser ships in the app, remember that old app versions will keep receiving responses for months. Add fields; don't rename or remove them.
  • When you change models, rerun your test inputs through the full parse pipeline before switching traffic.

Measure the pipeline, not just the model

You can't tell how well this works without counting outcomes. Log one event per request with the stage it reached and, on failure, the rule that failed. Leave out the user's content unless they've agreed to share it. From those events you can compute:

  • Share of requests that parse on the first try
  • Failure counts by stage (no JSON, decode, each validation rule)
  • Retry rate, and how often a retry succeeds
  • Fallback rate, which is the number your users actually feel

When one rule dominates the failures, that's your next prompt or schema fix. Every real input that broke the parser is also a good candidate for your test set. If you don't have one yet, Build a Small Eval Set Before You Ship an LLM Feature walks through setting one up.

A short checklist

  1. Write the app-side type first. Keep it small, use enums, make unknowns nullable.
  2. Use schema-constrained output or tool calling if available. Use prompt instructions only as a last resort.
  3. Parse in separate stages: extract, decode, validate, normalize.
  4. Validate ranges, references, consistency and lengths, not just types.
  5. Retry once with the specific error, then fall back to a non-AI path.
  6. Version the schema and prompt together. Make changes additive only.
  7. Log the outcome of each stage and add real failures to your test set.

Wrapping up

Structured output is where an LLM feature stops being a demo and starts being part of your app. The model will mostly give you what you asked for. Your job is to handle the rest: a clear contract, a parser that knows exactly what failed, and a fallback the user barely notices. None of it is complicated, and it lets you swap prompts, models or providers later without crossing your fingers.