If you have added a basic LLM feature to an app, such as summarizing text or drafting a reply, you have probably hit its limit already. The model only knows what you put in the prompt. It can't check today's data, look anything up in your database or take an action for the user. Tool use (also called function calling) is the usual next step. You describe a few functions to the model. The model replies with a structured request to call one of them. Your code decides whether to run it and sends the result back.
That one idea is behind most of what gets called "agents". This post explains how the loop works, how to design tools that are hard to misuse, and where the guardrails belong in a small app.
The loop, in plain terms
Most model providers handle tool use in roughly the same way, though field names differ. One full exchange goes like this:
- You send the conversation plus a list of tool definitions. Each definition has a name, a description and a JSON Schema for its arguments.
- The model replies with either a normal text answer or a tool call: a tool name and a JSON object of arguments.
- Your code validates the call, runs the function and gets a result.
- You send that result back as a new message tied to the tool call.
- The model either calls another tool or writes its final answer.
The important part: the model never runs anything itself. Your code does every action, so you decide what is allowed. Keep that in mind when you design the feature. The model suggests a call, and your app chooses whether to make it.
A tool definition usually looks something like this:
{
"name": "search_notes",
"description": "Search the current user's notes by keyword. Returns up to 5 matches with id, title and a short snippet. Use this before answering questions about the user's own notes.",
"input_schema": {
"type": "object",
"properties": {
"query": { "type": "string", "description": "Keywords, not a full sentence" },
"limit": { "type": "integer", "minimum": 1, "maximum": 5 }
},
"required": ["query"]
}
}The description matters more than you would expect. The model picks tools and fills in arguments mostly from that text, so write it the way you would brief a new teammate: what the tool does, what it returns, and when to use it.
Designing tools that are hard to misuse
The most common mistake is giving the model one very powerful tool, like run_sql or http_request, and hoping the prompt keeps it in line. Prompts aren't access control. Build the safety into the shape of the tools instead.
Make tools narrow and specific
Prefer get_order_status(order_id) over query_database(sql). A narrow tool has a small set of valid inputs, so a bad call can only do a small amount of damage. Narrow tools are also easier for the model to use correctly, because it has fewer decisions to make.
Scope everything to the current user on the server side
Never let the model pass in a user ID. Your tool handler should already know who the user is from the session and filter by it. If the model asks for order 12345, the handler looks for order 12345 belonging to this user and returns "not found" otherwise. That closes off a whole class of prompt-injection attacks where pasted text tells the model to fetch someone else's data.
Separate reading from writing
Read-only tools like search, lookup and list are low risk. Tools that change state, such as creating, deleting, sending or charging, should be fewer, more tightly constrained, and usually need confirmation (more on that below). One simple rule: a write tool should never be able to undo something without the user seeing it happen.
Return small, structured results
Everything a tool returns goes back into the context window. You pay for those tokens, and they can contain text that tries to steer the model. Return only the fields the model needs, cut long text down to snippets, and cap list lengths. If your LLM costs are already a worry, tool results are often where tokens quietly add up. The ideas in Keeping LLM Feature Costs Predictable in a Small App apply directly here.
Validate every call as if it came from the internet
Treat tool arguments as untrusted input, because that's what they are. The model may produce arguments that don't match the schema, invent IDs, or pass values that are valid but make no sense. A thin validation layer in front of every handler catches most of this:
async function runTool(call, session) {
const tool = registry[call.name];
if (!tool) return { error: "unknown_tool" };
const parsed = tool.schema.safeParse(call.arguments);
if (!parsed.success) {
return { error: "invalid_arguments", detail: parsed.error.issues };
}
if (tool.requiresConfirmation) {
return { status: "needs_confirmation", summary: tool.describe(parsed.data) };
}
return tool.handler(parsed.data, { userId: session.userId });
}Some things to note:
- Errors go back to the model as data, not exceptions. A clear error such as
invalid_argumentswith details often lets the model fix its own call on the next turn. - The user ID comes from the session, never from the arguments.
- Unknown tool names are handled explicitly. Models sometimes call tools that don't exist, especially if you renamed one recently.
Put a human in front of consequential actions
For anything that sends, deletes, spends or publishes, don't run it straight from a tool call. Turn the call into a confirmation step in your UI:
- The model calls
create_reminderwith a title and time. - Your app shows a native confirmation card: "Create reminder 'Call dentist' for Thursday 9:00?"
- The action only runs when the user taps Confirm, and the result goes back to the model.
This works well in practice. The user can see exactly what will happen, and you get a natural place to let them edit the values. Mistakes, like the wrong date or a misheard name, show up before they cause harm. Build the card from the validated arguments, not from the model's prose, so what the user approves is exactly what runs.
Bound the loop
A model that can call tools can also call them over and over. Without limits, a confused model can loop: searching, getting nothing useful, and searching again. Set hard limits in your code:
- Maximum tool calls per user request. Choose a small number that covers your real flows. Check it by logging how many calls successful sessions actually make.
- Maximum wall-clock time for the whole exchange, with a polite fallback message when it runs out.
- Per-tool rate limits for anything that hits an external API or costs money.
- Duplicate detection. If the model makes the same call with the same arguments twice in a row, return the cached result or stop the loop.
When a limit is hit, end with a useful message, not a generic error: "I couldn't find that in your notes. Try a different keyword."
Log the whole trace
Debugging tool use without traces is guesswork. For each request, log the tool calls in order: name, validated arguments, a trimmed result, how long each took, and why the loop ended. Leave out or redact personal content under the same rules you use for the rest of your logs. The privacy checklist for AI features covers what's reasonable to keep.
Those traces become your test material. Pick a few real requests, write down which tools should have been called, and check new prompt or model versions against them. Check whether the right tool was called, whether the arguments were sensible, and whether the loop finished within limits. You can measure all of that without judging the wording of the final answer.
When tool use is the wrong choice
Not every feature needs a model choosing actions. If the flow is always the same, such as "take this text, extract a date, create a reminder", just call the model once for structured output and run the action in normal code. That's cheaper, faster and easier to test. Tool use is worth the extra complexity when the model really has to decide which action to take or whether it needs more information first.
A quick test: if you can draw the flow as a straight line, write it as a straight line. Use tools when the flow branches based on what the user asked.
Wrapping up
Tool use is less magic than it looks. The model proposes a structured function call, and your code runs it. That split is useful because every safety decision stays in code you control. Keep tools narrow, scope data to the session user, validate arguments, confirm anything with consequences, cap the loop, and log the traces. Start with one or two read-only tools, watch how they get used, and add write actions only once you trust what you see.