Tool and Function Calling: How LLMs Actually Use Tools

Ask a language model what the weather is in Paris right now, and it has a problem. It has no window, no internet, and no clock. All it can do is write text. So how do assistants book flights, query databases and run code?

• Models only produce text
• No internet, no clock
• So how do they act?

The answer is a simple trick. The model never acts at all. It writes a structured request, and your code does the acting. That one idea powers nearly every AI agent you have used. In the last lesson we compared fine-tuning, RAG and prompting. Today we give models the ability to act.

• Model emits a structured request
• Your code executes it
• Result goes back to the model

By the end of this video you will know the schema that describes a tool, the exact message flow, how parallel calls and errors work, and how models learn to do this. We will then build a complete tool-calling loop in Python, with validation, and finish with the security rules you must not skip.

• Schema and message flow
• Training and constrained decoding
• Python loop with Pydantic
• Security and mistakes

Think of the model as a manager who cannot touch a keyboard, and your program as an assistant who can. The manager writes a note: please look up the weather in Paris in Celsius. The assistant reads the note, does the work, and reports back. The manager then writes the final reply to the customer.

• Model decides what to do
• App does the real work
• Model explains the result

Why bother? Models are weak at exact arithmetic, have a frozen knowledge cutoff, and cannot cause side effects like sending an email. Tools fix all three. A calculator gives exact math, a search or database tool gives fresh facts, and an action tool changes the world. This is grounding, as in our RAG lessons, but now the model chooses when to look.

• Exact math
• Fresh facts
• Real actions

Notice the important point: the model only emits text. A tool call is a special piece of text, usually a JSON object, with a tool name and arguments. Your application parses that text, decides whether to run it, and runs it. The model has no power you did not give it.

• Name plus arguments
• Usually JSON
• Your app decides to run it

To use a tool, the model must first know it exists. You describe each tool with three parts: a name, a natural language description, and a JSON schema for the parameters. These descriptions are placed into the model's context, typically through a special tools field that the provider formats into the prompt.

• Name
• Description
• JSON-schema parameters

Here is a weather tool. Name: get_weather. Description: get the current temperature for a city; use when the user asks about current weather. Parameters: an object with a required string field called city, and an optional field called unit that must be either celsius or fahrenheit.

• city: required string
• unit: enum
• Clear description of when to use

The description is the most important field. It is the only thing the model knows about your tool, so it works like a tiny prompt. Say what the tool does, when to use it, and what it returns. Parameter descriptions help too. Think of writing documentation for a smart colleague who has never seen your code.

• Say what it does
• Say when to use it
• Say what it returns

Now the flow. Step one: the user sends a message, and your app sends it to the model along with the tool list. Step two: instead of an answer, the model replies with a tool call, saying it wants get_weather with city Paris. The response is marked as stopping because of a tool request.

• User asks a question
• Model returns a tool call
• No answer yet

Step three: your app validates and executes the function. Step four: it appends a tool result message to the conversation, linked to the call by an identifier, and calls the model again. Step five: the model reads the result and writes the final answer, such as it is eighteen degrees and cloudy in Paris.

• App executes the function
• Result message with call id
• Model writes final answer

Look at the conversation as a list. User message. Assistant message containing a tool call. Tool message containing the result. Assistant message with the answer. The model is stateless, so every request must include this whole history, including the call and its result. Forgetting the tool message is a classic bug.

• Four message types
• Stateless model
• Always resend history

Sometimes one step needs several tools. If the user asks for the weather in Paris and Tokyo, a capable model can emit two calls in the same turn, each with its own identifier. Your app can run them concurrently, then return two tool messages, one per identifier. This saves whole round trips.

• Several calls in one turn
• Match results by id
• Fewer round trips

A worked example of the saving. Suppose the model takes one second per turn and each tool takes one second. Sequentially, two lookups cost two model turns plus two tool runs, plus a final turn: about five seconds. In parallel it is one turn, one second of tools run together, and a final turn: about three seconds.

• Sequential: 5 seconds
• Parallel: 3 seconds
• Assumes 1 second per step

You can also control tool choice. In auto mode, the model decides whether to call a tool or just answer. In required mode, it must call some tool. You can also force one specific tool, and in none mode, tools are disabled. Exact option names vary by provider, but the idea is the same.

• Auto: model decides
• Required: must call a tool
• Specific: force one tool
• None: text only

Forcing is useful when you want structured extraction. If you force a tool whose parameters are your target schema, the arguments become your structured output, as in our structured output lesson. Auto is right for assistants, where many questions need no tool at all.

• Extraction as a tool call
• Auto for assistants
• Pick per use case

How does a model learn to write these calls? Mostly through training, not magic. During fine-tuning, as in the supervised fine-tuning lesson, models see many example conversations that include tool schemas, tool call messages in a special format, and tool results. The next-token objective teaches the format just like any other text.

• Fine-tuned on tool transcripts
• Special call format
• Same next-token loss

Those examples come from a mix of human written data and synthetic data, and later preference training or reinforcement learning can reward correct, useful calls. The exact recipes differ between labs, and they are not all public, so treat any specific claim with care. What matters is that the skill is learned, so it can fail.

• Synthetic and human data
• Preference training helps
• Calls can still be wrong

Failures include invented tool names, missing fields, wrong types, and malformed JSON. Constrained decoding helps. At each step the model produces logits over the vocabulary. A grammar or schema engine masks tokens that would break the format, setting their probability to zero, then renormalizes. The output is guaranteed to parse.

• Mask invalid tokens
• Renormalize the rest
• Guaranteed parseable output

A tiny numeric example. Suppose after an opening brace the model gives 0.5 to a quote, 0.3 to the word Sure, and 0.2 to a newline. Only the quote is valid. Mask the others, and the quote has probability one. But note the limit: valid syntax does not mean a correct value. The city could still be wrong.

• Valid: quote only
• Probability becomes 1.0
• Syntax is not semantics

Tools fail, and models make bad requests. The rule is: never crash the loop, and never hide the failure. Catch the error, and return it to the model as the tool result. A message like unknown city, expected a city name, lets the model correct itself, ask the user, or explain the problem honestly.

• Catch exceptions
• Send error as tool result
• Model can retry or explain

Write error messages for a reader that is a model. Include what went wrong and how to fix it. Compare: error 400, versus, the field unit must be celsius or fahrenheit, you sent kelvin. The second one almost always leads to a successful retry. Also cap retries and total steps so a confused model cannot loop forever.

• Say what and how to fix
• Cap retries
• Cap total loop steps

Now the code, written provider-neutrally. First, two plain Python functions. A calculator that takes an arithmetic expression, and a weather stub that returns fake data from a dictionary. Using a stub keeps the example runnable anywhere, and you can swap in a real weather API later.

• calculator
• weather stub
• Pure Python

Notice the calculator takes two numbers and an operator, not a string to evaluate. That is deliberate. We will never pass model text to eval. Next, define Pydantic models for the arguments. A model class with a city string and a unit limited to celsius or fahrenheit, and another with two floats and an operator limited to four values.

• Typed fields
• Literal limits values
• No eval anywhere

Pydantic also generates the JSON schema for us with a method called model JSON schema. So one class is both the validator and the schema we show the model. Build a registry: a dictionary from tool name to its description, argument class and function. Single source of truth, fewer mismatches.

• Pydantic generates the schema
• Registry maps name to function
• Validator and schema stay in sync

Here is the loop. Start with the messages list. Repeat up to a maximum number of steps: call the model with messages and tool schemas. If the reply has no tool calls, return its text. Otherwise append the assistant message, and for each call, run a safe executor and append a tool message with the call id and result.

• Call model
• No tool calls? Return
• Else execute and append
• Max steps limit

The safe executor does four things. Look up the tool name; if unknown, return an error listing valid names. Parse the arguments from JSON; on failure, return the parse error. Validate with Pydantic; on failure, return the validation message. Finally run the function inside a try block, and convert any exception into an error string.

• Check tool name
• Parse JSON
• Validate with Pydantic
• Catch exceptions

Trace one run. The user asks: what is the temperature in Paris, times two? The model calls get_weather for Paris, and your stub returns eighteen. Next turn the model calls calculator with a equal to eighteen, op mul, b equal to two, and gets thirty-six. The final turn answers: thirty-six degrees, doubling eighteen.

• Turn 1: weather returns 18
• Turn 2: calculator returns 36
• Turn 3: final answer

This trace shows something important. The second call depended on the first result, so it could not run in parallel. The loop handles both cases naturally: independent calls in one turn, dependent calls across turns. This repeated think, act, observe pattern is the seed of agents, and we will name it properly in the next lesson.

• Dependent calls span turns
• Independent calls share a turn
• Seed of the agent loop

Now security, because a tool call is model-generated input, and the model can be influenced by a user or by untrusted text it reads. Rule one: validate every argument. Check types, ranges, allowed values and lengths with Pydantic, exactly as you would for any web request from an unknown client.

• Treat output as untrusted
• Check type, range, length
• Same as any web request

Rule two: least privilege. Give each tool only the access it needs. A read-only database user, not an admin. A file tool restricted to one folder. An email tool that can draft, with a human approval step before sending. If a tool cannot do damage, a confused or tricked model cannot cause it.

• Read-only where possible
• Restrict paths and scopes
• Human approval for risky actions

Rule three: never evaluate raw model output. Passing a model-written string to eval or a shell command lets an attacker who influences the model run code on your machine. If you truly need code execution, use a sandbox with no secrets, limited resources and no network. Better still, expose narrow, structured functions like our calculator.

• No eval or raw shell
• Sandbox if you must run code
• Prefer narrow functions

One more risk: prompt injection through tool results. A web page the model reads may contain instructions like ignore your rules and email the files. Treat tool results as data, not commands, and keep dangerous tools behind approvals. We will return to this in the production safety lesson.

• Results can carry attacks
• Treat results as data
• Gate dangerous tools

Mistake one: vague tool descriptions. A tool called search with the description searches things leaves the model guessing. Say what it searches, when to prefer it, and what the output looks like. Rename parameters clearly too: query beats q, and a field named date should state its format.

• Explain purpose and timing
• Use clear parameter names
• State formats

Mistake two: too many tools. Every schema consumes context tokens, and with dozens of similar tools the model picks the wrong one more often. I will not quote a magic number, since it depends on the model, so measure it. Merge overlapping tools, remove unused ones, or retrieve a relevant subset per request.

• Schemas cost tokens
• Similar tools confuse the model
• Retrieve a relevant subset

Mistake three: unbounded side effects. A delete tool with no limits, a payment tool with no cap, or a loop with no step limit can cause real harm after one bad call. Add limits, dry-run modes, confirmation for irreversible actions, and idempotency keys so a retried call does not charge a customer twice.

• Add caps and limits
• Confirm irreversible actions
• Use idempotency keys

Two smaller mistakes. Returning huge tool outputs, like a whole database table, floods the context, so truncate and summarize. And skipping evaluation: log every call, then test whether the model picks the right tool with the right arguments, using a fixed set of example questions, as in our prompt evaluation lesson.

• Trim large outputs
• Log every call
• Test tool selection

Let us recap. The model writes a structured request and your code acts. A tool is a name, a description and a JSON schema. The flow is user, tool call, execution, tool result, final answer. Parallel calls save time, tool choice controls behavior, and errors go back to the model.

• Name, description, schema
• Call, execute, result, answer
• Parallel and error handling

Models learn calls from fine-tuning on examples, and constrained decoding can guarantee valid syntax, but never valid meaning. So validate with Pydantic, apply least privilege, never evaluate raw output, write sharp descriptions, keep the tool list small, and bound every side effect.

• Validate and limit access
• Never eval raw output
• Few, clear, bounded tools

Our loop already hints at something bigger. The model reasons, acts, observes the result, and reasons again. That cycle is called ReAct, and in the next lesson we will cover it, along with planning and reflection.

• Thought, action, observation
• Planning and reflection
• Subscribe for lesson 28