Function Calling: LLMs Using Tools

Why it exists

Large language models are frozen neural networks with a fixed knowledge cutoff. They cannot access real time stock prices, send emails, or run Python code because they are text in, text out systems with no native connection to external tools. Function calling was invented to bridge this gap by teaching the model to emit structured machine readable intent rather than unstructured text, allowing developer code to act on the model's behalf.

The mental model

Think of function calling as a universal remote control that does not actually change the channel. The LLM reads your natural language request, decides which button to press, and hands you a labeled sticky note that says press power on device X. Your application is the person who receives the note, verifies it is sane, presses the actual button, and reports back what happened. The model proposes, the code disposes.

How it works

You register one or more available functions with the model by providing a JSON schema that defines each function name, description, and parameter types. When a user asks a question that requires external data, the model outputs a special JSON object instead of normal text. This object contains the target function name and a dictionary of arguments. Your application parses this payload, validates the arguments against your own constraints, invokes the real function, and then optionally returns the function result to the model in a follow up message so the model can synthesize a natural language answer for the user.

When to use it

Use function calling whenever the LLM needs information or actions beyond its parametric memory. Common scenarios include querying live databases, retrieving weather or stock data, performing deterministic calculations with a calculator or code interpreter, booking appointments, or controlling hardware and APIs. It is also valuable when you need structured extraction from user input, such as parsing a shipping address into labeled fields that map directly to an API.

When not to use it

Avoid function calling for tasks the model can handle entirely from its internal knowledge, like summarization, translation, or creative writing, because the extra latency and complexity add no value. Do not use it when strict deterministic guarantees are required for the argument values themselves, since the model can still hallucinate parameters or choose the wrong function. If your function schema is extremely large or deeply nested, the model may struggle to generate valid JSON and you may be better off with traditional parsing or fine tuning.

One canonical example

A user asks a travel assistant, what is the cheapest flight from New York to London tomorrow. The model does not know live airfare, so it emits a JSON payload calling search_flights with arguments origin set to JFK, destination set to LHR, and date set to tomorrow's date. Your backend receives this, calls the airline API, finds a four hundred dollar ticket, and returns that result to the model. The model then replies, the cheapest flight is four hundred dollars on Delta departing at 8 PM. The user sees a natural answer, but every real action happened in your code.