How does function calling work in modern LLMs?
whether you see function calling as client-side structured generation, not model execution.
schemas in the prompt; model emits JSON name and arguments; client executes and returns results.
What's really being asked
This question tests whether you understand the boundary between the LLM and the outside world. Interviewers want to know if you realize that function calling is a controlled form of structured output generation where the model only produces text shaped like a JSON payload, and the host application handles all side effects. It also checks if you can distinguish this pattern from retrieval augmented generation or autonomous agentic behavior.
The full answer
A strong answer walks through four steps in order. First, the developer declares available tools by sending schema objects to the API, typically including a function name, a natural language description, and a parameters object with types and required fields. Second, the model uses its instruction-following and pattern-matching capabilities to decide that a tool is needed based on the user prompt; it does not run code but instead generates a special content block or JSON object containing the function name and argument values. Third, the client application receives this payload, validates it, executes the actual function against a real database or API, and captures the return value. Fourth, the client sends the function result back to the model as a follow-up message, often with a specific role like function or tool, and the model then produces the final natural language response. Mentioning that modern APIs like Gemini assign a unique ID to each function call and expect that ID in the function response is a nice touch.
The mistakes people make
The biggest red flag is describing the model as having agency, such as saying it opens a browser, queries a database, or posts to an API endpoint directly. Another mistake is conflating function calling with retrieval augmented generation; RAG injects documents into the context, whereas tool use triggers an external execution loop. Some candidates also forget that argument validation and error handling happen on the client side, not inside the model.
What usually comes next
An interviewer might ask how the model chooses between multiple tools, how you handle parallel function calls, what happens if the model hallucinates a parameter, or how you secure the execution environment. They may also ask about the difference between forcing a tool choice versus letting the model decide, or how to handle long-running tools with the asynchronous live API.
A concrete example
Imagine a user says schedule a meeting with Alice and Bob next Tuesday at 3pm. The developer has registered a schedule_meeting tool with parameters for attendees, date, time, and topic. The model recognizes the intent, emits a JSON payload with name set to schedule_meeting and arguments filled in, such as attendees Alice and Bob, date 2025-08-05, time 15:00, and topic TBD. The Python client receives this, calls the internal calendar API, gets back a success message with the meeting link, and passes that result to the model. The model then replies your meeting with Alice and Bob is scheduled for next Tuesday at 3pm, here is the link.
Interview question
During LLM function calling, after the model decides a tool is needed, what does it actually do before the client takes over?
- a.It emits a JSON object with the function name and arguments for the client to executeCorrect
- b.It queries a vector database to fetch context relevant to the tool choice
- c.It executes the function via an internal sandbox and prepares a natural language summary
- d.It validates parameter types and makes the HTTP request to the external service directly
Why? this is the answer
The model only generates structured text shaped like a JSON payload containing the function name and arguments; the host application must execute the actual function and return the result. The distractor suggesting the model runs code in a sandbox reflects the common misconception that the LLM has agency, whereas modern APIs keep all side effects strictly on the client side.
Just read this? Test yourself on what you have been reading.
Read the original → ai.google.dev
- #llms
- #function-calling
- #tool-use
- #structured-generation
- #generative-ai
You just looked this up. Could you explain it out loud?
That is the part interviews actually test. Tezvyn takes questions like this one and gives you what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on llms — each one lists the topics its interview covers.
See open roles