How does function calling work in modern LLMs?

WHAT THIS TESTS: This question tests whether you understand the boundary between the LLM and the outside world. Interviewers want to know if you realize that function calling is a controlled form of structured output generation where the model only produces text shaped like a JSON payload, and the host application handles all side effects. It also checks if you can distinguish this pattern from retrieval augmented generation or autonomous agentic behavior.

A GOOD ANSWER COVERS: A strong answer walks through four steps in order. First, the developer declares available tools by sending schema objects to the API, typically including a function name, a natural language description, and a parameters object with types and required fields. Second, the model uses its instruction-following and pattern-matching capabilities to decide that a tool is needed based on the user prompt; it does not run code but instead generates a special content block or JSON object containing the function name and argument values. Third, the client application receives this payload, validates it, executes the actual function against a real database or API, and captures the return value. Fourth, the client sends the function result back to the model as a follow-up message, often with a specific role like function or tool, and the model then produces the final natural language response. Mentioning that modern APIs like Gemini assign a unique ID to each function call and expect that ID in the function response is a nice touch.

COMMON WRONG ANSWERS: The biggest red flag is describing the model as having agency, such as saying it opens a browser, queries a database, or posts to an API endpoint directly. Another mistake is conflating function calling with retrieval augmented generation; RAG injects documents into the context, whereas tool use triggers an external execution loop. Some candidates also forget that argument validation and error handling happen on the client side, not inside the model.

LIKELY FOLLOW-UPS: An interviewer might ask how the model chooses between multiple tools, how you handle parallel function calls, what happens if the model hallucinates a parameter, or how you secure the execution environment. They may also ask about the difference between forcing a tool choice versus letting the model decide, or how to handle long-running tools with the asynchronous live API.

ONE CONCRETE EXAMPLE: Imagine a user says schedule a meeting with Alice and Bob next Tuesday at 3pm. The developer has registered a schedule_meeting tool with parameters for attendees, date, time, and topic. The model recognizes the intent, emits a JSON payload with name set to schedule_meeting and arguments filled in, such as attendees Alice and Bob, date 2025-08-05, time 15:00, and topic TBD. The Python client receives this, calls the internal calendar API, gets back a success message with the meeting link, and passes that result to the model. The model then replies your meeting with Alice and Bob is scheduled for next Tuesday at 3pm, here is the link.

Source: ai.google.dev