Walk me through building a weather agent with get_weather

What's really being asked

The interviewer wants to see if you understand that modern LLMs do not directly call APIs. Instead they participate in a structured loop where they emit function-call requests, your code executes those requests, and you feed the results back. This tests your mental model of tool use versus monolithic generation, and whether you know which party owns execution.

The full answer

A strong answer walks through five steps in order. First, tool registration: you describe get_weather to the model with a JSON schema including the city parameter so the model knows the tool exists and what arguments it requires. Second, the initial inference call: you send the user message Whats the weather in Paris along with the tool definitions. The model does not answer immediately; instead it returns a structured function call object such as name get_weather and arguments city Paris. Third, execution in your code: you parse that structured output, validate the city parameter, and invoke get_weather Paris in your own application code outside the model. Fourth, the round trip: you append the function result to the conversation history as a function response message and send the updated history back to the model. Fifth, final synthesis: the model now generates a natural language answer like It is 22 degrees and sunny in Paris because it has both the original question and the real data. Mentioning that the model decides whether to call the tool rather than being forced is a bonus.

The mistakes people make

The biggest red flag is saying the LLM directly executes the API call or runs Python code inside its context window. Another mistake is describing a single prompt where you pre-fetch the weather and inject it into the context before calling the model; that skips the function-calling loop entirely and misses the point of the question. A third red flag is ignoring parameter validation or hallucinating that the model can handle authentication secrets and API keys on its own.

What usually comes next

The interviewer may ask how you handle a function call that fails or returns an error, such as when get_weather throws a 500 error. They might ask what happens if the model emits invalid JSON or hallucinates a parameter. They could also ask how you prevent infinite loops if the model keeps requesting the same tool, or how you parallelize multiple function calls when the model requests several tools at once.

A concrete example

Imagine using the Gemini API. You define a function declaration with name get_weather, description Fetches current weather for a city, and parameters type object with properties city type string. You pass this in the tools array via GenerateContentConfig. The user asks Whats the weather in Paris. The first response contains a function call with id abc123 and args city Paris. Your code extracts Paris, calls get_weather, and gets 22C sunny. You return a functionResponse with id abc123 and the result. On the second call the model outputs The current weather in Paris is 22 degrees and sunny.