LLMs as Tool Makers: Write Once, Solve Many

WHY IT EXISTS The most capable LLMs are also the most expensive and slowest to run. Using a top-tier model for every single request, especially for common or repetitive tasks, is highly inefficient. This creates a need for a system that can capture the reasoning power of the best models without paying the full inference cost for every use.

THE MENTAL MODEL Think of a senior engineer and a team of junior engineers. Instead of having the expensive senior engineer solve every simple, repetitive ticket, you have them write a robust library function for that common problem. The junior engineers can then just call that function. The senior's high cost is amortized across many uses, and the team gets the benefit of their expertise at scale. Here, a powerful LLM is the senior engineer, and a cheaper LLM is the junior engineer.

HOW IT WORKS The LLMs as Tool Makers (LATM) framework uses a two-phase, closed-loop process. First, in the 'tool making' phase, a powerful but expensive model (the 'tool maker') analyzes a problem and writes a reusable tool, like a Python function, to solve it. This tool is then cached. Second, in the 'tool using' phase, when subsequent similar problems arrive, a cheaper, lightweight model (the 'tool user') is invoked. It doesn't solve the problem from scratch; it simply calls the cached tool to get the answer efficiently.

WHEN TO USE IT This pattern is ideal for applications that handle complex but structured, repetitive reasoning tasks. It's a powerful cost-reduction strategy when you need the performance of a state-of-the-art model but the operational cost of a much cheaper one. It also enables 'functional caching'—storing reusable logic rather than static text responses, which is far more flexible than a traditional cache.

WHEN NOT TO USE IT Avoid this for highly unique, one-off tasks where the overhead of creating a tool provides no long-term value. If a problem is so novel that a tool for it will never be used again, the initial cost is wasted. The approach also depends on the 'tool maker' being capable of generating correct, robust code. If the task is too complex even for the best LLM, the generated tool will be unreliable.

ONE CANONICAL EXAMPLE A system receives a request to solve a specific type of logic puzzle. A powerful model like GPT-4 acts as the tool maker, writing a Python function that codifies the logic for solving that puzzle class. The function is saved. The next time a user submits a similar puzzle, a cheaper model like GPT-3.5 is used. It recognizes the puzzle type and calls the cached Python function with the new inputs, returning the correct answer at a fraction of the original cost.