When should you use Function Calling / Tool Use?

When the model needs real data (current weather, DB lookups, search). Agent workflows that act on the world (send emails, write files, query APIs). Anywhere the model would otherwise hallucinate facts. When you need structured machine-parseable output from a chat interface.

When NOT to use Function Calling / Tool Use?

Pure-creative tasks (writing, brainstorming) where there's nothing to retrieve. Trivial tasks where direct generation is faster and cheaper than the round-trip. When you can't trust the model with the tool's permissions (always sandbox).

How does Function Calling / Tool Use work?

Declare available functions as JSON-schema with name, description, and parameter shape. The model returns either a normal text response, or a structured call: { name: 'get_weather', arguments: { city: 'Tokyo' } }. Your code runs the real function with those arguments and returns the result to the model. Model uses the result to produce the final user-facing answer. Modern: parallel tool calls, multi-turn tool use, automatic schema generation.

All techniques

Glossary · Technique

Function Calling / Tool Use

Also known as: Tool use, Function tools, Structured invocation

Let the model decide when to invoke a real function or API instead of free-text answering. The foundation of every modern agent.

When to use it

When the model needs real data (current weather, DB lookups, search).
Agent workflows that act on the world (send emails, write files, query APIs).
Anywhere the model would otherwise hallucinate facts.
When you need structured machine-parseable output from a chat interface.

When not to use it

Pure-creative tasks (writing, brainstorming) where there's nothing to retrieve.
Trivial tasks where direct generation is faster and cheaper than the round-trip.
When you can't trust the model with the tool's permissions (always sandbox).

How it works

1Declare available functions as JSON-schema with name, description, and parameter shape.
2The model returns either a normal text response, or a structured call: { name: 'get_weather', arguments: { city: 'Tokyo' } }.
3Your code runs the real function with those arguments and returns the result to the model.
4Model uses the result to produce the final user-facing answer.
5Modern: parallel tool calls, multi-turn tool use, automatic schema generation.

Example

Lazy prompt

What's the weather in Tokyo? (The model will fabricate temperature data.)

Using the technique

(Define get_weather tool with city parameter, call it with the model's structured request, return the real data, model uses it in the final response.)

Common pitfalls

Bad function descriptions = wrong tool selection. The description is the prompt.
Models will sometimes invoke a tool when free-text would have worked, and vice-versa.
Returning huge tool outputs blows the context window — paginate or summarize first.
Security: never give tools un-audited access to dangerous side effects.

Where this came from

OpenAI introduced function calling June 2023. Anthropic followed with tool use. Now standard across all major LLM APIs.

Related techniques

ReAct (Reason + Act)

Alternate reasoning and acting in a tight loop. The dominant pattern for tool-using agents — think, act, observe, repeat.

Retrieval-Augmented Generation (RAG)

Don't train on it — retrieve it. Inject relevant documents into the prompt at runtime so the model answers from real source material.

System Prompt Design

The hidden instructions that set the model's role, constraints, and ground rules for the entire conversation. Where 80% of product behavior actually lives.