Skip to main content

Tool Calling

Let models call functions you define. The model decides when to call a tool, generates the arguments, and you execute the function and return the result. Same format as OpenAI function calling.

Define tools​

Tools are JSON Schema descriptions of functions the model can call:

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a city",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {"type": "string", "description": "City name"},
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["city"],
            },
        },
    }
]

response = client.chat.completions.create(
    model="phi-4-mini",
    messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
    tools=tools,
)

Handle tool calls​

When the model decides to call a tool, finish_reason is "tool_calls" and the response contains the function name and arguments:

choice = response.choices[0]

if choice.finish_reason == "tool_calls":
    for tool_call in choice.message.tool_calls:
        name = tool_call.function.name
        args = json.loads(tool_call.function.arguments)

        # Execute your function
        if name == "get_weather":
            result = get_weather(args["city"], args.get("unit", "celsius"))

        # Return the result to the model
        messages.append(choice.message)
        messages.append({
            "role": "tool",
            "tool_call_id": tool_call.id,
            "content": json.dumps(result),
        })

    # Continue the conversation with tool results
    follow_up = client.chat.completions.create(
        model="phi-4-mini",
        messages=messages,
        tools=tools,
    )

Parallel tool calls​

Models can request multiple tool calls in a single response. Process all of them before continuing:

if choice.finish_reason == "tool_calls":
    messages.append(choice.message)
    for tool_call in choice.message.tool_calls:
        result = dispatch_tool(tool_call.function.name, tool_call.function.arguments)
        messages.append({
            "role": "tool",
            "tool_call_id": tool_call.id,
            "content": json.dumps(result),
        })

Force or prevent tool use​

Control when the model uses tools:

# Force a specific tool
response = client.chat.completions.create(
    model="phi-4-mini",
    messages=messages,
    tools=tools,
    tool_choice={"type": "function", "function": {"name": "get_weather"}},
)

# Let the model decide (default)
tool_choice="auto"

# Prevent tool use
tool_choice="none"

On-device considerations​

Tool calling works on-device with models that support it. Keep in mind:

  • Smaller models are less reliable at generating valid tool call arguments. Use structured decoding to enforce argument schemas.
  • Latency -- tool calls add a round-trip. The model generates the call, you execute it, then the model processes the result.
  • Model support -- not all models support tool calling. Check the model catalog for the tools capability flag.