Function calling

Function calling lets a model select a function and produce its arguments. Your application executes the function and returns its result to the model for a final answer. For tools that Baseten executes within a request, use server-side web search.

How tool calling works

Send a user message and tool definitions.
Read the model’s response, which can contain text or one or more tool calls.
Execute each returned call once in your application.
Send a result for every call, using its call ID.
Read the model’s next response. It can contain a final answer or more tool calls.

Check your model’s tool support before starting. For dedicated deployments, check the engine or server configuration.

Define tools

A tool definition names a function, explains when to use it, and describes its arguments with JSON Schema. The calculator below defines an add function with two numeric arguments.

Tool-writing tips

Describe units and allowed values in your schemas. Validate model-provided arguments and check permissions before running functions that access external systems.

Serialize functions

Chat Completions nests a function definition under function. Responses puts name, description, and parameters directly on the tool object. Each procedure below uses the format for its API.

Call the model

You need a Baseten API key in BASETEN_API_KEY and the OpenAI Python SDK, installed with pip install openai.

Chat Completions
Responses

To execute a tool and return its result:Run the following script. It uses Model APIs by default. For an OpenAI-compatible dedicated deployment, set BASETEN_BASE_URL to its production base URL and BASETEN_MODEL to the served model name. For more information, see Call your model.The first request requires a tool call. The follow-up returns every result and selects none so the model produces a final answer:

import json
import os
from openai import OpenAI


def add(a: float, b: float):
    return a + b


def checked_message(response):
    choice = response.choices[0]
    if choice.finish_reason not in {"stop", "tool_calls"}:
        raise RuntimeError(f"Generation stopped: {choice.finish_reason}")
    if choice.message.refusal:
        raise RuntimeError(f"Model refused: {choice.message.refusal}")
    return choice.message


parameters = {
    "type": "object",
    "properties": {"a": {"type": "number"}, "b": {"type": "number"}},
    "required": ["a", "b"],
    "additionalProperties": False,
}

client = OpenAI(
    api_key=os.environ["BASETEN_API_KEY"],
    base_url=os.environ.get("BASETEN_BASE_URL", "https://inference.baseten.co/v1"),
)
model = os.environ.get("BASETEN_MODEL", "deepseek-ai/DeepSeek-V4.1-Flash-1017")
tools = [{
    "type": "function",
    "function": {
        "name": "add",
        "description": "Add two numbers.",
        "parameters": parameters,
        "strict": True,
    },
}]
messages = [{"role": "user", "content": "Use add to calculate 3.14 + 3.14."}]
response = client.chat.completions.create(
    model=model, messages=messages, tools=tools, tool_choice="required",
)
message = checked_message(response)
messages.append(message)

for call in message.tool_calls or []:
    if call.function.name != "add":
        raise ValueError(f"Unexpected tool: {call.function.name}")
    args = json.loads(call.function.arguments)
    result = add(**args)
    print(f"Tool result: {result}")
    messages.append({
        "role": "tool",
        "tool_call_id": call.id,
        "content": json.dumps({"result": result}),
    })

if message.tool_calls:
    response = client.chat.completions.create(
        model=model, messages=messages, tools=tools, tool_choice="none",
    )
    message = checked_message(response)
if response.choices[0].finish_reason != "stop" or message.tool_calls:
    raise RuntimeError("Expected a complete final answer without tool calls.")
if not message.content:
    raise RuntimeError("No final answer returned.")
print(message.content)

Tool result: 6.28
3.14 + 3.14 = **6.28**

To execute a tool and continue a response:Run the following script against Model APIs. It keeps the response’s output items, including reasoning items, and appends one function_call_output for every call before requesting a final answer:

import json
import os
from openai import OpenAI


def add(a: float, b: float):
    return a + b


def check_response(response):
    if response.status != "completed":
        details = response.error or response.incomplete_details or response.status
        raise RuntimeError(f"Response did not complete: {details}")
    for item in response.output:
        if item.type == "message":
            for block in item.content:
                if block.type == "refusal":
                    raise RuntimeError(f"Model refused: {block.refusal}")


parameters = {
    "type": "object",
    "properties": {"a": {"type": "number"}, "b": {"type": "number"}},
    "required": ["a", "b"],
    "additionalProperties": False,
}

client = OpenAI(
    api_key=os.environ["BASETEN_API_KEY"],
    base_url="https://inference.baseten.co/v1",
)
model = "deepseek-ai/DeepSeek-V4.1-Flash-1017"
tools = [{
    "type": "function",
    "name": "add",
    "description": "Add two numbers.",
    "parameters": parameters,
    "strict": True,
}]
input_items = [{"role": "user", "content": "Use add to calculate 3.14 + 3.14."}]
response = client.responses.create(
    model=model, input=input_items, tools=tools, tool_choice="required",
)
check_response(response)
# Preserve all output items, including reasoning, for the next request.
input_items.extend(response.output)
calls = [item for item in response.output if item.type == "function_call"]

for call in calls:
    if call.name != "add":
        raise ValueError(f"Unexpected tool: {call.name}")
    result = add(**json.loads(call.arguments))
    print(f"Tool result: {result}")
    input_items.append({
        "type": "function_call_output",
        "call_id": call.call_id,
        "output": json.dumps({"result": result}),
    })

if calls:
    response = client.responses.create(
        model=model, input=input_items, tools=tools, tool_choice="none",
    )
    check_response(response)
if any(item.type == "function_call" for item in response.output):
    raise RuntimeError("Expected a final answer without open function calls.")
if not response.output_text:
    raise RuntimeError("No final answer returned.")
print(response.output_text)

Tool result: 6.28
3.14 + 3.14 = **6.28**

The numeric tool result is 6.28. The model’s final wording can vary. For more information, see the Chat Completions reference and Responses reference.

Control tool selection

Set tool_choice to choose whether the model can call tools:

auto: The model can return text or tool calls.
required: The model must return at least one tool call.
none: The model returns text without calling a tool.

To force a named function, use the shape for your API:

Chat Completions
Responses

To select a specific function:Configure the request’s tool_choice field:

Request fields

{"tool_choice": {"type": "function", "function": {"name": "add"}}}

To select a specific function:Configure the request’s tool_choice field:

Request fields

{"tool_choice": {"type": "function", "name": "add"}}

Parse and execute tool calls

Chat Completions returns calls in message.tool_calls; Responses returns function_call items in response.output. Arguments are JSON strings. Decode and validate them before executing the named function.

Full loop: send tool output back for a final answer

Return one result for every call ID. Keep the assistant message for Chat Completions, or all output items for Responses, together with those results. The complete procedures above execute each call once and reuse its result in the follow-up. For an agent that allows further tool calls, repeat this cycle with a bounded number of turns. Before executing calls or accepting final text, check for refusal and incomplete or failed generation. A response with no tool calls is a final answer only when generation completed successfully and answer text is present. For more information, see Inference errors.

Practical tips

Use JSON Schema constraints for required fields and allowed values. Check model support before requesting parallel calls. For functions with side effects, use application-level idempotency so a retried conversation doesn’t repeat an operation.

Chains: Orchestrate multi-step workflows.
Web search: Let Baseten execute supported tools server-side.

Get started

Model APIs

Baseten Grid

Inference

Frontier Gateway

Training

Sandboxes

Development

Deployment

Engines

Model Labs

Organization

Hosting options

Observability

Troubleshooting

Function calling

How tool calling works

Define tools

Tool-writing tips

Serialize functions

Call the model

Control tool selection

Parse and execute tool calls

Full loop: send tool output back for a final answer

Practical tips

​How tool calling works

​Define tools

​Tool-writing tips

​Serialize functions

​Call the model

​Control tool selection

​Parse and execute tool calls

​Full loop: send tool output back for a final answer

​Practical tips

​Related

How tool calling works

Define tools

Tool-writing tips

Serialize functions

Call the model

Control tool selection

Parse and execute tool calls

Full loop: send tool output back for a final answer

Practical tips

Related