> ## Documentation Index
> Fetch the complete documentation index at: https://docs.baseten.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Overview

> Inference on Baseten: Model APIs, self-deployed models, how responses are delivered, structured outputs, tool calling, and client configuration.

Inference on Baseten is the path from your application to a model running in Baseten's infrastructure, whether you use [Model APIs](/inference/model-apis/overview) for hosted models or deploy your own with [Truss](/development/model/overview). You don't provision GPUs or build your own routing layer: Baseten authenticates each request, matches it to a deployment environment, and runs it on a replica. This page assumes you already have a Baseten account and an API key.

<CardGroup cols={3}>
  <Card title="Model APIs" icon="messages" href="/inference/model-apis/overview">
    Call hosted models with the OpenAI or Anthropic SDK. No deployment required.
  </Card>

  <Card title="Your own deployment" icon="cube" href="/development/model/overview">
    Serve a model you deploy with Truss on dedicated GPUs.
  </Card>

  <Card title="Frontier Gateway" icon="route" href="/frontier-gateway/overview">
    Route one set of slugs across Baseten-hosted and external models.
  </Card>
</CardGroup>

To call popular open models without a Truss project first, use the public OpenAI-compatible endpoint at `https://inference.baseten.co/v1` with your [Baseten API key](/organization/api-keys) and the OpenAI SDK pointed at that base URL. For pricing, rate limits, and usage, see [Pricing and limits](/inference/model-apis/pricing-and-limits). For what happens after the gateway (routing, replicas, queuing, retries, cold starts), see [Request lifecycle](/deployment/autoscaling/request-lifecycle).

## Inference API

When you deploy your own model, pick an interface that matches your payloads. Engine-Builder-LLM, BIS-LLM, and BEI expose `/v1/chat/completions` (or `/v1/embeddings` for BEI) on your deployment's own endpoint, `https://model-<id>.api.baseten.co/environments/production/sync/v1`, with OpenAI-compatible parameters for structured outputs, tool calling, reasoning, and streaming. Custom Truss code can use `/predict` for arbitrary JSON when chat or embeddings are not a good fit. Use the [Inference API reference](/reference/inference-api/overview) for paths, methods, and errors.

## Synchronous inference

Synchronous calls return a full response in one round trip, which fits interactive use (chat, code completion, classification, embeddings) where the client can wait for the answer. See [Call your model](/inference/calling-your-model) for predict-style URLs across development, environment, and published deployments.

## Streaming

Streaming sends tokens as they are generated over server-sent events, which suits long generations and UIs where partial output beats a blank wait. See [Streaming](/inference/streaming) for client patterns and engine notes.

## Asynchronous inference

Async inference returns a request ID quickly and completes later through webhook or polling, which suits batch work, long documents, or any case where the caller should not hold a connection open for minutes. See [Async inference](/inference/async) for webhooks, status endpoints, and failures.

## Structured outputs and tool calling

Structured outputs constrain the model to a JSON schema you define; tool calling lets the model invoke your functions and continue the turn. Both align with OpenAI SDK parameters where supported on [Model APIs](/inference/model-apis/overview) and engine-backed deployments. Read [Structured outputs](/inference/structured-outputs) and [Function calling](/inference/function-calling) for implementation details. Model APIs can also execute web search and fetch tools server-side within a request. See [Web search](/inference/model-apis/web-search).

## Client configuration

For sustained load, tune connection reuse, timeouts, and parallelism. The Baseten Performance Client covers common cases; see [Performance client](/inference/performance-client) and [HTTP client configuration](/inference/http-client-configuration) for direct HTTP tuning.