> ## Documentation Index
> Fetch the complete documentation index at: https://docs.baseten.co/llms.txt
> Use this file to discover all available pages before exploring further.
# Call your model
> Run inference on deployed models
Use this guide to call a model that you deployed on Baseten. To call a hosted model without creating a deployment, see [Model APIs](/inference/model-apis/overview).
Each deployed model has an [HTTPS endpoint](/reference/inference-api/overview). To send a request, you need:
* **Model ID**: Found in the Baseten dashboard or returned when you deploy.
* **[API key](/organization/api-keys)**: Authenticates your requests.
* **Model input:** A JSON body that matches the input expected by your model.
The model ID (and the deployment ID, when you need to target a specific
deployment) comes from the model's page URL in your workspace:
Call your model from server-side code to avoid exposing your Baseten API key.
Dedicated deployment endpoints do not currently include CORS response headers, so browsers may block direct calls.
## Authentication
The predict endpoint lives on your model's own subdomain:
Include your API key in the `Authorization` header:
```bash Request theme={"system"}
curl -X POST https://model-YOUR_MODEL_ID.api.baseten.co/environments/production/predict \
-H "Authorization: Bearer $BASETEN_API_KEY" \
-H "Content-Type: application/json" \
-d '{"prompt": "Hello, world!"}'
```
In Python with requests:
```python predict.py theme={"system"}
import requests
import os
api_key = os.environ["BASETEN_API_KEY"]
model_id = "YOUR_MODEL_ID"
response = requests.post(
f"https://model-{model_id}.api.baseten.co/environments/production/predict",
headers={"Authorization": f"Bearer {api_key}"},
json={"prompt": "Hello, world!"},
)
print(response.json())
```
Baseten also accepts the legacy `Authorization: Api-Key ` scheme on every endpoint, so existing scripts continue to work:
```bash Request theme={"system"}
curl -X POST https://model-YOUR_MODEL_ID.api.baseten.co/environments/production/predict \
-H "Authorization: Api-Key $BASETEN_API_KEY" \
-H "Content-Type: application/json" \
-d '{"prompt": "Hello, world!"}'
```
## Predict API endpoints
Baseten provides two predict endpoints:
* [`/predict`](/reference/inference-api/overview#predict-endpoints): Standard synchronous inference.
* [`/async_predict`](/reference/inference-api/overview#predict-endpoints): Asynchronous inference for long-running tasks.
Both endpoints can target an environment or a specific deployment. See the [inference API overview](/reference/inference-api/overview) for URL formats and behavior.
## Sync API endpoints
Custom servers also provide a `sync` endpoint that forwards requests to routes exposed by your server:
```text URL theme={"system"}
https://model-{model-id}.api.baseten.co/environments/{production}/sync/{route}
```
These examples show how the sync endpoint maps to the custom server's routes:
* `https://model-{model_id}.../sync/health` -> `/health`
* `https://model-{model_id}.../sync/items` -> `/items`
* `https://model-{model_id}.../sync/items/123` -> `/items/123`
## OpenAI SDK
Engine-Builder deployments expose an OpenAI-compatible server. To use an OpenAI SDK, set its base URL to your Baseten deployment and authenticate with your Baseten API key:
```python openai_client.py theme={"system"}
import os
from openai import OpenAI
model_id = "abcdef" # TODO: replace with your model id
api_key = os.environ.get("BASETEN_API_KEY")
model_url = f"https://model-{model_id}.api.baseten.co/environments/production/sync/v1"
client = OpenAI(
base_url=model_url,
api_key=api_key,
)
stream = client.chat.completions.create(
model="Qwen/Qwen2.5-3B-Instruct", # must match --served-model-name in the deployment
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is the capital of France?"}
],
stream=True,
)
for chunk in stream:
if chunk.choices[0].delta.content is not None:
print(chunk.choices[0].delta.content, end="")
```
## External LLM gateways
OpenAI-compatible LLM gateways, such as LiteLLM or OpenRouter, can route traffic to a Baseten deployment. Configure the gateway with three values:
* **Base URL**: `https://model-{model_id}.api.baseten.co/environments/production/sync/v1`, using the model ID for your deployment. Choose **API endpoint** on the model page in the Baseten dashboard to copy the full URL.
* **Model name**: The value of `--served-model-name` from your deployment's `start_command`. See the [vLLM example](/examples/vllm) for where this is set. When a single gateway routes to several deployments, use an `org/model` naming convention (for example, `acme/llama-3-70b`) to keep routing unambiguous.
* **API key**: A [Baseten API key](/organization/api-keys) with access to the deployment.
The gateway sends requests to `{base_url}/chat/completions` with `model` set to the served model name and an `Authorization: Bearer ` header.
## Alternative invocation methods
* **Baseten CLI**: [`baseten model predict`](/reference/cli/baseten/model)
* **Model Dashboard**: "Playground" button in the Baseten dashboard