> ## Documentation Index
> Fetch the complete documentation index at: https://docs.baseten.co/llms.txt
> Use this file to discover all available pages before exploring further.
# Build your model
> Deploy a Hugging Face model to Baseten with vLLM and a config.yaml file.
Deploy [Qwen 2.5 3B Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct) on an L4 with [vLLM](https://docs.vllm.ai/)'s OpenAI-compatible API. You'll define the model in `config.yaml`, then use the Baseten CLI to deploy and manage it.
You need a [Baseten account](https://app.baseten.co/signup). To call the model, use cURL or run the Python example with [uv](https://docs.astral.sh/uv/).
## Set up your environment
**To install the Baseten CLI and sign in**:
```bash Terminal theme={"system"}
brew tap basetenlabs/baseten
brew trust --formula basetenlabs/baseten/baseten
brew install baseten
```
Download and extract the binary, then move `baseten.exe` to a directory on your `PATH`:
```powershell Terminal theme={"system"}
Invoke-WebRequest `
https://github.com/basetenlabs/baseten-cli/releases/download/v1.0.0/baseten_1.0.0_windows_amd64.zip `
-OutFile baseten.zip; Expand-Archive -Force baseten.zip .
```
For other platforms or a specific version, see the [Baseten CLI install reference](/reference/cli/baseten/overview#install).
Browser login opens a tab where you approve the device. You do not need an API key until you call the deployed model:
```bash Terminal theme={"system"}
baseten auth login --web
```
If the browser doesn't open, copy the URL from the terminal and approve the device there.
## Create the config
**To create the project**:
1. Create a directory:
```bash Terminal theme={"system"}
mkdir qwen-2.5-3b && cd qwen-2.5-3b
```
2. Add a `config.yaml`:
```yaml config.yaml theme={"system"}
model_name: Qwen-2.5-3B
model_metadata:
tags:
- openai-compatible
base_image:
image: vllm/vllm-openai:v0.27.1
docker_server:
start_command: vllm serve /models/qwen --served-model-name Qwen/Qwen2.5-3B-Instruct --host 0.0.0.0 --port 8000
readiness_endpoint: /health
liveness_endpoint: /health
predict_endpoint: /v1/chat/completions
server_port: 8000
weights:
- source: "hf://Qwen/Qwen2.5-3B-Instruct@aa8e72537993ba99e69dfaafa59ed015b17504d1"
mount_location: /models/qwen
resources:
accelerator: L4
use_gpu: true
runtime:
predict_concurrency: 256
```
The key fields are:
* `model_metadata.tags`: Marks the server as OpenAI-compatible, which enables the chat playground for this model in the dashboard.
* `base_image.image`: Uses a pinned [vLLM Docker image](https://hub.docker.com/r/vllm/vllm-openai/tags).
* `docker_server.start_command`: Starts vLLM with the mounted weights. `--served-model-name` sets the model identifier in requests, and `--port 8000` must match `server_port`.
* `docker_server.predict_endpoint`: Sets vLLM's `/v1/chat/completions` route as the deployment's default request endpoint.
* `weights`: Mounts the pinned Hugging Face weights at `/models/qwen` through the [Baseten Delivery Network](/development/model/bdn). This model is ungated, so it doesn't need a token.
* `resources.accelerator`: Runs inference on a single L4.
* `runtime.predict_concurrency`: Allows up to 256 concurrent requests so vLLM can batch them.
For all available fields, see the [Truss configuration reference](/reference/truss-configuration).
## Deploy the model
From the project directory, run:
```bash Terminal theme={"system"}
baseten model push --tail --wait
```
You should see:
```output theme={"system"}
✨ Model Qwen-2.5-3B was successfully pushed ✨
Model: Qwen-2.5-3B (abc123)
Deployment: xyz123
Environment: production
🪵 View logs:
deployment: baseten model deployment logs --model-id abc123 --deployment-id xyz123
environment: baseten model environment logs --model-id abc123 --environment production (once deployed)
app: https://app.baseten.co/models/abc123/logs/xyz123
🚀 Invoke your model:
URL: https://model-abc123.api.baseten.co/deployment/xyz123/predict
CLI: baseten model predict --model-id abc123
```
The command streams deployment logs and exits when the model is **Active**.
## Call the model
vLLM serves an OpenAI-compatible API. Replace `{model_id}` with the model ID from the push output. The endpoint follows this shape:
**To send a request**:
1. Create a [personal API key](/organization/api-keys#create-an-api-key) and export it. Browser login does not set this variable:
```bash macOS/Linux theme={"system"}
export BASETEN_API_KEY="paste-your-api-key-here"
```
```powershell Windows theme={"system"}
setx BASETEN_API_KEY "paste-your-api-key-here"
```
2. Call the endpoint. The `model` value must match `--served-model-name` in the `start_command`. For the Python example, `uv run --with openai` installs the OpenAI SDK for that run:
```python call_model.py theme={"system"}
import os
from openai import OpenAI
client = OpenAI(
api_key=os.environ["BASETEN_API_KEY"],
base_url="https://model-{model_id}.api.baseten.co/environments/production/sync/v1",
)
response = client.chat.completions.create(
model="Qwen/Qwen2.5-3B-Instruct",
messages=[
{"role": "user", "content": "What is machine learning?"}
],
)
print(response.choices[0].message.content)
```
```bash Terminal theme={"system"}
uv run --with openai python call_model.py
```
```bash Request theme={"system"}
curl -s https://model-{model_id}.api.baseten.co/environments/production/sync/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $BASETEN_API_KEY" \
-d '{
"model": "Qwen/Qwen2.5-3B-Instruct",
"messages": [
{"role": "user", "content": "What is machine learning?"}
]
}'
```
You should see a response like:
```output theme={"system"}
Machine learning uses algorithms to learn patterns from data and make
predictions or decisions without being explicitly programmed for each task...
```
If the request fails, see [Inference errors](/inference/errors).
You configured vLLM to serve a pinned Hugging Face model, deployed it to an L4 with the Baseten CLI, and called its OpenAI-compatible endpoint.
## Choose your next learning path
### Operate your model in production
1. Use [deployment environments](/deployment/environments) to control which deployment serves production traffic.
2. Configure [autoscaling](/deployment/autoscaling/overview) for your traffic patterns.
3. Monitor the deployment with [logs](/observability/logs) and [metrics](/observability/metrics).
### Customize the serving stack
1. Explore [model deployment examples](/examples/models/overview) to deploy other models and serving engines.
2. Follow the [vLLM recipe](/examples/vllm) to tune the server and add streaming.
3. Learn how to deploy [custom Docker servers](/development/model/custom-server).
4. Configure [weight mounting](/development/model/bdn) for private repositories or cloud storage.
### Add custom model logic
1. Implement the [Model class](/development/model/model-class) for preprocessing, postprocessing, or other architectures.
2. Add [Python dependencies](/development/model/dependencies) required by your model code.
3. [Deploy and iterate](/development/model/deploy-and-iterate) as you develop locally.