Create a model with the REST API

Use POST /v1/models to deploy a Truss archive. To deploy a model published through Baseten for Model Labs, see Deploy a Model Labs listing. To create the model in a specific team, use POST /v1/teams/{team_id}/models with the same request body. Use GET /v1/teams to find the team ID. The organization-scoped endpoint uses the organization’s default team. Deploying an archive follows these steps:

Prepare: POST /v1/prepare_model_upload validates the payload and returns temporary credentials scoped to an S3 location.
Upload: push your Truss archive to that location.
Create: POST /v1/models commits the upload as a new model.

For a team-scoped archive deployment, also pass team_id when preparing the upload.

Prepare the upload

Send a Truss config as a JSON object with a model name. Add a weights block to load weights through the Baseten Delivery Network. Set dry_run to true to validate without issuing credentials. The response carries the upload credentials and the S3 location to upload to:

curl https://api.baseten.co/v1/prepare_model_upload \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "my-model",
    "deployment": {
      "config": { "model_name": "my-model", "resources": { "accelerator": "A10G" }, "weights": [{ "source": "hf://meta-llama/Llama-3.1-8B@main", "mount_location": "/models/llama" }] }
    }
  }'

{
  "creds": {
    "aws_access_key_id": "ASIA...",
    "aws_secret_access_key": "...",
    "aws_session_token": "..."
  },
  "s3_bucket": "baseten-user-models-xxxx",
  "s3_key": "organizations/.../models/.../model.tgz",
  "s3_region": "us-west-2"
}

Upload the archive

Package your Truss as a gzipped tar archive, then upload it to the returned s3_bucket and s3_key using the temporary credentials:

upload.py

import boto3

# resp is the JSON returned by the prepare step
creds = resp["creds"]
session = boto3.Session(
    aws_access_key_id=creds["aws_access_key_id"],
    aws_secret_access_key=creds["aws_secret_access_key"],
    aws_session_token=creds["aws_session_token"],
    region_name=resp["s3_region"],
)
session.client("s3").upload_file("model.tgz", resp["s3_bucket"], resp["s3_key"])

A successful upload returns nothing. boto3 raises an exception if the temporary credentials have expired or the s3_key doesn’t match the one from the prepare step.

Create the model

Commit the upload with source.kind set to model_archive, the same deployment payload you validated, and the s3_key from the prepare step. The response returns the created model and its first deployment:

curl https://api.baseten.co/v1/models \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "source": {
      "kind": "model_archive",
      "name": "my-model",
      "s3_key": "organizations/.../models/.../model.tgz",
      "deployment": {
        "config": { "model_name": "my-model", "resources": { "accelerator": "A10G" }, "weights": [{ "source": "hf://meta-llama/Llama-3.1-8B@main", "mount_location": "/models/llama" }] }
      }
    }
  }'

{
  "model": { "id": "abcd123", "name": "my-model" },
  "deployment": { "id": "1q2w3e4", "status": "BUILDING" }
}

Check the deployment

Creation returns before the deployment is ready. Use the model and deployment IDs from the response to poll GET /v1/models/{model_id}/deployments/{deployment_id} until its status is ACTIVE:

curl https://api.baseten.co/v1/models/abcd123/deployments/1q2w3e4 \
  -H "Authorization: Bearer $BASETEN_API_KEY"

{
  "id": "1q2w3e4",
  "model_id": "abcd123",
  "status": "ACTIVE",
  "environment": "production",
  "active_replica_count": 1
}

Call the model

If the deployment fails, check its deployment logs. Once the deployment is ACTIVE, send inference requests to the model’s predict endpoint, using the model id from the create response and your API key. The request and response shapes match whatever your model’s predict method accepts and returns:

curl https://model-abcd123.api.baseten.co/environments/production/predict \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"prompt": "Hello!"}'

{ "output": "Hello! How can I help you today?" }

Examples

Models

Engines

Custom Docker servers

Custom Python models

Chains

API workflows

Baseten for Model Labs

Create a model with the REST API

Prepare the upload

Upload the archive

Create the model

Check the deployment

Call the model

Next steps

Call your model

Add a deployment

​Prepare the upload

​Upload the archive

​Create the model

​Check the deployment

​Call the model

​Next steps

Call your model

Add a deployment

Prepare the upload

Upload the archive

Create the model

Check the deployment

Call the model

Next steps