Configure sandboxes

Baseten Sandboxes are now available and free to use until November 16. Each organization can run up to 10 sandboxes at once. To raise this limit, contact support.

Set a sandbox’s memory, vCPUs, environment variables, and labels when you create it. You can change labels and environment variables later; memory and vCPUs are fixed at creation. Set up your SDK and API key before running the code.

Choose memory and vCPUs

Set a sandbox’s memory, and Baseten gives it the matching number of vCPUs. Without a memory value, a sandbox gets 4,096 MiB and 2 vCPUs.

Memory	vCPUs
2,048 MiB	1
4,096 MiB	2
8,192 MiB	4
16,384 MiB	6
32,768 MiB	8
65,536 MiB	16
131,072 MiB or more	32

Each account has a maximum memory and vCPU count per sandbox. A request above either returns HTTP 429 with the limit, such as You have reached the maximum memory (4096) for your account. To raise your limits, Contact us. To create a sandbox with 2 GiB of memory:

TypeScript
Python
Go
API
CLI

Run the following code. The sandbox gets 1 vCPU:

import { SandboxClient } from "@basetenlabs/sandbox";

const client = new SandboxClient({ apiKey: process.env.BASETEN_API_KEY! });
const sandbox = await client.create({ name: "hello-world", memory: 2048 });
console.log(sandbox.name);

Run the following code. The sandbox gets 1 vCPU:

import os

from baseten.sandbox import SandboxClient

with SandboxClient(api_key=os.environ["BASETEN_API_KEY"]) as client:
    sandbox = client.create(name="hello-world", memory=2048)
    print(sandbox.name)

Run the following code. The sandbox gets 1 vCPU:

package main

import (
    "context"
    "fmt"
    "log"
    "os"

    "github.com/basetenlabs/baseten-go/sandbox"
)

func main() {
    ctx := context.Background()
    client, err := sandbox.NewClient(sandbox.ClientOptions{
        APIKey: os.Getenv("BASETEN_API_KEY"),
    })
    if err != nil {
        log.Fatal(err)
    }
    sb, err := client.Create(ctx, sandbox.CreateOptions{
        Name:   "hello-world",
        Memory: 2048,
    })
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(sb.Name())
}

Run the following command. The sandbox gets 1 vCPU:

curl --request POST \
  --url https://api.baseten.co/v1/sandboxes/instances \
  --header "Authorization: Bearer $SANDBOX_TOKEN" \
  --header "Content-Type: application/json" \
  --data '{"name": "hello-world", "memory": 2048}'

Run the following command. The sandbox gets 1 vCPU:

baseten sandbox create --name "$SANDBOX_NAME" --memory 2048

In the dashboard, the CPU field fills in the matching count when you enter Memory (MiB).

Set a custom CPU count

Most workloads should keep the vCPU count that matches their memory. Set a CPU count when a workload needs a different balance, such as a compile-heavy task that needs more vCPUs than its memory provides. The CPU count is a whole number from 1 to 32, and memory must be 1,024 to 8,192 MiB per vCPU. For example, 4 vCPUs allow 4,096 to 32,768 MiB. Set memory along with the CPU count: with only a CPU count, memory defaults to 8,192 MiB per vCPU, capped at your account’s memory limit. To create a sandbox with 2 vCPUs and 2 GiB of memory:

Run the following code:

import { SandboxClient } from "@basetenlabs/sandbox";

const client = new SandboxClient({ apiKey: process.env.BASETEN_API_KEY! });
const sandbox = await client.create({ name: "hello-world", memory: 2048, cpu: 2 });
console.log(sandbox.name);

Run the following code:

import os

from baseten.sandbox import SandboxClient

with SandboxClient(api_key=os.environ["BASETEN_API_KEY"]) as client:
    sandbox = client.create(name="hello-world", memory=2048, cpu=2)
    print(sandbox.name)

Run the following code:

package main

import (
    "context"
    "fmt"
    "log"
    "os"

    "github.com/basetenlabs/baseten-go/sandbox"
)

func main() {
    ctx := context.Background()
    client, err := sandbox.NewClient(sandbox.ClientOptions{
        APIKey: os.Getenv("BASETEN_API_KEY"),
    })
    if err != nil {
        log.Fatal(err)
    }
    sb, err := client.Create(ctx, sandbox.CreateOptions{
        Name:   "hello-world",
        Memory: 2048,
        CPU:    2,
    })
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(sb.Name())
}

Run the following command:

curl --request POST \
  --url https://api.baseten.co/v1/sandboxes/instances \
  --header "Authorization: Bearer $SANDBOX_TOKEN" \
  --header "Content-Type: application/json" \
  --data '{"name": "hello-world", "memory": 2048, "cpu": 2}'

Run the following command:

baseten sandbox create --name "$SANDBOX_NAME" --memory 2048 --cpu 2

Sign in to your workspace at app.baseten.co and choose Sandboxes in the sidebar, then choose Create sandbox.
For Memory (MiB), enter 2048.
For CPU, choose 2.
Complete the rest of the form, then choose Create sandbox.

A CPU count with memory outside its range returns HTTP 400 with the allowed range, such as for 8 vCPU, memory must be between 8192 and 65536 MB. The sandbox’s record shows its vCPU count as cpu. To delete the sandbox automatically, set an

Set environment variables

Environment variables pass configuration, such as an output format or log level, to the processes in a sandbox, so one image can serve different tasks. Set them when you create a sandbox, or replace them later. Names begin with a letter or underscore and contain only letters, digits, and underscores. Don’t put your Baseten API key in a sandbox’s environment.

To create a sandbox with environment variables:Run the following code:

import { SandboxClient } from "@basetenlabs/sandbox";

const client = new SandboxClient({ apiKey: process.env.BASETEN_API_KEY! });
const sandbox = await client.create({
  name: "hello-world",
  envs: {
    DB_PASSWORD: { value: "secret" },
    OUTPUT_FORMAT: { value: "json", secret: false },
  },
});
console.log(sandbox.name);

Set secret: false for a value you want to read back later.

To create a sandbox with environment variables:Run the following code:

import os

from baseten.sandbox import SandboxClient, SandboxEnvValue

with SandboxClient(api_key=os.environ["BASETEN_API_KEY"]) as client:
    sandbox = client.create(
        name="hello-world",
        envs={
            "DB_PASSWORD": SandboxEnvValue(value="secret"),
            "OUTPUT_FORMAT": SandboxEnvValue(value="json", secret=False),
        },
    )
    print(sandbox.name)

Set secret=False for a value you want to read back later.

To create a sandbox with environment variables:Run the following code:

package main

import (
    "context"
    "fmt"
    "log"
    "os"

    "github.com/basetenlabs/baseten-go/sandbox"
)

func main() {
    ctx := context.Background()
    client, err := sandbox.NewClient(sandbox.ClientOptions{
        APIKey: os.Getenv("BASETEN_API_KEY"),
    })
    if err != nil {
        log.Fatal(err)
    }
    sb, err := client.Create(ctx, sandbox.CreateOptions{
        Name: "hello-world",
        Envs: map[string]sandbox.EnvValue{
            "DB_PASSWORD":   {Value: "secret"},
            "OUTPUT_FORMAT": {Value: "json", NonSecret: true},
        },
    })
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(sb.Name())
}

Set NonSecret: true for a value you want to read back later.

To create a sandbox with environment variables:Run the following command:

curl --request POST \
  --url https://api.baseten.co/v1/sandboxes/instances \
  --header "Authorization: Bearer $SANDBOX_TOKEN" \
  --header "Content-Type: application/json" \
  --data '{
    "name": "hello-world",
    "envs": [
      {"name": "DB_PASSWORD", "value": "secret"},
      {"name": "OUTPUT_FORMAT", "value": "json", "secret": false}
    ]
  }'

Set "secret": false for a value you want to read back later.

To create a sandbox with environment variables:Run the following command:

baseten sandbox create --name "$SANDBOX_NAME" --env DB_PASSWORD=secret --plain-env OUTPUT_FORMAT=json

--env marks the value secret; --plain-env leaves it readable. To verify a value after deployment, run the following command:

baseten sandbox exec --name "$SANDBOX_NAME" -- 'echo $OUTPUT_FORMAT'

json

To create a sandbox with environment variables:

In Sandboxes, choose Create sandbox, then choose an image.
Under Environment variables, choose Add variable.
Type a name and value, such as OUTPUT_FORMAT and json.
Repeat steps 2 and 3 for each additional variable.
Complete the rest of the form, then choose Create sandbox.

Values are secret by default. Use secret values for credentials and tokens. Baseten masks secret values as **** when you read them back. For configuration you’ll want to see later, such as LOG_LEVEL or OUTPUT_FORMAT, mark the value as not secret so it stays readable. Run the following command to read the sandbox’s environment variables back:

baseten sandbox describe --name "$SANDBOX_NAME" --output json

{
  "envs": [
    {
      "name": "DB_PASSWORD",
      "secret": true,
      "value": "****"
    },
    {
      "name": "OUTPUT_FORMAT",
      "secret": false,
      "value": "json"
    }
  ],
  ...
}

Secret masking protects values in ordinary configuration responses. Code running inside the sandbox can read the actual values, including an agent that can run commands. Only provide credentials that you’re willing to make available to that code. To reveal secret values in the configuration, run the following command:

baseten sandbox describe --name "$SANDBOX_NAME" --show-secrets

Don’t include this output in logs or agent prompts. Don’t pass --plain-env or secret: false for credentials; those values remain readable in ordinary configuration responses.

Replace environment variables

An update that includes environment variables replaces the sandbox’s entire set, and removes any variable you omit. Include every variable you want to keep, including existing secrets with their actual values. Don’t copy masked **** values from a configuration response. An update without environment variables leaves the existing set unchanged. Set DB_PASSWORD in your local shell before running these examples. To replace a sandbox’s environment variables:

TypeScript
Python
Go
API
CLI

Run the following code:

import { SandboxClient } from "@basetenlabs/sandbox";

const client = new SandboxClient({ apiKey: process.env.BASETEN_API_KEY! });
const updated = await client.update({
  name: "hello-world",
  envs: {
    DB_PASSWORD: { value: process.env.DB_PASSWORD! },
    OUTPUT_FORMAT: { value: "json", secret: false },
  },
});
console.log(updated.name);

Run the following code:

import os

from baseten.sandbox import SandboxClient, SandboxEnvValue

with SandboxClient(api_key=os.environ["BASETEN_API_KEY"]) as client:
    info = client.update(
        name="hello-world",
        envs={
            "DB_PASSWORD": SandboxEnvValue(value=os.environ["DB_PASSWORD"]),
            "OUTPUT_FORMAT": SandboxEnvValue(value="json", secret=False),
        },
    )
    print(info.name)

Run the following code:

package main

import (
    "context"
    "fmt"
    "log"
    "os"

    "github.com/basetenlabs/baseten-go/sandbox"
)

func main() {
    ctx := context.Background()
    client, err := sandbox.NewClient(sandbox.ClientOptions{
        APIKey: os.Getenv("BASETEN_API_KEY"),
    })
    if err != nil {
        log.Fatal(err)
    }
    updated, err := client.Update(ctx, sandbox.UpdateOptions{
        Name: "hello-world",
        Envs: map[string]sandbox.EnvValue{
            "DB_PASSWORD":   {Value: os.Getenv("DB_PASSWORD")},
            "OUTPUT_FORMAT": {Value: "json", NonSecret: true},
        },
    })
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(updated.Name)
}

Run the following command:

curl --request PATCH \
  --url "https://api.baseten.co/v1/sandboxes/instances/$SANDBOX_NAME" \
  --header "Authorization: Bearer $SANDBOX_TOKEN" \
  --header "Content-Type: application/json" \
  --data '{
    "envs": [
      {"name": "DB_PASSWORD", "value": "'"$DB_PASSWORD"'"},
      {"name": "OUTPUT_FORMAT", "value": "json", "secret": false}
    ]
  }'

Run the following command:

baseten sandbox update --name "$SANDBOX_NAME" \
  --env "DB_PASSWORD=$DB_PASSWORD" --plain-env OUTPUT_FORMAT=json

For more information, see Update a sandbox.

Change sandbox labels

Labels help you find and group sandboxes, such as marking every sandbox an agent creates with purpose=agent. A new set of labels replaces all previous labels, so include every one you want to keep. This example changes the purpose label from hello-world to hello-universe.

To change sandbox labels:Run the following code:

import { SandboxClient } from "@basetenlabs/sandbox";

const client = new SandboxClient({ apiKey: process.env.BASETEN_API_KEY! });
const updated = await client.update({
  name: "hello-world",
  labels: { purpose: "hello-universe" },
});
console.log(updated.labels);

To change sandbox labels:Run the following code:

import os

from baseten.sandbox import SandboxClient

with SandboxClient(api_key=os.environ["BASETEN_API_KEY"]) as client:
    info = client.update(name="hello-world", labels={"purpose": "hello-universe"})
    print(info.labels)

To change sandbox labels:Run the following code:

package main

import (
    "context"
    "fmt"
    "log"
    "os"

    "github.com/basetenlabs/baseten-go/sandbox"
)

func main() {
    ctx := context.Background()
    client, err := sandbox.NewClient(sandbox.ClientOptions{
        APIKey: os.Getenv("BASETEN_API_KEY"),
    })
    if err != nil {
        log.Fatal(err)
    }
    updated, err := client.Update(ctx, sandbox.UpdateOptions{
        Name:   "hello-world",
        Labels: map[string]string{"purpose": "hello-universe"},
    })
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(updated.Labels)
}

To change sandbox labels:Run the following command:

curl --request PATCH \
  --url "https://api.baseten.co/v1/sandboxes/instances/$SANDBOX_NAME" \
  --header "Authorization: Bearer $SANDBOX_TOKEN" \
  --header "Content-Type: application/json" \
  --data '{"labels": {"purpose": "hello-universe"}}'

To change sandbox labels:Run the following command:

baseten sandbox update --name "$SANDBOX_NAME" --label purpose=hello-universe

The command prints the updated sandbox.

Select a team

If you belong to one team, sandbox requests use it. If you belong to more than one, select the team that owns the sandbox in every request that manages it. To select a team:

TypeScript
Python
Go
API
CLI

Replace <TEAM_ID> with your team’s ID, then configure the client as follows:

import { SandboxClient } from "@basetenlabs/sandbox";

const client = new SandboxClient({
  apiKey: process.env.BASETEN_API_KEY!,
  teamId: "<TEAM_ID>",
});

Replace <TEAM_ID> with your team’s ID, then configure the client as follows:

import os

from baseten.sandbox import SandboxClient

client = SandboxClient(api_key=os.environ["BASETEN_API_KEY"], team_id="<TEAM_ID>")

Replace <TEAM_ID> with your team’s ID, then configure the client as follows:

package main

import (
    "log"
    "os"

    "github.com/basetenlabs/baseten-go/sandbox"
)

func main() {
    _, err := sandbox.NewClient(sandbox.ClientOptions{
        APIKey: os.Getenv("BASETEN_API_KEY"),
        TeamID: "<TEAM_ID>",
    })
    if err != nil {
        log.Fatal(err)
    }
}

Send the team’s ID in the X-Team-Id header with each request:

Authorization: Bearer <SANDBOX_TOKEN>
X-Team-Id: <TEAM_ID>

For more information, see Authentication and teams.

Run the following command to list your organization’s teams:
baseten org team list
Replace <TEAM_ID> with the team’s name or ID, then pass it with --team to each sandbox command, such as the following:
baseten sandbox list --team <TEAM_ID>

For more information, see Create a sandbox and Update a sandbox.

Get started

Model APIs

Baseten Grid

Inference

Frontier Gateway

Training

Sandboxes

Development

Deployment

Engines

Model Labs

Organization

Hosting options

Observability

Troubleshooting

Configure sandboxes

Choose memory and vCPUs

Set a custom CPU count

Set environment variables

Replace environment variables

Change sandbox labels

Select a team

​Choose memory and vCPUs

​Set a custom CPU count

​Set environment variables

​Replace environment variables

​Change sandbox labels

​Select a team

Choose memory and vCPUs

Set a custom CPU count

Set environment variables

Replace environment variables

Change sandbox labels

Select a team