SSH access

SSH into any running model deployment on Baseten and get a full terminal inside the model container: debug, inspect files, run commands, edit code, or transfer data with scp and sftp. Standard OpenSSH tooling works unchanged, so anything that speaks SSH, from your terminal to your IDE, can reach the container.

Prerequisites

Inference SSH must be enabled for your organization. Contact support to request access.

Baseten account: Sign up and generate an API key.
Baseten CLI: Install the CLI and log in:
Terminal
baseten auth login
OpenSSH client: Pre-installed on macOS and Linux. On Windows, use the OpenSSH optional feature or WSL.

Set up your machine

Run setup once per machine. It generates an SSH keypair and adds a managed block to ~/.ssh/config that routes *.ssh.baseten.co connections through the CLI.

Baseten CLI
Truss CLI

To set up SSH access:

baseten ssh setup

Generated SSH keypair: /Users/<you>/.ssh/baseten/id_ed25519
SSH config updated.
Connect with:
   deployment:    ssh model-<model-id>-<deployment-id>.ssh.baseten.co
   environment:   ssh <environment>.model-<model-id>.ssh.baseten.co
   training job:  ssh training-job-<job-id>-<node>.ssh.baseten.co

Connections authenticate with the profile pinned at setup time. To pin a different profile, pass --profile <name>. Re-running setup refreshes the keypair and the managed config block.If you previously ran truss ssh setup, remove its block (between # --- baseten-ssh --- and # --- end baseten-ssh ---) from ~/.ssh/config first; setup refuses to touch a file that already configures *.ssh.baseten.co hosts outside its own managed block.

To set up SSH access:

uvx truss ssh setup

SSH keypair: /Users/<you>/.ssh/baseten/id_ed25519
Proxy script: /Users/<you>/.ssh/baseten/proxy-command.py
SSH config updated: ~/.ssh/config
Default remote: <remote>

SSH access configured. Connect to a running workload with:

  Training job: ssh training-job-<job-id>-<node>.ssh.baseten.co
  Inference model: ssh model-<model-id>-<deployment-id>.ssh.baseten.co

The Truss setup doesn’t support the <environment>.model-<model-id> hostname form; use the Baseten CLI to connect by environment name.

Enable SSH on a deployment

SSH is enabled per deployment in config.yaml:

Set runtime.remote_ssh.enabled to true in your model’s config.yaml:

config.yaml

model_name: my-model

runtime:
  remote_ssh:
    enabled: true

resources:
  accelerator: H100
  use_gpu: true

Push the model:
Terminal
baseten model push

SSH access is available as soon as the deployment is ACTIVE. Re-deploying without this field disables SSH for the new deployment. Active SSH sessions don’t block scale-to-zero or scale-down; for longer interactive sessions, set a non-zero min_replicas so your replica isn’t reclaimed mid-session.

SSH requires the default container user (app, uid 60000). Setting docker_server.run_as_user_id to a different value is incompatible with SSH and the push will fail validation.

Connect

To connect to a running deployment:

Find the model and deployment IDs:

baseten model deployment list --model-id abc12345

ID       NAME           ENVIRONMENT  STATUS  INSTANCE                    REPLICAS  CREATED
def4567  deployment-31  production   ACTIVE  4x16 - 4 vCPUs, 16 GiB RAM  1         2026-07-03T22:03:00Z
...

Find model IDs with baseten model list.

SSH in with the deployment hostname:

Terminal

ssh model-abc12345-def4567.ssh.baseten.co

You’re connected when you see a shell prompt inside the model container. Your container runs as the non-root app user. To connect to whatever deployment is currently live in an environment, put the environment name in front of the model ID instead of naming a deployment:

Terminal

ssh production.model-abc12345.ssh.baseten.co

The CLI resolves the environment to its current deployment on every connection, so the same hostname keeps working across promotions. This form requires setup through the Baseten CLI.

How it works

When you connect to a *.ssh.baseten.co hostname, the managed ~/.ssh/config block runs the CLI. The CLI calls Baseten’s signing API to issue a short-lived SSH certificate scoped to that deployment, then relays the connection to a running replica’s container. Certificates refresh automatically on every connection, so you never need to manage keys or tokens manually. Authorization uses your existing model permissions, so only users who can manage the model can SSH into it.

Hostname formats

SSH hostnames take two forms, by deployment or by environment:

Hostname

model-<model_id>-<deployment_id>[-<replica_id>].ssh.baseten.co
<environment>.model-<model_id>.ssh.baseten.co

Segment	Description	Example
`model_id`	Model ID (8 lowercase alphanumeric characters). Find it with `baseten model list` or in the deployment URL.	`abc12345`
`deployment_id`	Deployment ID (7 lowercase alphanumeric characters). Each new push creates a new deployment.	`def4567`
`replica_id`	Optional. Suffix that uniquely identifies one replica when the deployment has multiple.	`xyz9a`
`environment`	Environment name. Targets the environment’s current deployment; not combinable with a deployment ID.	`production`

Examples:

Terminal

# Connect to any running replica of this deployment
ssh model-abc12345-def4567.ssh.baseten.co

# Connect to a specific replica by suffix
ssh model-abc12345-def4567-xyz9a.ssh.baseten.co

# Connect to the current deployment in the production environment
ssh production.model-abc12345.ssh.baseten.co

IDE integration

Because setup configures standard OpenSSH, tools that speak SSH can connect with the same hostnames:

VS Code: Install the Remote - SSH extension, then connect to model-<model_id>-<deployment_id>.ssh.baseten.co.
Cursor: Use the built-in SSH remote feature with model-<model_id>-<deployment_id>.ssh.baseten.co.

Target a specific replica

Deployments with autoscaling can have many replicas. By default, Baseten routes your SSH session to one running replica. To pin to a specific replica (useful when reproducing a bug that only appears on one replica), append a unique replica-name suffix to the hostname:

Terminal

ssh model-abc12345-def4567-xyz9a.ssh.baseten.co

Find replica names in the deployment’s logs view in the Baseten dashboard or with baseten model deployment replica list.

Active SSH sessions don’t protect a replica from being scaled down. If the autoscaler removes the replica you’re connected to, your session is terminated along with it.

File transfer

Use scp or sftp with the same hostname to transfer files:

Terminal

# Copy a file into the model container
scp ./data.json model-abc12345-def4567.ssh.baseten.co:/tmp/data.json

# Copy a file out of the container
scp model-abc12345-def4567.ssh.baseten.co:/tmp/output.json ./output.json

# Interactive file browser
sftp model-abc12345-def4567.ssh.baseten.co

Multiple workspaces

With the Baseten CLI, connections authenticate with the profile pinned at setup time. To pin a different profile, re-run setup with --profile <name>; to override for a single connection, set BASETEN_PROFILE:

Terminal

BASETEN_PROFILE=staging ssh model-abc12345-def4567.ssh.baseten.co

With the Truss CLI, include the remote name from ~/.trussrc in the hostname instead: model-abc12345-def4567.<remote>.ssh.baseten.co.

Troubleshooting

”SSH access is not enabled for your organization”

Inference SSH is gated per organization. Contact support to request access.

”ssh proxy rejected the connection; is the workload running with SSH enabled?”

The deployment was pushed without runtime.remote_ssh.enabled: true, or it isn’t ACTIVE with at least one running replica. Add the config field and re-push to create a new deployment with SSH enabled; existing deployments can’t be changed in place. If the deployment is scaled to zero, send a request to wake it, or set a non-zero min_replicas while debugging.

”no SSH keypair found; run `baseten ssh setup` first”

Run baseten ssh setup to configure your machine.

”already configures *.ssh.baseten.co hosts outside the managed block”

Another tool (typically truss ssh setup) has already added Baseten SSH entries to ~/.ssh/config. Remove them and re-run baseten ssh setup. The Truss block sits between # --- baseten-ssh --- and # --- end baseten-ssh ---.

”baseten was not found on your PATH”

The managed config block invokes baseten at connect time. Make sure the binary is on your PATH in the environment where you run ssh (including inside IDEs, which might launch with a reduced PATH).

Truss CLI: “No api_key for remote”

Truss 0.17.2 through 0.18.17 stored your API key in the OS keyring, which the Truss SSH proxy can’t read. Truss 0.18.18 and later keep the key in ~/.trussrc. Re-run login on the latest version to move it back:

Terminal

uvx truss login

Truss CLI: TLS errors

The Truss proxy script requires Python 3.10 or newer. If you see TLS errors, re-run setup with a newer Python interpreter:

Terminal

uvx truss ssh setup --python $(which python3.12)

Next steps

SSH gives you the fastest loop for debugging a live deployment; pair it with the rest of the deployment tooling:

baseten ssh: full command reference, including profile pinning and the managed config block.
Autoscaling: control replica lifecycle so sessions aren’t reclaimed mid-debug.
SSH for training jobs: the same workflow for training containers.

Overview

Get started

Model APIs

Inference

Development

Deployment

Engines

Frontier Gateway

Training

Organization

Observability

Troubleshooting

Prerequisites

Set up your machine

Enable SSH on a deployment

Connect

How it works

Hostname formats

IDE integration

Target a specific replica

File transfer

Multiple workspaces

Troubleshooting

”SSH access is not enabled for your organization”

”ssh proxy rejected the connection; is the workload running with SSH enabled?”

”no SSH keypair found; run `baseten ssh setup` first”

”already configures *.ssh.baseten.co hosts outside the managed block”

”baseten was not found on your PATH”

Truss CLI: “No api_key for remote”

Truss CLI: TLS errors

Next steps

​Prerequisites

​Set up your machine

​Enable SSH on a deployment

​Connect

​How it works

​Hostname formats

​IDE integration

​Target a specific replica

​File transfer

​Multiple workspaces

​Troubleshooting

​”SSH access is not enabled for your organization”

​”ssh proxy rejected the connection; is the workload running with SSH enabled?”

​”no SSH keypair found; run baseten ssh setup first”

​”already configures *.ssh.baseten.co hosts outside the managed block”

​”baseten was not found on your PATH”

​Truss CLI: “No api_key for remote”

​Truss CLI: TLS errors

​Next steps

Prerequisites

Set up your machine

Enable SSH on a deployment

Connect

How it works

Hostname formats

IDE integration

Target a specific replica

File transfer

Multiple workspaces

Troubleshooting

”SSH access is not enabled for your organization”

”ssh proxy rejected the connection; is the workload running with SSH enabled?”

”no SSH keypair found; run `baseten ssh setup` first”

”already configures *.ssh.baseten.co hosts outside the managed block”

”baseten was not found on your PATH”

Truss CLI: “No api_key for remote”

Truss CLI: TLS errors

Next steps