Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.baseten.co/llms.txt

Use this file to discover all available pages before exploring further.

SSH (Secure Shell) is a protocol for encrypted, authenticated access to a remote machine. With model deployments on Baseten, you can SSH into any running deployment and get a full terminal inside the model container to debug, inspect files, run commands, edit code, or transfer data with scp and sftp. This gives you the same control you’d have on a local box, just running on Baseten-managed hardware.

​
Prerequisites

  • Baseten account: Sign up and generate an API key.
  • uv: This guide uses uvx to run Truss commands without a separate install step. Log in to Baseten: 
    uvx truss login
  • OpenSSH client: Pre-installed on macOS and Linux. On Windows, use the OpenSSH optional feature or WSL.

​
Configuration

To enable SSH access on a deployment, set runtime.remote_ssh.enabled to true in your config.yaml:
config.yaml
model_name: my-model

runtime:
  remote_ssh:
    enabled: true

resources:
  accelerator: H100
  use_gpu: true
Then push the model: 
uvx truss push
SSH access is available as soon as the deployment is ACTIVE. Re-deploying without this field disables SSH for the new deployment.
SSH requires the default container user (app, uid 60000). Setting docker_server.run_as_user_id to a different value is incompatible with SSH and the push will fail validation.
Active SSH sessions don’t block scale-to-zero or scale-down. For longer interactive sessions, set a non-zero min_replicas so your replica isn’t reclaimed mid-session.

​
Quick start

This walkthrough pushes a small model with SSH enabled, then connects to it from your terminal.

​
1. Set up your machine

Run uvx truss ssh setup once to configure OpenSSH: 
uvx truss ssh setup
This generates an SSH keypair, installs a ProxyCommand helper, and adds wildcard Host entries to ~/.ssh/config. You only need to do this once per machine. The expected output is: 
SSH keypair: /Users/<you>/.ssh/baseten/id_ed25519
Proxy script: /Users/<you>/.ssh/baseten/proxy-command.py
SSH config updated: ~/.ssh/config
Default remote: <remote>

SSH access configured. Connect to a running workload with:

  Training job: ssh training-job-<job-id>-<node>.ssh.baseten.co
  Inference model: ssh model-<model-id>-<deployment-id>.ssh.baseten.co
If you’ve already run setup on this machine, the first line instead reads WARNING: Existing SSH keypair found at <path>, reusing it. You can safely ignore it.

​
2. Enable SSH and push

In your model’s config.yaml, add the runtime.remote_ssh block shown in Configuration, then push: 
uvx truss push
The expected output ends with a deployment URL and IDs. Note both the model ID (8 characters) and the deployment ID (7 characters) as you’ll use them to connect.

​
3. Connect

Once the deployment is ACTIVE, SSH in with: 
ssh model-<model_id>-<deployment_id>.ssh.baseten.co
For example: 
ssh model-abc12345-def4567.ssh.baseten.co
You’re connected when you see a shell prompt inside the model container. Your container runs as the non-root app user.

​
How it works

When you run ssh model-<model_id>-<deployment_id>.ssh.baseten.co, Baseten authenticates you using the API key stored by uvx truss ssh setup, issues a short-lived SSH certificate scoped to that deployment, and routes the connection to a running replica’s container. Certificates refresh automatically on every connection, so you never need to manage keys or tokens manually. Authorization uses your existing model permissions so only users who can manage the model can SSH into it.

​
Hostname format

model-<model_id>-<deployment_id>[-<replica_id>].ssh.baseten.co
SegmentDescriptionExample
model_idModel ID (8 lowercase alphanumeric characters). Find it in the deployment URL or with the Baseten CLI.abc12345
deployment_idDeployment ID (7 lowercase alphanumeric characters). Each new push creates a new deployment.def4567
replica_idOptional. Suffix that uniquely identifies one replica when the deployment has multiple.xyz9a
Examples: 
# Connect to any running replica of this deployment
ssh model-abc12345-def4567.ssh.baseten.co

# Connect to a specific replica by suffix
ssh model-abc12345-def4567-xyz9a.ssh.baseten.co

​
IDE integration

Because uvx truss ssh setup configures standard OpenSSH, tools that speak SSH can connect with the same hostname:
  • VS Code: Install the Remote - SSH extension, then connect to model-<model_id>-<deployment_id>.ssh.baseten.co.
  • Cursor: Use the built-in SSH remote feature with model-<model_id>-<deployment_id>.ssh.baseten.co.

​
Targeting a specific replica

Deployments with autoscaling can have many replicas. By default, Baseten routes your SSH session to one running replica. To pin to a specific replica — useful when reproducing a bug that only appears on one pod — append a unique replica-name suffix to the hostname: 
ssh model-abc12345-def4567-xyz9a.ssh.baseten.co
You can find replica names in the deployment’s logs view in the Baseten dashboard or by listing pods via the platform’s debug tools.
Active SSH sessions do not protect a replica from being scaled down. If the autoscaler decides to remove the replica you’re connected to, your session will be terminated along with it. For long debugging sessions, consider raising min_replicas so the replica isn’t reclaimed mid-session.

​
File transfer

Use scp or sftp with the same hostname to transfer files: 
# Copy a file into the model container
scp ./data.json model-abc12345-def4567.ssh.baseten.co:/tmp/data.json

# Copy a file out of the container
scp model-abc12345-def4567.ssh.baseten.co:/tmp/output.json ./output.json

# Interactive file browser
sftp model-abc12345-def4567.ssh.baseten.co

​
Multiple remotes

If you only have one remote configured in ~/.trussrc, you can skip this section. Baseten uses it automatically. If you have multiple remotes, include the remote name in the hostname so the proxy script knows which credentials to use: 
model-<model_id>-<deployment_id>.<remote>.ssh.baseten.co
For example, to connect using the baseten-dev remote: 
ssh model-abc12345-def4567.baseten-dev.ssh.baseten.co

​
Troubleshooting

​
”SSH is not enabled for this deployment”

The deployment was pushed without runtime.remote_ssh.enabled: true. Add it to config.yaml and re-push to create a new deployment with SSH enabled. Existing deployments cannot be changed in place.

​
”SSH keypair not found” or “command not found”

Run uvx truss ssh setup to configure your machine.

​
Connection refused or deployment unreachable

SSH requires the deployment to be in the ACTIVE state with at least one running replica. If the deployment is scaled to zero, send a request to wake it, or set a non-zero min_replicas while debugging. Confirm the deployment status in the Baseten dashboard.

​
TLS errors

The proxy script requires Python 3.10 or newer. If you see TLS errors, re-run setup with a newer Python interpreter: 
uvx truss ssh setup --python $(which python3.12)