> ## Documentation Index
> Fetch the complete documentation index at: https://docs.baseten.co/llms.txt
> Use this file to discover all available pages before exploring further.

# SSH access

> Connect to training containers directly from your terminal with standard SSH.

SSH into any running training job on Baseten and get a full terminal inside the training container: debug, inspect files, run commands, edit code, or transfer data with `scp` and `sftp`. You get the same control you'd have on a local GPU box, running on Baseten-managed hardware.

Unlike [VS Code and Cursor remote tunnels](/training/interactive-sessions), SSH is terminal-first and works with any OpenSSH-compatible tool.

## Prerequisites

<Note>
  SSH sessions must be enabled for your Baseten workspace. [Contact support](mailto:support@baseten.co) to request access.
</Note>

* **Baseten account**: [Sign up](https://app.baseten.co/) and generate an [API key](https://app.baseten.co/settings/account/api_keys).
* **Baseten CLI**: [Install the CLI](/reference/cli/baseten/overview#install) and log in:

  ```sh Terminal theme={"system"}
  baseten auth login
  ```
* **[uv](https://docs.astral.sh/uv/)**: The Baseten CLI runs training commands through Truss, which it launches with `uv`, so keep `uv` on your `PATH`. Your `baseten auth login` credentials are forwarded to Truss automatically.
* **OpenSSH client**: Pre-installed on macOS and Linux. On Windows, use the OpenSSH optional feature or WSL.

## Enable SSH on a training job

To enable SSH access on a training job, set `session_provider` to `SSH` in your `config.py`:

```python config.py {17-20} theme={"system"}
from truss_train import TrainingProject, TrainingJob, Image, Compute, Runtime
from truss_train.definitions import (
    InteractiveSession,
    InteractiveSessionTrigger,
    InteractiveSessionProvider,
)
from truss.base.truss_config import AcceleratorSpec

training_job = TrainingJob(
    image=Image(base_image="pytorch/pytorch:2.7.0-cuda12.8-cudnn9-runtime"),
    compute=Compute(
        accelerator=AcceleratorSpec(accelerator="H200", count=1),
    ),
    runtime=Runtime(
        start_commands=["python train.py"],
    ),
    interactive_session=InteractiveSession(
        trigger=InteractiveSessionTrigger.ON_STARTUP,
        session_provider=InteractiveSessionProvider.SSH,
    ),
)

training_project = TrainingProject(name="my-training-project", job=training_job)
```

This example sets `trigger=ON_STARTUP` so the session is available as soon as the job starts running. The default is `ON_DEMAND`, which activates the session on your first SSH connection; see [Trigger modes](/training/remote-access#trigger-modes) for all options.

Training jobs support H200 and H100 GPUs. For all `InteractiveSession` fields, see the [SDK reference](/reference/sdk/training#interactivesession). For supported compute, see [Compute resources](/training/concepts/basics#compute-resources).

## Create a workstation with the CLI

To create an SSH-enabled workstation without writing a `config.py`, use the `baseten train workstation create` command. It configures an interactive session with `InteractiveSessionTrigger.ON_STARTUP` and runs `sleep infinity` to keep the container alive:

```sh Terminal theme={"system"}
# Default: 1x H100
baseten train workstation create

# 4x H200 with custom project name
baseten train workstation create --accelerator H200 --gpu-count 4 --project my-dev-box

# Custom base image
baseten train workstation create --image pytorch/pytorch:2.7.0-cuda12.8-cudnn9-runtime
```

Once the workstation is running, connect using the SSH command provided in the output. See the [`baseten train workstation`](/reference/cli/baseten/train-workstation) reference for all available options.

For multi-node workstations, pass `--node-count` instead of `--gpu-count`: Baseten bootstraps a Slurm cluster across the nodes. See [Slurm workstations](/training/slurm).

## Push an SSH-enabled training job

This walkthrough uses the
[MNIST PyTorch example](https://github.com/basetenlabs/ml-cookbook/tree/main/examples/mnist-pytorch/training)
to push a training job with SSH enabled, then connects to the container from your terminal.

### Set up your machine

Run setup once per machine. It generates an SSH keypair and adds a managed block to `~/.ssh/config` that routes `*.ssh.baseten.co` connections through the CLI.

<Tabs>
  <Tab title="Baseten CLI">
    **To set up SSH access**:

    <CodeGroup>
      ```sh Terminal theme={"system"}
      baseten ssh setup
      ```

      ```text Output theme={"system"}
      Generated SSH keypair: /Users/<you>/.ssh/baseten/id_ed25519
      SSH config updated.
      Connect with:
         deployment:    ssh model-<model-id>-<deployment-id>.ssh.baseten.co
         environment:   ssh <environment>.model-<model-id>.ssh.baseten.co
         training job:  ssh training-job-<job-id>-<node>.ssh.baseten.co
      ```
    </CodeGroup>

    Connections authenticate with the profile pinned at setup time. To pin a different profile, pass `--profile <name>`. Re-running setup refreshes the keypair and the managed config block.

    If you previously ran `truss ssh setup`, remove its block (between `# --- baseten-ssh ---` and `# --- end baseten-ssh ---`) from `~/.ssh/config` first; setup refuses to touch a file that already configures `*.ssh.baseten.co` hosts outside its own managed block.
  </Tab>

  <Tab title="Truss CLI">
    **To set up SSH access**:

    <CodeGroup>
      ```sh Terminal theme={"system"}
      uvx truss ssh setup
      ```

      ```text Output theme={"system"}
      SSH keypair: /Users/<you>/.ssh/baseten/id_ed25519
      Proxy script: /Users/<you>/.ssh/baseten/proxy-command.py
      SSH config updated: ~/.ssh/config
      Default remote: <remote>

      SSH access configured. Connect to a running workload with:

        Training job: ssh training-job-<job-id>-<node>.ssh.baseten.co
        Inference model: ssh model-<model-id>-<deployment-id>.ssh.baseten.co
      ```
    </CodeGroup>

    Where `<remote>` is your default remote from `~/.trussrc` (typically `baseten`).
  </Tab>
</Tabs>

### Clone the example

Clone the cookbook and change into the MNIST training example:

```sh Terminal theme={"system"}
git clone https://github.com/basetenlabs/ml-cookbook.git
cd ml-cookbook/examples/mnist-pytorch/training
```

### Configure and push the job

Edit `config.py` to add an `interactive_session` with `session_provider=SSH`, as shown in [Enable SSH on a training job](#enable-ssh-on-a-training-job). Then push:

```sh Terminal theme={"system"}
uvx truss train push config.py
```

The expected output is:

```text Output theme={"system"}
✨ Training job successfully created!
🪵 View logs for your job via 'truss train logs --job-id <job_id> --tail'
🔍 View metrics for your job via 'truss train metrics --job-id <job_id>'
🌐 View job in the UI: https://app.baseten.co/training/<project_id>/logs/<job_id>
```

Where:

* `<job_id>` is your new training job's ID. Note it down; you'll use it to connect in the next step.
* `<project_id>` is the Baseten-assigned ID for the training project.

### Connect

Once the job is running, find its ID with `uvx truss train view`, then SSH in:

```sh Terminal theme={"system"}
ssh training-job-<job_id>-0.ssh.baseten.co
```

For example, to connect to node 0 of job `abc1234`:

```sh Terminal theme={"system"}
ssh training-job-abc1234-0.ssh.baseten.co
```

You're connected when you see a shell prompt like `root@baseten-training-job-<job_id>-multinode-0:~#`.

By default, your source files are extracted to `/b10/workspace` (available as `$BT_WORKING_DIR`). If you set [`enable_baseten_workdir=False`](/reference/sdk/training#param-enable-baseten-workdir), Baseten uses your base image's `WORKDIR` instead.

## How it works

When you connect to a `training-job-*.ssh.baseten.co` hostname, the managed `~/.ssh/config` block runs the CLI. The CLI calls the [training job SSH signing API](/reference/training-api/signs-an-ssh-certificate-for-a-training-job) to issue a short-lived certificate and relays the connection to the correct job. Certificates refresh automatically on every connection, so you never need to manage keys or tokens manually.

## Hostname format

Training hostnames name a job and a node:

```text Hostname theme={"system"}
training-job-<job_id>-<node>.ssh.baseten.co
```

| Segment  | Description                                                                       | Example   |
| -------- | --------------------------------------------------------------------------------- | --------- |
| `job_id` | Training job ID. Find it with `uvx truss train view` or in the Baseten dashboard. | `abc1234` |
| `node`   | Node index, starting at 0.                                                        | `0`       |

Examples:

```sh Terminal theme={"system"}
# Single-node job
ssh training-job-abc1234-0.ssh.baseten.co

# Second node of a multi-node job
ssh training-job-xyz5678-1.ssh.baseten.co
```

## IDE integration

Because setup configures standard OpenSSH, tools that speak SSH can connect with the same hostname:

* **VS Code**: Install the [Remote - SSH](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-ssh) extension, then connect to `training-job-<job_id>-<node>.ssh.baseten.co`.
* **Cursor**: Use the built-in SSH remote feature with `training-job-<job_id>-<node>.ssh.baseten.co`.

## Multi-node jobs

For [multi-node training jobs](/training/concepts/multinode), specify the node index in the hostname. Node 0 is the leader:

```sh Terminal theme={"system"}
# Leader node
ssh training-job-abc1234-0.ssh.baseten.co

# Worker node 1
ssh training-job-abc1234-1.ssh.baseten.co
```

## File transfer

Use `scp` or `sftp` with the same hostname to transfer files:

```sh Terminal theme={"system"}
# Copy a file to the training container
scp ./data.csv training-job-abc1234-0.ssh.baseten.co:/workspace/data.csv

# Copy results from the container
scp training-job-abc1234-0.ssh.baseten.co:/workspace/results.json ./results.json

# Interactive file browser
sftp training-job-abc1234-0.ssh.baseten.co
```

## Multiple workspaces

With the Baseten CLI, connections authenticate with the [profile](/reference/cli/baseten/auth) pinned at setup time. To pin a different profile, re-run setup with `--profile <name>`; to override for a single connection, set `BASETEN_PROFILE`:

```sh Terminal theme={"system"}
BASETEN_PROFILE=staging ssh training-job-abc1234-0.ssh.baseten.co
```

With the Truss CLI, include the remote name from `~/.trussrc` in the hostname instead: `training-job-abc1234-0.<remote>.ssh.baseten.co`.

## Session management

For how to view session status, change triggers, and extend session expiry, see the [Remote access overview](/training/remote-access#session-management).

## Troubleshooting

### "Invalid job id: must be a valid hash id"

Check that the job ID in the hostname is correct. Find your job ID with `uvx truss train view` or in the Baseten dashboard.

### "no SSH keypair found; run `baseten ssh setup` first"

Run `baseten ssh setup` to configure your machine.

### "already configures \*.ssh.baseten.co hosts outside the managed block"

Another tool (typically `truss ssh setup`) has already added Baseten SSH entries to `~/.ssh/config`. Remove them and re-run `baseten ssh setup`. The Truss block sits between `# --- baseten-ssh ---` and `# --- end baseten-ssh ---`.

### Connection refused or job unreachable

SSH requires the training job to be in the `RUNNING` state. Check with:

```sh Terminal theme={"system"}
uvx truss train view --job-id <job_id>
```

If the job is running but SSH still fails, the job may not have SSH enabled. Confirm `session_provider=InteractiveSessionProvider.SSH` is set in your config, as shown in [Enable SSH on a training job](#enable-ssh-on-a-training-job).

### Truss CLI: "No api\_key for remote"

Truss 0.17.2 through 0.18.17 stored your API key in the OS keyring, which the Truss SSH proxy can't read. Truss 0.18.18 and later keep the key in `~/.trussrc`. Re-run login on the latest version to move it back:

```sh Terminal theme={"system"}
uvx truss login
```

### Truss CLI: TLS errors

The Truss proxy script requires Python 3.10 or newer. If you see TLS errors, re-run setup with a newer Python interpreter:

```sh Terminal theme={"system"}
uvx truss ssh setup --python $(which python3.12)
```

### Missing Python certificates

If `ssh` fails with a `CERTIFICATE_VERIFY_FAILED` error, the Python interpreter the proxy
script runs under has no SSL root certificates, so it cannot verify TLS connections to
Baseten. To fix it, either install certificates for that Python, or re-run setup pointing
at a Python that already has them:

```sh Terminal theme={"system"}
uvx truss ssh setup --python $(which python3.12)
```

This is most common on macOS with Python installed from [python.org](https://www.python.org),
which ships without certificates until you run its installer:

```sh Terminal theme={"system"}
# Substitute your Python version, e.g. 3.12
open "/Applications/Python 3.12/Install Certificates.command"
```

On other platforms, make sure your Python has a root certificate bundle (for example the
`ca-certificates` package on Debian/Ubuntu).

## Next steps

SSH is one of several ways into a running job; these pages cover the rest of the remote-access surface:

* [Remote access overview](/training/remote-access): trigger modes, session status, and expiry.
* [VS Code and Cursor tunnels](/training/interactive-sessions): editor-native remote sessions instead of a raw terminal.
* [Slurm workstations](/training/slurm): multi-node workstations with a Slurm cluster bootstrapped for you.