> ## Documentation Index
> Fetch the complete documentation index at: https://docs.baseten.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Loops

> Resolve capacity and quota errors and stop interrupted Loops runs.

Resolve capacity and quota errors and stop a run after an interruption.

<span id="check-model-access" />

## Resolve capacity and quota errors

Use the error message and HTTP status code to choose the next step:

| Status | Error message | What to do |
| - | - | - |
| `429` | `Could not schedule trainer. Please try again later.` | Retry later. If the error persists, [contact support](mailto:support@baseten.co) with your model, sequence length, and full error message. |
| `403` | `Your organization has reached its Loops trainer capacity. Stop an existing trainer deployment or contact your Baseten representative to raise the limit.` | [Deactivate a run](#stop-a-run-after-an-interruption) to free trainer GPUs, or [request a higher limit](/loops/supported-models#request-more-access) for the concurrency you need. |

This `429` means no eligible GPU capacity is available. Loops rejects the request before creating a trainer or run instead of queuing it.

## Handle SDK request errors

The SDK treats provisioning errors and requests to a running trainer or sampler differently. Use the operation and error message to decide whether to wait or inspect the run:

| Request | Status or error | SDK behavior |
| - | - | - |
| Trainer or sampler requests using readiness retries | `400`, `404`, `408`, or `502` | Retries with backoff until `ready_timeout` expires. |
| Sampling | `503` | Retries with backoff until `ready_timeout` expires. A retry can restart generation. |
| Trainer operation submission or retrieval | `ServerShutdownError` (`503`) | Reports that the server is shutting down instead of treating it as a readiness retry. |

Read sampling retry warnings to identify repeated generation attempts. For an interrupted trainer operation, inspect the run's status before restarting work. A client timeout doesn't stop the run; follow [Stop a run after an interruption](#stop-a-run-after-an-interruption) when cleanup is needed.

## Resolve accelerator errors

If provisioning fails with either of these messages, contact your organization admin or [Baseten support](mailto:support@baseten.co):

* `accelerator priority ... has no overlap with your organization's allowlist`: The requested GPU type isn't permitted by your organization's configuration.
* `has no candidate on those accelerators`: The permitted GPUs don't support the requested model and sequence length.

Include the model ID, maximum sequence length, and full error message.

## Stop a run after an interruption

An SDK timeout stops the client from waiting; it doesn't stop the work on Baseten. If your script or machine stops before cleanup, deactivate the run before retrying.

Use the `run_id` saved by your script. If you only have a `session_id`, [find the run](#find-a-run-from-its-session-id) first.

Set your [API key](/organization/api-keys) and run ID:

```bash theme={"system"}
export BASETEN_API_KEY="YOUR_API_KEY"
export RUN_ID="YOUR_RUN_ID"
```

<Tabs>
  <Tab title="Baseten CLI">
    Use the [Baseten CLI](/reference/cli/baseten/overview#install) to [deactivate the run](/reference/cli/baseten/loops-run#deactivate). `--yes` skips the confirmation prompt:

    ```bash theme={"system"}
    baseten loops run deactivate --run-id "$RUN_ID" --yes
    ```

    Then [check the run's status](/reference/cli/baseten/loops-run#describe):

    ```bash theme={"system"}
    baseten loops run describe --run-id "$RUN_ID" \
      --jq '{run: .status.name, sampler: .sampler.status.name}'
    ```
  </Tab>

  <Tab title="REST API">
    Call [Deactivate a run](/reference/loops-api/runs/deactivate-a-run):

    ```bash theme={"system"}
    curl --fail-with-body --request POST \
      "https://api.baseten.co/v1/loops/runs/$RUN_ID/deactivate" \
      --header "Authorization: Bearer $BASETEN_API_KEY"
    ```

    Then call [Get a run](/reference/loops-api/runs/get-a-run) and check `run.status.name` and `run.sampler.status.name` in the response:

    ```bash theme={"system"}
    curl --fail-with-body \
      "https://api.baseten.co/v1/loops/runs/$RUN_ID" \
      --header "Authorization: Bearer $BASETEN_API_KEY"
    ```
  </Tab>
</Tabs>

Confirm that the run and sampler report `INACTIVE`. If the run never created a sampler, `sampler` is `null`.

## Find a run from its session ID

The training client can time out before returning a run ID. If your script saved a `session_id`, list your runs and find the matching session.

<Tabs>
  <Tab title="Baseten CLI">
    [List your runs](/reference/cli/baseten/loops-run#list) and filter by the saved session ID. Replace `YOUR_SESSION_ID` with that value. `--all` includes inactive runs:

    ```bash theme={"system"}
    baseten loops run list --all \
      --jq '.runs[] | select(.session_id == "YOUR_SESSION_ID") | {id, status}'
    ```
  </Tab>

  <Tab title="REST API">
    Call [List runs](/reference/loops-api/runs/list-runs):

    ```bash theme={"system"}
    curl --fail-with-body \
      "https://api.baseten.co/v1/loops/runs" \
      --header "Authorization: Bearer $BASETEN_API_KEY"
    ```

    In the response's `runs` array, find the entry whose `session_id` matches your saved value.
  </Tab>
</Tabs>

Use the matching run's `id` as `RUN_ID` in the deactivation command. If its status is already `INACTIVE`, it doesn't need deactivation.