Skip to main content
baseten volume sync copies a remote source into a volume as a new sealed version. The sync runs as a durable background job on Baseten, so nothing downloads to your machine.

​
Start a sync

Pass the source URI with --source and the destination volume with --dest. A tag on --dest tags the version the sync publishes:
The source’s URI scheme selects the provider: Repeat --include and --exclude with glob patterns to filter source-relative paths:
Without --wait, the command returns as soon as Baseten creates the job and prints its sync ID. With --wait, it follows the job until it’s READY, FAILED, or CANCELED, writes progress to stderr, and prints the result, including the new version’s reference, to stdout. Pressing Ctrl+C stops waiting without cancelling the sync, and prints the commands to check or cancel it.

​
Check a sync

Check the status of a sync using the ID returned when you started it:
A READY sync includes the reference of the version it published, and a FAILED sync includes an error code and message.

​
List syncs

List syncs for a destination volume:
The command shows syncs newest first. --dest matches one exact destination reference.

​
Cancel a sync

Cancel a sync using its ID:
Cancelling a sync that already finished returns it unchanged and doesn’t delete a version it published.

​
Fix a failed sync

When the source rejects a sync, Baseten stops the job instead of retrying it and reports one of these error codes. The message names the source URI and, when the provider returned one, its error code, such as NoSuchBucket or HTTP_404. Any other failure reports SYNC_FAILED. Baseten retries transient errors, such as throttling and timeouts, before it fails a sync.

​
Authenticate with a secret

Public sources need no credentials. For a private source, store its credentials as a Baseten secret in the same team, and pass the secret’s name with --auth-secret-name:
The secret’s value depends on the source: These are the same formats that weights sources use. Baseten Training sources authenticate automatically.

​
Authenticate with OIDC or AWS AssumeRole

For S3 and GCS, you can grant access with short-lived credentials instead of a stored secret. Pass the flags for one method; a sync accepts only one authentication method. Volume sync uses its own OIDC workload identity, so a trust policy written for model or training workloads doesn’t cover it. Before your first OIDC sync, set up the AWS OIDC provider or GCP workload identity provider, then scope its trust policy to volume sync, as described in Volume sync in the OIDC guide. Grant the role or service account read access to the source bucket and prefix. Amazon S3 with AWS OIDC:
Google Cloud Storage with GCP OIDC:
Amazon S3 with AWS AssumeRole: pass --auth-aws-assume-role-arn and --auth-aws-assume-role-region. See AWS AssumeRole to set up the role. For every flag, see baseten volume sync.