Self-hosted

Self-hosted runs the Baseten workload plane in your own cloud account. Inference payloads stay inside your VPC, and you decide who can reach them. Baseten runs the control plane and keeps the platform current. You control the workload plane’s infrastructure and network boundary.

​

Data security

Self-hosted gives you the greatest control over infrastructure and data boundaries:

• Network isolation: Your workload plane runs in your own VPC, behind your own load balancer and security groups. Model inputs and outputs stay entirely within your infrastructure.
• Payload retention: Baseten doesn’t store synchronous inference inputs or outputs. Async inference inputs queue inside your infrastructure until the request runs; see Async inference.
• Model weight security: Models can pull weights from a private source you control. Talk to your account team to configure a customer-owned weights bucket.
• Private networking: You control inference ingress through your load balancer, security groups, and provider-native networking controls.

Inference payloads never leave your environment. Some platform data does. Model builds run in the Baseten control plane, so your model code and the resulting container images transit Baseten. Your cluster also forwards model metrics and billing data to Baseten for platform operations. Logs and metrics remain in your cluster. Baseten queries them remotely to make them available in the web app and to provide operational support. For the authoritative account of what Baseten processes and retains, see the Baseten Trust Center.

​

Architecture

Self-hosted separates Baseten into a control plane that manages your models and a workload plane that runs inference. The control plane runs in Baseten infrastructure. The workload plane runs in yours. Baseten manages the control plane, which handles the web app, REST API, model building, and workload orchestration. The workload plane is a Kubernetes cluster running in your infrastructure that handles request routing, weight caching, and GPU-backed model serving. The control plane makes scaling decisions, and the replicas run in your cluster. mTLS secures connections between the control plane and your workload plane, using a centrally managed root certificate and a per-customer intermediate certificate authority. Inference requests addressed to your own domain route directly to your workload plane within your VPC.

​

Cloud provider support

Self-hosted runs on the major cloud providers, including AWS, GCP, and OCI. Each workload plane is backed by a Kubernetes cluster running in a single provider region. You can deploy multiple workload planes across regions and clouds for broader geographic coverage. Check with your account team for support on a specific provider.

​

Capacity

You procure and pay for GPU capacity directly with your cloud provider, using existing reservations, cloud credits, or other commitments. GPU availability depends on your cloud provider, region, and account. System components that support the workload plane add overhead costs; your account team can help set expectations.

​

Hybrid

Hybrid runs self-hosted and Baseten Cloud side by side. It fits teams whose infrastructure, data-residency, or regulatory requirements differ across workloads. Constrained workloads run in your own cluster, and eligible workloads run on Baseten Cloud. You define which workloads carry which requirements, and Baseten places each one accordingly:

• Constrained workloads run in your workload plane to meet residency or compliance requirements.
• Unconstrained workloads run on Baseten Cloud’s on-demand capacity, covered by its compliance programs.

Hybrid placement is configured with your account team rather than self-service. Talk to us to scope it.

​

Get started

Self-hosted is available with custom pricing on the Enterprise plan. Talk to us to scope your deployment. We’ll work with you to understand your requirements, set up the workload plane to match how your team operates, and walk through provisioning. For more background, read the blog post: How Baseten Self-hosted Works.