Skip to main content
Baseten Cloud is the default hosting option. Baseten runs the infrastructure, and you configure autoscaling for each deployment. Models run on a multi-tenant infrastructure layer with namespace- and container-level isolation, and GPUs are never shared across customers. The global compute fleet scales without reservations or capacity planning. Baseten Cloud is self-service. Sign up and start deploying today. For enterprise features, talk to us from your work email. Baseten Cloud architecture: models running on Baseten-managed multi-cloud compute

​
Global compute fleet

Baseten Cloud orchestrates thousands of GPUs across multiple cloud providers and regions globally. Multi-cloud Capacity Management (MCM) provisions and manages that compute, so you avoid cloud lock-in while optimizing for latency, GPU availability, and cost. Benefits include:
  • Geographic distribution: Deploy across multiple cloud providers and regions to serve users with low latency worldwide.
  • GPU availability: Reliable access to a variety of leading-edge GPU types, at the scale your workloads demand.
  • Capacity pooling: Shared infrastructure enables rapid scaling, with no reservations or capacity planning.
  • High availability: Active-active reliability across providers keeps inference available through individual provider outages. See current uptime on the status page.

​
Scaling and performance

The global fleet sets the ceiling on capacity. The Inference Stack determines how fast your model serves requests. Baseten Cloud automatically scales your models based on demand:
  • Scale-to-zero: Set the minimum replica count to zero to eliminate idle compute costs when a model has no traffic.
  • Fast cold starts: The Baseten Delivery Network (BDN) accelerates cold starts through multi-tier caching and single-flight downloads that prevent thundering herd issues during burst scaling.
  • Configurable autoscaling: Set autoscaling thresholds to match your workload requirements.
The Inference Stack optimizes every layer of inference, from parallelized weight downloads and cold start acceleration to load-aware routing and dynamic batching, delivering out-of-the-box performance for throughput-sensitive workloads. Multi-region deployment, automatic health checks, failover, and zero-downtime rolling updates keep your models available.

​
Data security

  • Direct routing: Inference traffic routes directly to your workload plane, not through the control plane.
  • Data residency: Use regional environments to constrain workloads and inference traffic to specific regions.
  • Payload retention: Baseten doesn’t store synchronous inference inputs or outputs. Async inference stores inputs until the request runs; see Async inference. Weight caching is optional, and you can erase cached weights on request.
Baseten Cloud supports SOC 2 Type II, HIPAA, and GDPR compliance programs. See Secure model inference and the Baseten Trust Center for more information.

​
Get started

To deploy your own model on Baseten Cloud, follow Build your first model. To call a hosted model right away, follow the quickstart guide.