Skip to main content
Custom metrics measure behavior inside your model server, such as cache hits, batch sizes, or validation failures. Expose these metrics through a Prometheus endpoint that your observability system can scrape, or push them directly to your provider with OpenTelemetry.

​
Expose a Prometheus endpoint

If you deploy a custom server, add a route that returns your metrics in Prometheus format. For example, a /metrics route is available through this Baseten endpoint:
URL
Configure your external collector to scrape this URL with a Baseten API key that can invoke the model. Send the key in the Authorization header with the Bearer scheme. A workspace-export-metrics key only authorizes the organization metrics endpoint, so don’t use it for this route. Check the route with the Baseten CLI before you configure a collector:
The response contains the Prometheus metadata and samples from your server:
Every scrape sends a request to the deployment. If the deployment has scaled to zero, a scrape triggers a cold start. Regular scrapes can keep a replica active. See Sync API endpoints for route mapping and the Prometheus configuration reference for scrape options.

​
Push metrics with OpenTelemetry

You can also instrument your model server with an OpenTelemetry SDK and push metrics to an OTLP endpoint. The exporter sends each batch to your collector or observability provider. To configure push-based export:
  1. Add the OpenTelemetry metrics SDK and an OTLP exporter for your language.
  2. Create counters, histograms, or gauges for the application behavior you want to measure.
  3. Configure the exporter with your provider’s OTLP endpoint and authentication settings.
  4. Store provider credentials as Baseten secrets, not in your model code or configuration.
  5. For a custom server, add the Baseten-provided BT_MODEL_ID and BT_MODEL_DEPLOYMENT_ID values as resource attributes so you can filter metrics by model and deployment.
For a Python implementation, see the OpenTelemetry guides for metrics instrumentation and OTLP exporters.