Tracing

Baseten’s Truss server includes built-in OpenTelemetry (OTEL) instrumentation. Export these traces to any OTLP-compatible collector, including collectors that forward data to Datadog, Honeycomb, or another observability provider. Tracing helps diagnose prediction latency and performance bottlenecks. It can add request overhead, so it is disabled by default.

This page covers traces from Dedicated Inference deployments that use the Truss server. It does not configure Model API telemetry or export Model API billing data.

Export built-in traces with OTLP

Send Truss’s built-in spans to an OpenTelemetry Collector or another OTLP/HTTP endpoint:

Configure an OTLP/HTTP receiver that the deployment can reach. For example, use an OpenTelemetry Collector that exports data to your observability provider. To send traces to Datadog, configure the collector with the Datadog exporter.

Set the receiver’s full trace endpoint in config.yaml and enable tracing:

config.yaml

environment_variables:
  OTEL_EXPORTER_OTLP_ENDPOINT: https://otel-collector.example.com/v1/traces
runtime:
  enable_tracing_data: true

Send requests with a traceparent header to continue an existing distributed trace. If you omit the header, Truss generates a trace ID.

OTEL_EXPORTER_OTLP_ENDPOINT must contain the complete OTLP/HTTP trace endpoint. Include /v1/traces when your collector uses the standard OTLP path.

Do not place authentication credentials in environment_variables. If your provider requires credentials, send traces through a collector that stores them securely, or use the secret-backed Honeycomb configuration below.

Export built-in traces to Honeycomb

Truss has a dedicated Honeycomb configuration that reads the API key from Baseten secrets:

Add your Honeycomb API key to Baseten secrets as HONEYCOMB_API_KEY.

Update config.yaml:

config.yaml

environment_variables:
  HONEYCOMB_DATASET: your_dataset_name
runtime:
  enable_tracing_data: true
secrets:
  HONEYCOMB_API_KEY: null

Send requests with a traceparent header to continue an existing distributed trace. If you omit the header, Truss generates a trace ID.

Add custom OTEL instrumentation

Use the OpenTelemetry packages included with the Truss server to create custom spans and events. The following example sends custom spans to an OTLP/HTTP collector:

model.py

import time
from typing import Any, Generator

from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import SERVICE_NAME, Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor

trace_provider = TracerProvider(
    resource=Resource.create({SERVICE_NAME: "UserModel"})
)
trace_provider.add_span_processor(
    BatchSpanProcessor(
        OTLPSpanExporter(
            endpoint="https://otel-collector.example.com/v1/traces"
        )
    )
)
trace.set_tracer_provider(trace_provider)
tracer = trace.get_tracer(__name__)


class Model:
    @tracer.start_as_current_span("load_model")
    def load(self):
        ...

    def preprocess(self, model_input):
        with tracer.start_as_current_span("preprocess"):
            return model_input

    @tracer.start_as_current_span("predict")
    def predict(self, model_input: Any) -> Generator[str, None, None]:
        with tracer.start_as_current_span("start-predict") as span:
            def inner():
                time.sleep(0.01)
                for i in range(5):
                    span.add_event("yield")
                    yield str(i)

            return inner()

Truss isolates its built-in trace context from custom instrumentation. Custom spans use the provider that your model configures and do not become children of Truss’s internal spans.

Get started

Model APIs

Inference

Training

Development

Deployment

Engines

Model Labs

Organization

Hosting options

Observability

Troubleshooting

Export built-in traces with OTLP

Export built-in traces to Honeycomb

Add custom OTEL instrumentation

​Export built-in traces with OTLP

​Export built-in traces to Honeycomb

​Add custom OTEL instrumentation

Export built-in traces with OTLP

Export built-in traces to Honeycomb

Add custom OTEL instrumentation