Tracing
Investigate the prediction flow in detail
The truss server has OpenTelemetry (OTEL) instrumentation builtin. Additionally, users can add their custom instrumentation.
Traces can be useful to investigate performance bottlenecks or other issues.
By default, tracing is not enabled as it can lead to some minor performance overhead which in some use cases is undesirable. Follow below guides to collect trace data.
Exporting builtin trace data to Honeycomb
To enable data export, create a Honeycomb API key and add it as a secret to baseten. Then make add the following settings to the truss config of the model that you want to enable tracing for:
When making requests to the model, you can provide trace parent IDs with the OTEL
standard header key traceparent
. If not provided Baseten will add random IDs.
An example trace, visualized on Honeycomb, resolving preprocessing, predict and postprocessing. Additionally, these traces have some span events timing (de-)serialization inputs and outputs.
Adding custom OTEL instrumentation
If you want a different resolution of tracing spans and event recording, you also add your own OTEL tracing implementation.
We made sure that our builtin tracing instrumentation does not mix the trace context with user defined tracing.