Deploy a basic Truss for binary I/O
If you need a deployed model to try the invocation examples below, follow these steps to create and deploy a super basic Truss that accepts and returns binary data. The Truss performs no operations and is purely illustrative.Steps for deploying an example Truss
Steps for deploying an example Truss
1
Create a Truss
To create a Truss, run:This creates a Truss in a new directory
binary_test
. By default, newly created Trusses implement an identity function that returns the exact input they are given.2
Add logging
Optionally, modify
binary_test/model/model.py
to log that the data received is of type bytes
:binary_test/model/model.py
3
Deploy the Truss
Deploy the Truss to Baseten with:
Send raw bytes as model input
To send binary data as model input:- Set the
content-type
HTTP header toapplication/octet-stream
- Use
msgpack
to encode the data or file - Make a POST request to the model
Gettysburg.mp3
in the current working directory. You can download the 11-second file from our CDN or replace it with your own file.
call_model.py
To support certain types like numpy and datetime values, you may need to
extend client-side
msgpack
encoding with the same encoder and decoder used
by
Truss.Parse raw bytes from model output
To use the output of a non-streaming model response, decode the response content.call_model.py
Streaming binary outputs
You can also stream output as binary. This is useful for sending large files or reading binary output as it is generated. In themodel.py
, you must create a streaming output.
model/model.py
stream_model.py