Integrate deployed Truss models with stubs
Chains can be combined with existing Truss models using Stubs.
A Stub acts as a substitute (client-side proxy) for a remotely deployed dependency, either a Chainlet or a Truss model. The Stub performs the remote invocations as if it were local by taking care of the transport layer, authentication, data serialization and retries.
Stubs can be integrated into Chainlets by passing in a URL of the deployed
model. They also require
context
to be initialized
(for authentication).
There are various ways how you can make a call to the other deployment:
output_model
argument.predict_async
(recommended) or predict_async
.predict_async_stream
which returns an async
bytes iterator.RPCOptions
.See the StubBase reference for all APIS.