Add an existing Truss model to a Chain

If you have a working Truss model, either a model.py implementation or a custom server such as vLLM, you can add it to a Chain without rewriting it as a ChainletBase. Declare it as a TrussChainlet and it deploys as part of the chain. Other chainlets call it through a TrussHandle, which gives them the URL and headers to call it with their own HTTP or WebSocket client. A TrussChainlet only receives calls. It can’t be the chain’s entrypoint, and it can’t call other chainlets. If your model is already deployed on its own, outside the chain, call it with a stub instead.

TrussChainlet requires truss>=0.18.5.

Declare a TrussChainlet

In your chain file, subclass chains.TrussChainlet and point truss_dir at the Truss directory. A relative path is resolved from the file that declares the class:

chain.py

import truss_chains as chains


class STT(chains.TrussChainlet):
    truss_dir = "./stt_truss"

The Truss directory needs no changes to join a chain. It can be a standard Truss with a Model class in model/model.py, or a docker_server Truss that runs a custom server image such as vLLM and has no model/ directory at all. Deploy the chain as usual:

truss chains push chain.py

Call a TrussChainlet

When one chainlet depends on another ChainletBase, it calls it through a generated stub with typed methods. A TrussChainlet gets no stub, because the framework doesn’t know what endpoints your Truss serves. Instead, chains.depends() returns a TrussHandle. The handle doesn’t make calls itself. Its http_call_args() and ws_call_args() methods return a URL and headers, and you make the call with your own client, such as httpx, websockets, or openai. Requests and responses aren’t validated; their shapes are whatever your Truss serves. Import TrussHandle from its module:

from truss_chains.remote_chainlet.truss_chainlet import TrussHandle

Make HTTP calls

Depend on the TrussChainlet from a ChainletBase and call it with any HTTP client:

chain.py

import httpx
import truss_chains as chains
from truss_chains.remote_chainlet.truss_chainlet import TrussHandle


class STT(chains.TrussChainlet):
    truss_dir = "./stt_truss"


@chains.mark_entrypoint
class Entry(chains.ChainletBase):
    def __init__(self, stt: TrussHandle = chains.depends(STT)):
        self._stt = stt

    async def run_remote(self, audio_b64: str) -> dict[str, str]:
        url, headers = self._stt.http_call_args(prefer_internal=True)
        async with httpx.AsyncClient() as client:
            resp = await client.post(url, json={"audio_b64": audio_b64}, headers=headers)
            resp.raise_for_status()
            return resp.json()

http_call_args() accepts three optional keyword arguments:

prefer_internal: Call the model over Baseten’s internal network instead of its public URL. Internal calls skip a round trip through the public internet, so use this for chainlet-to-chainlet traffic.
sync_path: Route the call to a path your server exposes, instead of the predict endpoint. See Call custom server endpoints.
api_key: Authenticate with a different API key. By default, the handle uses the chain’s built-in key.

Make WebSocket calls

If the wrapped Truss serves WebSockets (runtime.transport.kind: websocket, see WebSockets), get connection arguments from ws_call_args():

url, headers = self._stt.ws_call_args()
async with websockets.connect(url, additional_headers=headers) as ws:
    await ws.send(payload)
    reply = await ws.recv()

The URL starts with wss:// and ends in /websocket. ws_call_args() accepts sync_path and api_key. There’s no prefer_internal because WebSocket clients don’t allow the header override that internal routing depends on.

Call custom server endpoints

A docker_server Truss serves its own API on its own paths. Pass sync_path to reach them. For example, wrap an OpenAI-compatible vLLM server and point the OpenAI client at it:

base_url, headers = self._llm.http_call_args(sync_path="v1")
client = openai.AsyncOpenAI(base_url=base_url, api_key="EMPTY", default_headers=headers)

Authentication runs through default_headers; vLLM ignores the api_key value.

Test locally

run_local() skips TrussChainlet dependencies because there’s nothing local to run: they wrap Truss directories that only exist as deployments. Construct the entrypoint with a stand-in object that has the same call-args methods, such as Entry(stt=FakeSTT()). If you leave it out, run_local() raises a ChainsUsageError. For the full workflow, see local development.

Next steps

Chainlet concepts: how chainlets compose into a chain.
Custom servers: package a docker_server Truss to wrap.
Truss integration with stubs: call a model that’s deployed outside the chain.

Overview

Get started

Model APIs

Inference

Development

Deployment

Engines

Frontier Gateway

Training

Organization

Observability

Troubleshooting

Add an existing Truss model to a Chain

Declare a TrussChainlet

Call a TrussChainlet

Make HTTP calls

Make WebSocket calls

Call custom server endpoints

Test locally

Next steps

​Declare a TrussChainlet

​Call a TrussChainlet

​Make HTTP calls

​Make WebSocket calls

​Call custom server endpoints

​Test locally

​Next steps

Declare a TrussChainlet

Call a TrussChainlet

Make HTTP calls

Make WebSocket calls

Call custom server endpoints

Test locally

Next steps