How to structure your Chainlets
aynscio
-tasks that run concurrently.
That way you get the most out of the parallelism that Chains offers. This
design pattern is extensively used in the
audio transcription example.
asyncio
is essential for performance, it can also be tricky.
Here are a few caveats to look out for:AsyncBatcher
useful.
If there is no async support, consider running blocking code in a
thread/process pool (as an attribute of a Chainlet).asyncio.ensure_future
) does not start
the task immediately. In particular, when starting several tasks in a loop,
ensure_future
must be alternated with operations that yield to the event
loop that, so the task can be started. If the loop is not async for
or
contains other await
statements, a “dummy” await can be added, for example
await asyncio.sleep(0)
. This allows the tasks to be started concurrently.