Zembed-1

Embeddings

ZeroEntropy’s Zembed-1 is a 4B-parameter text embedding model built for high-quality semantic search and retrieval. It produces native 2,560-dimensional embeddings and supports smaller dimensions through learned projection layers rather than standard Matryoshka truncation.

Setup

Install the Baseten CLI and sign in, then install the OpenAI SDK.

Install and sign in to Baseten

macOS or Linux
Windows

Terminal

brew tap basetenlabs/baseten
brew trust --formula basetenlabs/baseten/baseten
brew install baseten

Download and extract the binary, then move baseten.exe to a directory on your PATH:

Terminal

Invoke-WebRequest `
  https://github.com/basetenlabs/baseten-cli/releases/download/v1.0.0/baseten_1.0.0_windows_amd64.zip `
  -OutFile baseten.zip; Expand-Archive -Force baseten.zip .

For other platforms or a specific version, see the Baseten CLI install reference.

baseten auth login

Install the OpenAI SDK

uv pip install openai

To install without Homebrew, see the Baseten CLI install reference. This preset serves Zembed-1 Embedding 4B on a single H100 through Hugging Face Text Embeddings Inference, optimized for batch embedding throughput with a high concurrency ceiling.

Hardware

H100

Engine

Text Embeddings Inference cuda-1.9

Context

40K

Concurrency

Write the config

Create and move into the project directory:

mkdir zembed-1-embedding-4b-throughput && cd zembed-1-embedding-4b-throughput

Then create a file named config.yaml and paste the following:

config.yaml

# Registry copy of the config serving the ZeroEntropy lab listing zeroentropy/zembed-1
# (Hugging Face Text Embeddings Inference). Values below are the deployed ones. Two things
# differ in form, not effect: model_name follows the repo convention, and the checkpoint is
# pinned to cf13c81f (the Apache-2.0 relicense) instead of the deployed 10378878; every
# model file is byte-identical between the two revisions, only LICENSE/README changed.
model_name: model:zembed-1-embedding-4b preset:throughput
model_metadata:
  repo_id: zeroentropy/zembed-1-embedding
  example_model_input:
    inputs:
      - What is backpropagation?
    prompt_name: query
    dimensions: 1280
base_image:
  image: ghcr.io/huggingface/text-embeddings-inference:cuda-1.9
  python_executable_path: /usr/bin/true
environment_variables:
  HF_HUB_DISABLE_XET: "1"
build_commands:
  - mkdir -p /data /tmp/hf-cache && chmod -R 777 /data /tmp/hf-cache
weights:
  - source: hf://zeroentropy/zembed-1-embedding@cf13c81f3274394053d166740294f7eea4586f7a
    mount_location: /data/zembed-1-embedding
docker_server:
  start_command: >-
    text-embeddings-router-90
    --model-id /data/zembed-1-embedding
    --served-model-name zembed-1
    --hostname 0.0.0.0
    --port 3000
    --dtype float16
    --max-concurrent-requests 4096
    --max-client-batch-size 128
    --max-batch-tokens 8192
    --payload-limit 8388608
    --huggingface-hub-cache /tmp/hf-cache
    --default-prompt-name document
    --json-output
  server_port: 3000
  predict_endpoint: /embed
  readiness_endpoint: /health
  liveness_endpoint: /health
resources:
  accelerator: H100
  instance_type: H100
  use_gpu: true
runtime:
  predict_concurrency: 64
  streaming_read_timeout: 300

This config tells Baseten to serve zeroentropy/zembed-1-embedding on a single H100 with ghcr.io/huggingface/text-embeddings-inference:cuda-1.9 running the text-embeddings-router in FP16. The weights: block pins the checkpoint to the Apache-2.0 relicense revision, mirrors it to the Baseten Delivery Network at deploy time, and mounts it at /data/zembed-1-embedding, so the router loads from local disk and never calls Hugging Face at runtime. The router accepts batches of up to 8192 tokens and returns 2,560-dimensional embeddings.

Flags

The start_command passes these flags to the engine. Each one controls a runtime or serving behavior:

Flag	Value	What it does
`--model-id`	`/data/zembed-1-embedding`	Model to serve, either a Hugging Face model ID or a local directory holding the model files.
`--hostname`	`0.0.0.0`	IP address the server listens on. The `0.0.0.0` default accepts requests from outside the container.
`--dtype`	`float16`	Weight precision loaded at runtime. float16: FP16 weights.
`--max-concurrent-requests`	`4096`	Maximum concurrent requests the deployment accepts. Past it, the server rejects requests instead of queueing them.
`--max-client-batch-size`	`128`	Maximum number of inputs a client can send in a single request.
`--max-batch-tokens`	`8192`	Maximum total tokens in a batch, the main control on how much of the GPU one batch can use.
`--payload-limit`	`8388608`	Request payload size limit in bytes. The TEI default is 2 MB.
`--huggingface-hub-cache`	`/tmp/hf-cache`	Location of the Hugging Face Hub cache, which you can point at a mounted disk.
`--default-prompt-name`	`document`	Prompt from the model’s sentence-transformers `prompts` configuration that the server prepends by default. No prompt is applied when it is unset.
`--json-output`	(no value)	Emit server logs as JSON for telemetry pipelines.

Deploy

Push the config to Baseten with the Baseten CLI, or with the Truss CLI if you prefer it:

baseten model push

uvx truss push

You should see output similar to:

Pushing model "zembed-1-embedding-4b-throughput"...
Uploading model...
Uploaded model in 0s
✨ Model zembed-1-embedding-4b-throughput was successfully pushed ✨

  Model:       zembed-1-embedding-4b-throughput (abc1d2ef)
  Deployment:  xyz123
  Environment: production

🪵 View logs:
   deployment:   baseten model deployment logs --model-id abc1d2ef --deployment-id xyz123
   environment:  baseten model environment logs --model-id abc1d2ef --environment production  (once deployed)
   app:          https://app.baseten.co/models/abc1d2ef/logs/xyz123

🚀 Invoke your model:
   URL:  https://model-abc1d2ef.api.baseten.co/deployment/xyz123/predict
   CLI:  baseten model predict --model-id abc1d2ef

baseten model push prints your model ID (abc1d2ef in the example). The examples below use it wherever you see {model_id}, and read your API key from the BASETEN_API_KEY environment variable.

Call the model

Your deployment serves an embeddings API at /embed. Now call your deployment to generate embeddings:

Python
cURL

main.py

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["BASETEN_API_KEY"],
    base_url="https://model-{model_id}.api.baseten.co/environments/production/sync/v1",
)

response = client.embeddings.create(
    model="zembed-1",
    input=[
        "Baseten is a fast inference provider.",
        "Embeddings power semantic search and RAG.",
    ],
)

for item in response.data:
    print(len(item.embedding), item.embedding[:4])

curl -s https://model-{model_id}.api.baseten.co/environments/production/sync/v1/embeddings \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -d '{
    "model": "zembed-1",
    "input": [
      "Baseten is a fast inference provider.",
      "Embeddings power semantic search and RAG."
    ]
  }'

Zembed-1 is instruction-aware. Pass prompt_name: query for search queries and prompt_name: document for indexed text. The router falls back to the document prompt when the field is absent, so set prompt_name: query on the query side. For smaller vectors, pass the dimensions parameter instead of truncating the output yourself. Zembed-1 uses learned projection layers, not Matryoshka truncation, so naive truncation degrades quality. For higher throughput, use the Baseten Performance Client, which batches and pipelines requests automatically.

Examples

Models

Engines

Custom Docker servers

Custom Python models

Chains

Setup

Hardware

Engine

Context

Concurrency

Write the config

Flags

Deploy

Call the model

Next steps

Call your model

Autoscaling

​Setup

Hardware

Engine

Context

Concurrency

​Write the config

​Flags

​Deploy

​Call the model

​Next steps

Call your model

Autoscaling

Setup

Write the config

Flags

Deploy

Call the model

Next steps