Zembed-1

Embeddings

ZeroEntropy’s Zembed-1 is a 4B-parameter text embedding model built for high-quality semantic search and retrieval. It produces native 2,560-dimensional embeddings and supports smaller dimensions through learned projection layers rather than standard Matryoshka truncation.

Setup

Sign in to Baseten

uvx truss login --browser

Install the OpenAI SDK

uv pip install openai

This preset serves Zembed-1 Embedding 4B on a single H100 through vLLM’s OpenAI-compatible server, optimized for batch embedding throughput with a concurrency of 64.

Hardware

H100

Engine

vLLM 0.25.0

Context

32K

Concurrency

Write the config

Create and move into the project directory:

mkdir zembed-1-embedding-4b-throughput && cd zembed-1-embedding-4b-throughput

Then create a file named config.yaml and paste the following:

config.yaml

# yaml-language-server: $schema=https://raw.githubusercontent.com/basetenlabs/truss/main/truss/config.schema.json
model_name: model:zembed-1-embedding-4b preset:throughput

model_metadata:
  example_model_input:
    model: zeroentropy/zembed-1-embedding
    # zembed-1 is instruction-aware (query vs document). vLLM does NOT auto-apply
    # sentence-transformers prompts on /v1/embeddings, so the client must prepend
    # the correct instruction. Pull the exact prefix strings from the repo's
    # config_sentence_transformers.json / model card before using in production.
    input:
      - Baseten is a fast inference provider
      - Embeddings let you do semantic search.
    encoding_format: float
    truncate_prompt_tokens: 32768
    truncation_side: right
  repo_id: zeroentropy/zembed-1-embedding
  tags:
    - openai-compatible
    - embedding
    - vllm

base_image:
  image: vllm/vllm-openai:v0.25.0

docker_server:
  start_command: >-
    vllm serve /models/zembed-1
    --host 0.0.0.0
    --port 8000
    --served-model-name zeroentropy/zembed-1-embedding
    --runner pooling
    --max-model-len 32768
    --max-num-batched-tokens 131072
    --max-num-seqs 64
    --enable-chunked-prefill
    --gpu-memory-utilization 0.90
    --trust-remote-code

  readiness_endpoint: /health
  liveness_endpoint: /health
  predict_endpoint: /v1/embeddings
  server_port: 8000

weights:
  # Apache-2.0 variant. Pin @<commit-sha> instead of @main for reproducible builds.
  - source: "hf://zeroentropy/zembed-1-embedding@main"
    mount_location: "/models/zembed-1"

resources:
  accelerator: H100
  use_gpu: true

runtime:
  predict_concurrency: 64
  health_checks:
    restart_threshold_seconds: 600
    stop_traffic_threshold_seconds: 240

environment_variables:
  VLLM_LOGGING_LEVEL: INFO

secrets: {}
system_packages: []
requirements: []

This config tells Baseten to serve zeroentropy/zembed-1-embedding on a single H100 with the stock vllm/vllm-openai:v0.25.0 image running the pooling runner. The weights: block pins the checkpoint to a commit SHA, mirrors it to the Baseten Delivery Network at deploy time, and pre-mounts it at /models/zembed-1, so vLLM loads from local disk and never calls Hugging Face at runtime. The deployment exposes an OpenAI-compatible /v1/embeddings endpoint with 2,560-dimensional output.

Flags

The start_command passes these flags to the engine. Each one controls a runtime or serving behavior:

Flag	Value	What it does
`--runner`	`pooling`	Which runner type vLLM uses for the model. pooling: Serve the model as a pooling model (embeddings or reranking) instead of text generation.
`--max-model-len`	`32768`	Maximum context length (tokens) the server accepts per request.
`--max-num-batched-tokens`	`131072`	Maximum total tokens processed per scheduler step.
`--max-num-seqs`	`64`	Maximum number of concurrent sequences in the batch.
`--enable-chunked-prefill`	(no value)	Process long prompts in chunks so decode requests keep running.
`--gpu-memory-utilization`	`0.90`	Fraction of GPU memory vLLM may use for weights and KV cache.
`--trust-remote-code`	(no value)	Execute model-specific Python from the checkpoint (required for many Qwen, Phi, and custom architectures).

Deploy

Push the config to Baseten:

uvx truss push

You should see output similar to:

✨ Model zembed-1-embedding-4b-throughput was successfully pushed ✨

   Model ID:      abc1d2ef
   Deployment ID: xyz123
   Endpoint:      model-abc1d2ef.api.baseten.co
   Logs:          https://app.baseten.co/models/abc1d2ef/logs/xyz123

truss push prints your model ID (abc1d2ef in the example). The examples below use it wherever you see {model_id}, and read your API key from the BASETEN_API_KEY environment variable.

Call the model

Your deployment serves an OpenAI-compatible embeddings API at /v1/embeddings. Now call your deployment to generate embeddings:

Python
cURL

main.py

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["BASETEN_API_KEY"],
    base_url="https://model-{model_id}.api.baseten.co/environments/production/sync/v1",
)

response = client.embeddings.create(
    model="zeroentropy/zembed-1-embedding",
    input=[
        "Baseten is a fast inference provider",
        "Embeddings let you do semantic search.",
    ],
)

for item in response.data:
    print(len(item.embedding), item.embedding[:4])

curl -s https://model-{model_id}.api.baseten.co/environments/production/sync/v1/embeddings \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -d '{
    "model": "zeroentropy/zembed-1-embedding",
    "input": [
      "Baseten is a fast inference provider",
      "Embeddings let you do semantic search."
    ]
  }'

Zembed-1 is instruction-aware. The /v1/embeddings endpoint does not auto-apply sentence-transformers prompts, so prepend the correct instruction prefix yourself. Pull the exact strings from the model’s config_sentence_transformers.json or the Hugging Face model card. For smaller vectors, pass the OpenAI dimensions parameter. Zembed-1 uses learned projection layers, not Matryoshka truncation, so naive truncation can degrade quality. Always use the dimensions parameter rather than truncating the output yourself. For higher throughput, use the Baseten Performance Client, which batches and pipelines requests automatically.

Examples

Models

Engines

Custom Docker servers

Custom Python models

Chains

Setup

Hardware

Engine

Context

Concurrency

Write the config

Flags

Deploy

Call the model

Next steps

Call your model

Autoscaling

​Setup

Hardware

Engine

Context

Concurrency

​Write the config

​Flags

​Deploy

​Call the model

​Next steps

Call your model

Autoscaling

Setup

Write the config

Flags

Deploy

Call the model

Next steps