> ## Documentation Index > Fetch the complete documentation index at: https://docs.baseten.co/llms.txt > Use this file to discover all available pages before exploring further. # Deployments > Troubleshoot common problems during model deployment ## Issue: `baseten model push` can't find `config.yaml` ```sh theme={"system"} config.yaml not found in ".": is this a model directory? Pass --dir to point to one ``` ### Fix: set correct target directory The directory `baseten model push` is looking at is not a model directory. Make sure you're giving `baseten model push` access to the correct directory by: * Running `baseten model push` from the directory containing the model. You should see the file `config.yaml` when you run `ls` in your working directory. * Or passing the target directory with `--dir`, such as `baseten model push --dir /path/to/my-model`. With the Truss CLI, the equivalent error is `[Errno 2] No such file or directory: '/path/to/your-truss/config.yaml'`; pass the target directory as an argument instead, such as `truss push /path/to/my-truss`. ## Issue: unexpected failure during model build During the model build step, there can be unexpected failures from temporary circumstances. An example is a network error while downloading model weights from Hugging Face or installing a Python package from PyPi. ### Restart the deployment in the Baseten dashboard First, check your model logs to determine the exact cause of the error. If the failure occurred during a model download, package installation, or similar step, restart the deployment from the Baseten dashboard. ## Troubleshoot a stuck or unhealthy replica One replica of a multi-replica deployment can fail while the others keep serving, for example by hanging, becoming unresponsive, or consuming excessive memory. Restarting the whole deployment is heavier than the problem warrants. ### Fix: terminate the affected replica Terminate just the bad replica. The deployment's autoscaler brings up a fresh replica to maintain your target replica count, so traffic keeps flowing on the healthy replicas while the replacement starts. First find the ID of the bad replica. Every log line is tagged with the replica that emitted it; platform-level lines carry an empty replica ID: **To find the replica ID**: 1. Sign in to your workspace at [app.baseten.co](https://app.baseten.co) and choose **Dedicated Inference** in the sidebar. 2. Select your model. 3. Select the deployment under **Deployments**. 4. Choose **Logs**. 5. Open the **Replica** filter. The dropdown lists each replica ID. Then terminate the replica from the Baseten CLI or the Management API. **To find the replica ID**: ```bash Command theme={"system"} baseten model deployment logs --model-id --deployment-id --since 1h --jq '.replica' ``` ```txt Output theme={"system"} "" "nvsmp" ``` The non-empty IDs are your replicas. **To terminate the replica**: ```bash Command theme={"system"} baseten model deployment replica terminate --model-id --deployment-id --replica-id ``` ```txt Output theme={"system"} Terminated replica of deployment ``` The CLI prompts for confirmation; pass `--yes` to skip it when scripting. **To find the replica ID**: Each log entry carries a `replica` field: ```bash Request theme={"system"} curl "https://api.baseten.co/v1/models/{model_id}/deployments/{deployment_id}/logs" \ -H "Authorization: Bearer $BASETEN_API_KEY" ``` ```json Response theme={"system"} { "logs": [ { "timestamp": "1783445624380306972", "message": "Completed model.load() execution in 6 ms", "replica": "nvsmp", "request_id": null, "level": "INFO" } ] } ``` **To terminate the replica**: ```bash Request theme={"system"} curl -X DELETE "https://api.baseten.co/v1/models/{model_id}/deployments/{deployment_id}/replicas/{replica_id}" \ -H "Authorization: Bearer $BASETEN_API_KEY" ``` ```json Response theme={"system"} {"success": true} ``` After termination, the autoscaler starts a replacement replica to maintain the deployment's target count, so capacity dips by one replica until the replacement finishes its cold start. If the whole deployment is unhealthy across replicas, [deactivate or replace it](/deployment/manage/lifecycle) instead. For more information, see [`baseten model deployment replica`](/reference/cli/baseten/model-deployment-replica) and the [terminate replica endpoint](/reference/management-api/deployments/terminates-deployment-replica). ## Issue: requests fail with "Model version is deactivated" Requests to a deactivated deployment fail with a `400` error: ```txt theme={"system"} Model version {deployment_id} is deactivated. It needs to be activated before running predictions ``` ### Fix: activate the deployment Activate the deployment from the model's page in your workspace, or run: ```sh theme={"system"} baseten model deployment activate --model-id --deployment-id ``` Activation redeploys the model, so the deployment passes through `DEPLOYING` before serving requests again. If you deactivated the deployment to stop idle spend, consider [scaling to zero](/deployment/manage/scaling#scale-to-zero) instead: the endpoint stays live and replicas start on demand. ## Issue: "You've reached the maximum number of deployed models" Pushing a model, pushing a new deployment, or activating an inactive deployment fails with a `400` error: ```txt theme={"system"} You've reached the maximum number of deployed models (2000) for your tier. Either remove older model deployments or email us at support@baseten.co. We are happy to give you a higher limit. ``` Every workspace has a limit on how many deployments it can keep deployed at once, so a workspace can't accumulate deployments without bound. The limit applies across the whole workspace, not per model, and the number in the message is how many of your deployments currently count against it. Two things about the count catch people out: * **It counts deployments, not models**: A model with four deployments uses four slots. Chainlet deployments count individually, so a chain with five chainlets uses five slots. * **Scaled-to-zero deployments count**: A deployment scaled to zero runs no replicas and incurs no running-replica charges, but its endpoint stays live and ready to cold-start, so it holds its slot. Only deactivated deployments release theirs. **Counts against the limit** * `ACTIVE` * `SCALED_TO_ZERO` * `WAKING_UP` * `UPDATING` * `UNHEALTHY` * `BUILDING` * `DEPLOYING` **Doesn't count** * `INACTIVE` * `DEACTIVATING` * `BUILD_FAILED` * `DEPLOY_FAILED` * `BUILD_STOPPED` Models you call through [Model APIs](/inference/model-apis/overview) don't count, since you don't deploy them into your workspace. ### Fix: free deployment slots First, find which deployments hold a slot. Deployments are listed per model, so this means checking each of your models. **To check a model's deployments**: 1. Sign in to your workspace at [app.baseten.co](https://app.baseten.co) and choose **Dedicated Inference** in the sidebar. 2. Select your model. 3. Review the status of each deployment under **Deployments**. The dashboard shows one model at a time. To audit a whole workspace at once, use the Baseten CLI or the Management API. **To list every deployment that holds a slot**: ```bash Command theme={"system"} while IFS= read -r model_id; do model_id=${model_id#\"} model_id=${model_id%\"} baseten model deployment list --model-id "$model_id" \ --jq '.deployments[] | select(.status | IN("ACTIVE","SCALED_TO_ZERO","WAKING_UP","UPDATING","UNHEALTHY","BUILDING","DEPLOYING")) | { model_id, deployment_id: .id, status, active_replica_count }' done < <(baseten model list --jq '.models[].id') ``` ```json Output theme={"system"} { "active_replica_count": 0, "deployment_id": "def456", "model_id": "abc123", "status": "SCALED_TO_ZERO" } { "active_replica_count": 1, "deployment_id": "ghi789", "model_id": "abc123", "status": "ACTIVE" } ``` Each object is a deployment holding a slot. The Baseten CLI evaluates the filters through its built-in `--jq` option. **To list every deployment that holds a slot**: These cURL commands require the external `jq` command. 1. List your models: ```bash Request theme={"system"} curl "https://api.baseten.co/v1/models" \ -H "Authorization: Bearer $BASETEN_API_KEY" \ | jq -r '.models[].id' ``` ```txt Response theme={"system"} abc123 jkl012 ``` 2. Then list each model's deployments, keeping the ones that hold a slot: ```bash Request theme={"system"} curl "https://api.baseten.co/v1/models/{model_id}/deployments" \ -H "Authorization: Bearer $BASETEN_API_KEY" \ | jq -r '.deployments[] | select(.status | IN("ACTIVE","SCALED_TO_ZERO","WAKING_UP","UPDATING","UNHEALTHY","BUILDING","DEPLOYING")) | "\(.id) \(.status) \(.active_replica_count)"' ``` ```txt Response theme={"system"} def456 SCALED_TO_ZERO 0 ghi789 ACTIVE 1 ``` Then work through the deployments that shouldn't hold a slot: 1. **Deactivate deployments you don't expect to serve traffic soon**: [Deactivating](/deployment/manage/lifecycle#deactivate-a-deployment) keeps the deployment's configuration and releases its slot, and you can [activate](/deployment/manage/lifecycle#activate-a-deployment) it again later. This is the fastest way to unblock a push. 2. **Delete deployments you're finished with**: [Deleting](/deployment/manage/lifecycle#delete-a-deployment) is irreversible, so reserve it for finished experiments and stale versions. 3. **Stop accumulating superseded deployments**: After a promotion, the target environment's `promotion_cleanup_strategy` decides what happens to the deployment it replaced. The default, `SCALE_TO_ZERO`, leaves that deployment holding its slot. Set the strategy to `DEACTIVATE` so each promotion releases the slot it supersedes. See [deployment cleanup](/deployment/rolling-deployments#deployment-cleanup). 4. **Request a higher limit**: If every counted deployment is one you need, email [support@baseten.co](mailto:support@baseten.co). Deactivating and deleting both free a slot within a few seconds, so you can retry the push right away. For more information, see [`baseten model deployment list`](/reference/cli/baseten/model-deployment#list) and the [list deployments endpoint](/reference/management-api/deployments/gets-all-deployments-of-a-model). *** ## Autoscaling issues Before troubleshooting, review [Autoscaling](/deployment/autoscaling/overview) for parameter details, [Traffic patterns](/deployment/autoscaling/traffic-patterns) for pattern-specific recommendations, and [Request lifecycle](/deployment/autoscaling/request-lifecycle) for HTTP status codes and timeout behavior. ### Replica count doesn't match my settings **Symptoms**: You see fewer replicas than the `min_replica` you configured, or the count won't climb toward `max_replica` under load. **Check the desired line first**: Open the [Replicas graph](/observability/metrics#replicas) and compare **desired** against your active + starting replicas. Desired shows what your settings ask for, so it tells you where the problem is: * **Desired matches what you configured**: Your settings are correct and Baseten is provisioning the difference. See **Slow scale-up** for what affects provisioning time. * **Desired sits at `max_replica`**: Your ceiling is the limit. Raise `max_replica`, or raise `concurrency_target` if each replica can serve more traffic. * **Desired is lower than you expect**: Your scaling triggers aren't seeing the load you assume. See **Concurrency target misconfigured**. ### Latency spikes during scaling events **Symptoms**: TTFT (time to first token) or p95/p99 latency degrades when replicas are added or removed. **Causes**: * Replicas terminated while handling in-flight requests * Cold start delays while new replicas initialize **Solutions** (in order of priority): 1. Increase [**scale-down delay**](/deployment/autoscaling/overview#param-scale-down-delay) to reduce how often the autoscaler removes replicas. For example, increase it from 300 seconds to 900 seconds. 2. Increase [**min replicas**](/deployment/autoscaling/overview#param-min-replica) to reduce cold start frequency. 3. Lower [**target utilization**](/deployment/autoscaling/overview#param-target-utilization-percentage) to provide more headroom during scaling. ### Replicas oscillating (thrash) **Symptoms**: Replica count bounces repeatedly (for example, 8↔9) even with relatively stable traffic. **Causes**: Autoscaler reacting to short-term traffic noise or internal model fluctuations. **Solutions** (in order of priority): 1. Increase **scale-down delay**: this is the primary lever for oscillation. 2. Increase [**autoscaling window**](/deployment/autoscaling/overview#param-autoscaling-window) to smooth out noise. 3. Lower [**max scale-down rate**](/deployment/autoscaling/overview#param-max-scale-down-rate) so each scale-down step removes fewer replicas. 4. Only then consider lowering **target utilization** for more headroom. Don't use target utilization as the primary fix for thrash. Scale-down delay is more effective and doesn't waste capacity. ### Slow scale-up / "Scaling up replicas" persists **Symptoms**: New replicas take many minutes (or longer) to become ready. The deployment shows "Scaling up replicas" for an extended period. **Confirm the cause**: Compare **desired** against the total of active + starting on the [Replicas graph](/observability/metrics#replicas). Desired above that total means your autoscaling settings are correct and Baseten is still provisioning replicas. Replicas that reach **starting** but take a long time to become active point to model initialization instead. **Causes**: * GPU capacity not available in your region * Slow model initialization (large weights, slow downloads) **Solutions**: 1. **Pre-warm** by bumping min replicas through the API before expected load spikes. 2. Contact support about capacity pool availability. 3. Check if optimized images are being used (look for "streaming-enabled image" in logs). ### Model scales to zero before testing **Symptoms**: A newly deployed model scales down to zero before you can send your first test request. **Solution**: Set `min_replica = 1` during testing. After testing, you can set it back to 0 if you want scale-to-zero behavior. ### Async queue growing without bound **Symptoms**: The async queue size keeps increasing and requests are not being processed fast enough. **Cause**: Requests are arriving faster than the deployment can process them. **Solutions**: 1. Increase [**max replicas**](/deployment/autoscaling/overview#param-max-replica) to add more processing capacity. 2. Increase [**concurrency target**](/deployment/autoscaling/overview#param-concurrency-target) if your model can handle more concurrent requests. 3. Lower **target utilization** to trigger scaling earlier. ### Bill higher than expected **Symptoms**: GPU costs are higher than anticipated, especially during low-traffic periods. **Solutions**: 1. Raise **concurrency target** to squeeze more throughput from each replica. 2. Monitor **p95 latency** as you raise concurrency. If latency stays stable, keep raising; if it rises sharply, you've gone too far. 3. Enable **scale-to-zero** (min replicas = 0) for intermittent workloads. 4. Review your traffic patterns and adjust settings accordingly. See [Traffic patterns](/deployment/autoscaling/traffic-patterns). ### Cold starts taking too long **Symptoms**: Replica startup exceeds your workload's latency target. Synchronous requests can wait while a replica starts, and logs show extended time in model loading or container initialization. **Causes**: * Model weight loading * Slow network downloads from model registries * Heavy initialization code in `load()` method **Solutions**: 1. Look for "streaming-enabled image" in logs. This confirms image streaming is active. 2. Keep `min_replica ≥ 1` to avoid scale-from-zero cold starts. 3. Pre-warm before expected traffic spikes with an [autoscaling schedule](/deployment/autoscaling/schedules). See [Cold starts](/deployment/autoscaling/cold-starts) for detailed optimization strategies. ### Development deployment won't scale **Symptoms**: Development deployment won't scale beyond 1 replica under load. Can't change autoscaling settings. **Cause**: Development deployments have fixed autoscaling settings that cannot be modified. They run 0 to 1 replicas, so they scale to zero when idle, but max replicas is locked at 1. **Solution**: [Promote the development deployment to an environment](/deployment/manage/lifecycle#promote-to-an-environment) to configure the full set of autoscaling settings. Baseten optimizes development deployments for iteration with live reload, not production traffic. See [Development deployments](/deployment/autoscaling/overview#development-deployments) for the fixed settings. ### Not sure which traffic pattern I have **Symptoms**: Unsure how to configure autoscaling because traffic behavior is unclear. **Solution**: 1. Sign in to your workspace at [app.baseten.co](https://app.baseten.co) and choose **Dedicated Inference** in the sidebar, then select your model. 2. Choose the **Metrics** tab. 3. Look at **Inference volume** and **Replicas** over the past week. 4. Identify your pattern: | You see... | Pattern | Key settings to adjust | | ------------------------------------------- | --------------- | ------------------------------------------- | | Frequent small spikes returning to baseline | Noisy/jittery | Longer autoscaling window | | Sharp jumps that stay high | Bursty | Short window, long delay, lower utilization | | Long flat periods with occasional bursts | Batch/scheduled | Scale-to-zero, pre-warming | | Gradual rises and falls | Smooth/steady | Higher utilization is safe | See [Traffic patterns](/deployment/autoscaling/traffic-patterns) for detailed recommendations. ### Concurrency target misconfigured **Symptoms**: Either unexpectedly high costs OR high latency despite having replicas available. **Diagnosis**: * **Too low** (common): Running many more replicas than needed. Default of 1 is conservative but expensive. * **Too high**: Requests queue at replicas, causing latency even when replica count looks healthy. **Solutions**: 1. Benchmark your model to find actual throughput capacity. 2. Use starting points by model type: | Model type | Starting concurrency | | ----------------------- | -------------------- | | Standard Truss | 1 | | vLLM / LLM inference | 32–128 | | Text embeddings (TEI) | 32 | | Image generation (SDXL) | 1 | 3. Gradually increase while monitoring p95 latency. Stop when latency rises sharply. See [Concurrency target](/deployment/autoscaling/overview#param-concurrency-target) for full guidance. For detailed autoscaling configuration, see [Autoscaling](/deployment/autoscaling/overview). For pattern-specific recommendations, see [Traffic patterns](/deployment/autoscaling/traffic-patterns).