Model endpoint migration guide

This guide covers the new predict endpoints in two parts:

• Showing the format of the new model predict endpoint.
• Showing the change that you must make to how you parse model output when you switch to the new endpoint.

The change to model output format only applies when you switch to the new endpoints. Model output is unchanged for the old endpoints.

​

Updates to endpoint paths

The new endpoint uses model_id as part of the subdomain, where formerly it was part of the path:

# Old endpoint (for production deployment)
https://app.baseten.co/models/{model_id}/predict
# New endpoint (for production deployment)
https://model-{model_id}.api.baseten.co/production/predict

Updated endpoints:

• The old /models/id/predict endpoint is now the production deployment endpoint.
• The old /model_versions/id/predict endpoint is now the published deployment endpoint.
• There’s a new endpoint just for the development deployment of a model, the development deployment endpoint.

​

Model output response format

With the new model endpoints, we’ve changed the output format of the model response. This change simplifies model responses and removes a step in parsing model output.

​

Old endpoint response format

For the old endpoint, formatted https://app.baseten.co/models/<model-id>/predict, the model output was wrapped in a JSON dictionary with the model ID and model version ID (which is now the deployment ID):

{
  "model_id":"MODEL_ID",
  "model_version_id":"VERSION_ID",
  "model_output": {
    // Output varies by model, this is just an example
    "prediction": true,
    "confidence": 0.7839
  }
}

These old endpoints will stay available and the response format for these old endpoints will not change. You only need to change the way you parse your model output when switching to the new endpoints.

​

New endpoint response format

For the new endpoint, formatted https://model-<model-id>.api.baseten.co/production/predict, the model output is no longer wrapped:

// Output varies by model, this is just an example
{
  "prediction": true,
  "confidence": 0.7839
}

So, when you change your code to use the new endpoints, also update any code for parsing model responses, as it is no longer wrapped in an additional dictionary:

# On old endpoints:
model_output = resp.json()["model_output"]
# On new endpoints:
model_output = resp.json()