> ## Documentation Index > Fetch the complete documentation index at: https://docs.baseten.co/llms.txt > Use this file to discover all available pages before exploring further. # Create a team BIS-LLM deployment ## OpenAPI ````yaml post /v1/teams/{team_id}/llm_models openapi: 3.1.0 info: description: REST API for management of Baseten resources title: Baseten management API version: 1.0.0 servers: - url: https://api.baseten.co security: - BearerAuth: [] paths: /v1/teams/{team_id}/llm_models: parameters: - $ref: '#/components/parameters/team_id' post: summary: Creates a new BIS-LLM deployment requestBody: content: application/json: schema: $ref: '#/components/schemas/CreateLLMModelRequestV1' required: true responses: '200': description: '' content: application/json: schema: $ref: '#/components/schemas/LLMModelHandleV1' x-codeSamples: - lang: bash source: |- curl --request POST \ --url https://api.baseten.co/v1/teams/{team_id}/llm_models \ --header "Authorization: Bearer $BASETEN_API_KEY" \ --data '{ "resources": null, "region": null, "llm_version": null, "model_metadata": null, "autoscaling_settings": { "autoscaling_window": 600, "concurrency_target": null, "max_replica": 5, "max_scale_down_rate": null, "min_replica": 1, "scale_down_delay": 300, "target_in_flight_tokens": null, "target_utilization_percentage": null }, "additional_autoscaling_config": { "metrics": [ { "name": "in_flight_tokens", "target": 40000 } ] }, "metadata": { "environment": "production", "git_sha": "abc123" }, "weights": [ { "mount_location": "/models/base", "source": "hf://meta-llama/Llama-3-8B" } ], "name": null }' - lang: python source: |- import requests import os API_KEY = os.environ.get("BASETEN_API_KEY", "") url = "https://api.baseten.co/v1/teams/{team_id}/llm_models" headers = {"Authorization": f"Bearer {API_KEY}"} response = requests.request( "POST", url, headers=headers, json={'resources': None, 'region': None, 'llm_version': None, 'model_metadata': None, 'autoscaling_settings': {'autoscaling_window': 600, 'concurrency_target': None, 'max_replica': 5, 'max_scale_down_rate': None, 'min_replica': 1, 'scale_down_delay': 300, 'target_in_flight_tokens': None, 'target_utilization_percentage': None}, 'additional_autoscaling_config': {'metrics': [{'name': 'in_flight_tokens', 'target': 40000}]}, 'metadata': {'environment': 'production', 'git_sha': 'abc123'}, 'weights': [{'mount_location': '/models/base', 'source': 'hf://meta-llama/Llama-3-8B'}], 'name': None} ) print(response.text) components: parameters: team_id: schema: type: string name: team_id in: path required: true schemas: CreateLLMModelRequestV1: description: A request to create a BIS-LLM model properties: resources: additionalProperties: true description: Resources allocated to the model title: Resources type: object region: anyOf: - type: string - type: 'null' default: null description: Region in which to deploy the model title: Region llm_version: anyOf: - type: string - type: 'null' default: null description: Version of the helm chart to use. title: Llm Version llm_config: additionalProperties: true description: Configuration specific to the LLM model title: Llm Config type: object environment_variables: additionalProperties: true description: Environment variables for the model title: Environment Variables type: object model_metadata: anyOf: - additionalProperties: true type: object - type: 'null' default: null description: Model metadata persisted into model_config title: Model Metadata autoscaling_settings: anyOf: - $ref: '#/components/schemas/UpdateAutoscalingSettingsV1' - type: 'null' default: null description: Autoscaling settings for the model examples: - autoscaling_window: 600 concurrency_target: null max_replica: 5 max_scale_down_rate: null min_replica: 1 scale_down_delay: 300 target_in_flight_tokens: null target_utilization_percentage: null additional_autoscaling_config: anyOf: - additionalProperties: true type: object - type: 'null' default: null description: Additional autoscaling configuration (e.g. target in-flight tokens) examples: - metrics: - name: in_flight_tokens target: 40000 title: Additional Autoscaling Config metadata: anyOf: - additionalProperties: true type: object - type: 'null' default: null description: User-defined metadata for the deployment examples: - environment: production git_sha: abc123 title: Metadata weights: anyOf: - items: additionalProperties: true type: object type: array - type: 'null' default: null description: Weight configurations for BDN model weight distribution examples: - - mount_location: /models/base source: hf://meta-llama/Llama-3-8B title: Weights name: description: Name of the model title: Name type: string required: - resources - name title: CreateLLMModelRequestV1 type: object LLMModelHandleV1: description: Handle for a BIS-LLM model deployment. properties: model_id: description: Unique identifier of the model title: Model Id type: string version_id: description: Unique identifier of the model version title: Version Id type: string hostname: description: Hostname used to invoke the model title: Hostname type: string instance_type_name: anyOf: - type: string - type: 'null' default: null description: Name of the instance type the model deployment is running on title: Instance Type Name required: - model_id - version_id - hostname title: LLMModelHandleV1 type: object UpdateAutoscalingSettingsV1: additionalProperties: false description: >- A request to update autoscaling settings for a deployment. All fields are optional, and we only update ones passed in. properties: min_replica: anyOf: - type: integer - type: 'null' default: null description: Minimum number of replicas examples: - 0 title: Min Replica max_replica: anyOf: - type: integer - type: 'null' default: null description: Maximum number of replicas examples: - 7 title: Max Replica autoscaling_window: anyOf: - type: integer - type: 'null' default: null description: Timeframe of traffic considered for autoscaling decisions examples: - 600 title: Autoscaling Window scale_down_delay: anyOf: - type: integer - type: 'null' default: null description: Waiting period before scaling down any active replica examples: - 120 title: Scale Down Delay concurrency_target: anyOf: - type: integer - type: 'null' default: null description: Number of requests per replica before scaling up examples: - 2 title: Concurrency Target target_utilization_percentage: anyOf: - type: integer - type: 'null' default: null description: Target utilization percentage for scaling up/down. examples: - 70 title: Target Utilization Percentage target_in_flight_tokens: anyOf: - type: integer - type: 'null' default: null description: >- Target number of in-flight tokens for autoscaling decisions. Early access only. examples: - 40000 title: Target In Flight Tokens max_scale_down_rate: anyOf: - maximum: 50 minimum: 1 type: integer - type: 'null' default: null description: >- Maximum percentage of replicas that can be removed per autoscaling window (1–50). E.g. 20 means at most 20% of replicas are removed per window. examples: - 20 title: Max Scale Down Rate title: UpdateAutoscalingSettingsV1 type: object securitySchemes: BearerAuth: type: http scheme: bearer description: >- Send `Authorization: Bearer `. The legacy `Authorization: Api-Key ` scheme is also accepted. ````