Dedicated model inference adds one-command deploy, A/B tests and autoscaling

Dedicated model inference (DMI) serves a model on reserved GPUs, giving you higher throughput, lower latency, and predictable performance with no hard rate limits. It uses the same inference APIs as serverless, so you can prototype on serverless and deploy on dedicated hardware without changing your application code.

What's new:

• Deploy in one command: tg beta endpoints deploy creates an endpoint, attaches a deployment, and routes all traffic to it.
• New resource model: Compose models, configs, endpoints, deployments, and traffic splits to control how your model is served. See Concepts.
• A/B tests and shadow experiments: Split live traffic across variants, or mirror traffic to a new deployment without serving its responses.
• Autoscaling and serving controls: Scale on request or GPU metrics, scale to zero on idle, and tune serving behavior per deployment. See Configure autoscaling.

If you're already using dedicated endpoints, you can migrate to the v2 API by following the migration guide.