Skip to content
Back to feed

July 23, 2026 · Together AI

Dedicated endpoint metrics add KV-cache utilization and cache hit rate gauges

Dedicated endpoint Prometheus metrics now expose worker_engine_kv_cache_utilization and worker_engine_cache_hit_rate gauges. Use them to track KV-cache pressure and prefix-reuse efficiency on deployment workers.

See Monitor endpoints and deployments.

Latest Together AI platform changes

All Together AI platform changes