---
title: "Amazon ElastiCache now supports distributed KV caching for self-hosted LLM inference — Cloud Changelog"
description: "Amazon ElastiCache for Valkey now offers fully managed distributed KV caching, a new capability that reduces self-hosted large language model (LLM) infere…"
url: "https://cloudchangelog.com/a/2026-11-announcing-distributed-kv-caching-for-self-hosted-llms-with-amazon-elasticache"
updated: "2026-11-19"
---

[Back to feed](https://cloudchangelog.com/feed/2026-W47)

November 18, 2026 · AWS

# Amazon ElastiCache now supports distributed KV caching for self-hosted LLM inference

- [Amazon ElastiCache](https://cloudchangelog.com/products/aws/elasticache)
- [Databases](https://cloudchangelog.com/search?q=databases)

[Amazon ElastiCache for Valkey](https://aws.amazon.com/elasticache/) now offers fully managed distributed KV caching, a new capability that reduces self-hosted large language model (LLM) inference costs by up to 40% and time to first token (TTFT) by up to 80%. Customers who self-host LLMs pay for redundant GPU work because small, isolated caches on each server can't hold or share the growing context AI agents and chat applications resend. Distributed KV caching extends GPU-local caches with a shared tier so any server can reuse KV state for days across sessions, routing shifts, and server failures. GPUs can fetch cached KV state in milliseconds to serve more requests and respond faster.

Distributed KV caching runs on cache.i8ge.48xlarge nodes in the same Availability Zone as the GPU fleet and connects to GPUs through Remote Direct Memory Access (RDMA) over [Elastic Fabric Adapter (EFA).](https://aws.amazon.com/hpc/efa/) Distributed KV caching works with open-weight, fine-tuned, and custom models, whether they use full, sparse, hybrid, or other attention designs. You can add distributed KV caching to the inference engines you already use, including vLLM, SGLang, and TensorRT-LLM, through the LMCache connector with a configuration change. You can also use the [open-source KVTrace tool](https://github.com/aws-samples/amazon-elasticache-samples/tree/main/kvtrace) to estimate the cost and latency reductions on your own application traffic before deploying anything, at no charge.

Distributed KV caching is available today on ElastiCache for Valkey in 13 AWS Regions across the US, Europe, and Asia Pacific. To get started, create a cluster with cache.i8ge.48xlarge nodes using the [AWS CLI](https://aws.amazon.com/cli/) in the same AZ as your GPU fleet and adding the endpoint to your inference engine. To learn more, read the [AWS blog](https://aws.amazon.com/blogs/database/announcing-distributed-kv-caching-for-self-hosted-llms-with-amazon-elasticache/) for a technical deep dive and performance results, or visit the [Amazon ElastiCache documentation](https://docs.aws.amazon.com/AmazonElastiCache/latest/dg/distributed-kv-caching.html). For pricing, see the [Amazon ElastiCache pricing page](https://aws.amazon.com/elasticache/pricing/).

Free. Unsubscribe any time.

[Read on aws.amazon.com](https://aws.amazon.com/about-aws/whats-new/2026/11/announcing-distributed-kv-caching-for-self-hosted-llms-with-amazon-elasticache/)

## Related updates

- Nov 16, 2026

  [Amazon SageMaker AI now supports KV cache-aware routing for real-time inference endpoints](https://cloudchangelog.com/a/2026-11-sagemaker-kv-cache-routing)
- Oct 13, 2025

  [Announcing vector search for Amazon ElastiCache](https://cloudchangelog.com/a/2025-10-amazon-elasticache-vector-search)
- Nov 17, 2026

  [Amazon DynamoDB caching is now available with Amazon ElastiCache Serverless for Valkey](https://cloudchangelog.com/a/2026-11-dynamodb-caching-elasticache)
- Nov 10, 2026

  [Amazon ElastiCache Serverless now supports vector, full-text and hybrid searches plus aggregations](https://cloudchangelog.com/a/2026-11-amazon-elasticache-serverless-search-aggregations)
- Nov 10, 2026

  [Public Preview: Distributed accelerator for cloud storage on AKS](https://cloudchangelog.com/azure/578454) · Azure

[All Amazon ElastiCache changes](https://cloudchangelog.com/products/aws/elasticache)

## Contact

- [YouTube](https://www.youtube.com/@cloudchangelog)
