Amazon SageMaker HyperPod Inference now supports NVIDIA Dynamo through an add-on. It brings multi-node serving, separate prefill and decode stages and KV-aware routing, with vLLM, SGLang and TensorRT-LLM as engines. Teams supply their own Dynamo manifest or use an endpoint config with a dynamoSpec, and the post covers KEDA autoscaling. It describes deployments using EFA networking and benchmarks on p5en instances. Regions and pricing were not visible in the page I read.
