Tools · AWS ML Blog ·
Disaggregated prefill and decode for LLM inference on SageMaker HyperPod
The post explains how to implement disaggregated prefill and decode (DPD) for LLM inference on Amazon SageMaker HyperPod using vLLM and the HyperPod Inference Operator. It focuses on a deployment approach for improving inference workflows on AWS infrastructure.