[2412.03594] BatchLLM: Optimizing Large Batched LLM Inference with Global Prefix Sharing and Throughput-oriented Token Batching
![[2412.03594] BatchLLM: Optimizing Large Batched LLM Inference with Global Prefix Sharing and Throughput-oriented Token Batching](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
Summary
Abstract page for arXiv paper 2412.03594: BatchLLM: Optimizing Large Batched LLM Inference with Global Prefix Sharing and Throughput-oriented Token Batching
Description
Abstract page for arXiv paper 2412.03594: BatchLLM: Optimizing Large Batched LLM Inference with Global Prefix Sharing and Throughput-oriented Token Batching
Original reporting
AFBytes is a read-only aggregator. Use the original source for full context and complete reporting.
Open original source