PPoPP 2024
Sat 2 - Wed 6 March 2024 Edinburgh, United Kingdom
Sun 3 Mar 2024 18:00 - 20:00 at Strathblane Hall - Reception and Poster Session

Recent breakthroughs in Large-scale language models (LLMs) have demonstrated impressive performance on various tasks. The immense sizes of LLMs have led to very high resource demand and cost for running the models. Though the models are largely served using uniform high-caliber GPUs nowadays, utilizing a heterogeneous cluster with a mix of available high- and low-capacity GPUs can potentially substantially reduce the serving cost. There is a lack of designs to support efficient LLM serving using a heterogeneous cluster, while the current solutions focus on model partition and uniform compression among homogeneous devices. This paper proposes LLM-PQ, a system that advocates adaptive model quantization and phase-aware partition to improve LLM serving efficiency on heterogeneous GPU clusters. We carefully decide mixed-precision model quantization together with phase-aware model partition and micro-batch sizing in distributed LLM serving with an efficient algorithm, to greatly enhance inference throughput while fulfilling user-specified model quality targets. Extensive experiments on production inference workloads in 11 different clusters demonstrate that LLM-PQ achieves up to 2.88x (2.26x on average) throughput improvement in inference, showing great advantages over state-of-the-art works.

Sun 3 Mar

Displayed time zone: London change

18:00 - 20:00
Reception and Poster SessionMain Conference at Strathblane Hall
18:00
2h
Poster
POSTER - H3: A Hash-table Based and Holistically Optimized High-Performance Sparse Tensor Contraction
Main Conference
Guofeng Feng Institute of Computing Technology, Chinese Academy of Sciences; University of Chinese Academy of Sciences, Weile Jia Institute of Computing Technology, Chinese Academy of Sciences, Ninghui Sun State Key Laboratory of Computer Architecture, Institute of Computing Technology, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Guangming Tan Chinese Academy of Sciences(CAS), Jiajia Li North Carolina State University
18:00
2h
Poster
POSTER - P2Res: Pattern-Aware Sparse Communication for Scalable Recommendation Model Training
Main Conference
Jiaao He Tsinghua University, China, Shengqi Chen Tsinghua University, Jidong Zhai Tsinghua University
18:00
2h
Poster
POSTER - gZCCL: Compression-Accelerated Collective Communication Framework for GPU Clusters
Main Conference
Jiajun Huang University of California, Riverside, Sheng Di Argonne National Laboratory, Xiaodong Yu Stevens Institute of Technology, Yujia Zhai University of California Riverside, Jinyang Liu University of California, Riverside, Yafan Huang The University of Iowa, Ken Raffenetti Argonne National Laboratory, Hui Zhou Argonne National Laboratory, Kai Zhao Florida State University, zizhong chen University of California, Riverside, Franck Cappello Argonne National Laboratory, Yanfei Guo Argonne National Laboratory, Rajeev Thakur Argonne National Laboratory
18:00
2h
Poster
POSTER - RadiK: Scalable Radix Top-K Selection on GPUs
Main Conference
Yifei Li Alibaba Group, Bole Zhou Independent, Jiejing Zhang Alibaba Group, Xuechao Wei Alibaba Group, Yinghan Li Alibaba Group, Yingda Chen Alibaba Group
18:00
2h
Poster
POSTER - Accelerating High-Precision Integer Multiplication used in Cryptosystems with GPUs
Main Conference
Zhuoran Ji Shandong University, Zhaorui Zhang The Hong Kong Polytechnic University, Jiming Xu Ant Group, Lei Ju Shandong University
18:00
2h
Poster
POSTER - Enabling Extreme-Scale Phase Field Simulation with In-situ Feature Extraction
Main Conference
Zhichen Feng Computer Network Information Center, Chinese Academy of Sciences; University of Chinese Academy of Sciences, Jialin Li Computer Network Information Center, Chinese Academy of Sciences; University of Chinese Academy of Sciences, Yaqian Gao Computer Network Information Center, Chinese Academy of Sciences; University of Chinese Academy of Sciences, TianShaobo Computer Network Information Center, Chinese Academy of Sciences; University of Chinese Academy of Sciences, Huang Ye Computer Network Information Center, Chinese Academy of Sciences, Jian Zhang Computer Network Information Center, Chinese Academy of Sciences
18:00
2h
Poster
POSTER - ParGNN: Efficient Training for Large-Scale Graph Neural Network on GPU Clusters
Main Conference
lishunde Computer Network Information Center, Chinese Academy of Sciences;University of Chinese Academy of Sciences, Junyu Gu Computer Network Information Center, Chinese Academy of Sciences;University of Chinese Academy of Sciences, Jue Wang Computer Network Information Center, Chinese Academy of Sciences, Tiechui Yao Computer Network Information Center, Chinese Academy of Sciences, ZhiQiang Liang Computer Network Information Center, Chinese Academy of Sciences, Yumeng Shi Computer Network Information Center, Chinese Academy of Sciences, Shigang Li Beijing University of Posts and Telecommunications, Weiting Xi North China Electric Power University, Shushen Li North China Electric Power University, Chunbao Zhou Computer Network Information Center, Chinese Academy of Sciences, Yangang Wang Computer Network Information Center, Chinese Academy of Sciences, Xuebin Chi Computer Network Information Center, Chinese Academy of Sciences;University of Chinese Academy of Sciences
18:00
2h
Poster
POSTER - RELAX: Durable Data Structures with Swift Recovery
Main Conference
Almog Zur Technion, Nachshon Cohen Amazon, Michal Friedman ETH Zurich, Switzerland, Erez Petrank Technion
18:00
2h
Poster
POSTER - FineCo: Fine-grained Heterogeneous Resource Management for Concurrent DNN Inferences
Main Conference
Lixian Ma State Key Lab of Processors, Institute of Computing Technology, CAS, Beijing, Haoruo Chen State Key Lab of Processors, Institute of Computing Technology, CAS, Beijing, En Shao State Key Lab of Processors, Institute of Computing Technology, CAS, Beijing, Leping Wang State Key Lab of Processors, Institute of Computing Technology, CAS, Beijing, Quan Chen Shanghai Jiao Tong University, Guangming Tan Chinese Academy of Sciences(CAS)
18:00
2h
Poster
POSTER - LLM-PQ:Serving LLM on Heterogeneous Clusters with Phase-Aware Partition and Adaptive Quantization
Main Conference
Juntao Zhao The University of Hong Kong, Borui Wan The University of Hong Kong, Chuan Wu The University of Hong Kong, Yanghua Peng ByteDance Inc., Haibin Lin ByteDance Inc.
18:00
2h
Poster
POSTER - StructMG: A Fast and Scalable Structured Multigrid
Main Conference
Yi Zong Tsinghua University, Xinliang Wang Huawei Technologies Co., Ltd, Haopeng Huang Tsinghua University, Chensong Zhang Academy of Mathematics and Systems Science, Xiaowen Xu Institute of Applied Physics and Computational Mathematics, Jian Sun CMA Earth System Modeling and Prediction Center, Bowen Yan Tsinghua University, Qin Wang Huawei Technologies Co., Ltd, Sicong Li Huawei Technologies Co., Ltd, Zhaohui Ding Huawei Technologies Co., Ltd, Wei Xue Tsinghua University
18:00
2h
Poster
POSTER - OCToPus: Semantic-aware Concurrency Control for Blockchain Transactions
Main Conference
dePaul Miller Lehigh University, Henry F. Korth Lehigh University, Roberto Palmieri Lehigh University