PPoPP 2024
Sat 2 - Wed 6 March 2024 Edinburgh, United Kingdom
Wed 6 Mar 2024 10:40 - 11:00 at Moorfoot - Linear Algebra Chair(s): I-Ting Angelina Lee

We propose a novel approach to iterated sparse matrix dense matrix multiplication, a fundamental computational kernel in scientific computing and graph neural network training. In cases where matrix sizes exceed the memory of a single compute node, data transfer becomes a bottleneck. An approach based on dense matrix multiplication algorithms leads to suboptimal scalability and fails to exploit the sparsity in the problem. To address these challenges, we propose decomposing the sparse matrix into a small number of highly structured matrices called \emph{arrow} matrices, which are connected by permutations. Our approach enables communication-avoiding multiplications, achieving a polynomial reduction in communication volume per iteration for matrices corresponding to planar graphs and other minor-excluded families of graphs. Our evaluation demonstrates that our approach outperforms a state-of-the-art method for sparse matrix multiplication on matrices with hundreds of millions of rows, offering near-linear strong and weak scaling.

Wed 6 Mar

Displayed time zone: London change

10:00 - 11:00
Linear AlgebraMain Conference at Moorfoot
Chair(s): I-Ting Angelina Lee Washington University in St. Louis, USA
10:00
20m
Talk
A Row Decomposition-based Approach for Sparse Matrix Multiplication on GPUs
Main Conference
Pang Meng Department of Computer Science and Technology, Tsinghua University, Xiang Fei Department of Computer Science and Technology, Tsinghua University, Peng Qu Department of Computer Science and Technology, Tsinghua University, Youhui Zhang Department of Computer Science and Technology, Tsinghua University, Zhaolin Li Department of Computer Science and Technology, Tsinghua University
Link to publication DOI
10:20
20m
Talk
Fast Kronecker Matrix-Matrix Multiplications on GPUs
Main Conference
Abhinav Jangda Microsoft Research, Mohit Yadav University of Massachusetts Amherst
Link to publication DOI
10:40
20m
Talk
Arrow Matrix Decomposition: A Novel Approach for Communication-Efficient Sparse Matrix Multiplication
Main Conference
Lukas Gianinazzi ETH Zurich, Alexandros Nikolaos Ziogas ETH Zurich, Piotr Luczynski ETH Zurich, Langwen Huang ETH Zurich, Saleh Ashkboosh ETH Zurich, Florian Scheidl ETH Zurich, Armon Carigiet ETH Zurich, Chio Ge ETH Zurich, Nabil Abubaker ETH Zurich, Maciej Besta ETH Zurich, Tal Ben-Nun Lawrence Livermore National Laboratory, Torsten Hoefler ETH Zurich
Link to publication DOI