Abstract
All-pairs compute problems apply a user-defined function to each combination of two items of a given data set. Although these problems present an abundance of parallelism, data reuse must be exploited to achieve good performance. Several researchers considered this problem, either resorting to partial replication with static work distribution or dynamic scheduling with full replication. In contrast, we present a solution that relies on hierarchical multi-level software-based caches to maximize data reuse at each level in the distributed memory hierarchy, combined with a divide-and-conquer approach to exploit data locality, hierarchical work-stealing to dynamically balance the workload, and asynchronous processing to maximize resource utilization. We evaluate our solution using three real-world applications (from digital forensics, localization microscopy, and bioinformatics) on different platforms (from a desktop machine to a supercomputer). Results shows excellent efficiency and scalability when scaling to 96 GPUs, even obtaining super-linear speedups due to a distributed cache.
Original language | English |
---|---|
Title of host publication | SC20: International Conference for High Performance Computing, Networking, Storage and Analysis |
Subtitle of host publication | [Proceedings] |
Publisher | IEEE Computer Society |
Number of pages | 12 |
ISBN (Electronic) | 9781728199986 |
ISBN (Print) | 9781728199993 |
DOIs | |
Publication status | Published - 2021 |
Event | 2020 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2020 - Virtual, Atlanta, United States Duration: 9 Nov 2020 → 19 Nov 2020 |
Publication series
Name | International Conference for High Performance Computing, Networking, Storage and Analysis |
---|---|
Publisher | IEEE |
Number | November |
Volume | 2020 |
ISSN (Print) | 2167-4329 |
ISSN (Electronic) | 2167-4337 |
Conference
Conference | 2020 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2020 |
---|---|
Country/Territory | United States |
City | Virtual, Atlanta |
Period | 9/11/20 → 19/11/20 |
Bibliographical note
Publisher Copyright:© 2020 IEEE.
Keywords
- all-pairs computation
- data reuse
- distributed cache
- GPU
- heterogeneous computing
- work-stealing