Exploring HPC and Big Data Convergence: A Graph Processing Study on Intel Knights Landing

Alexandru Uta, Ana Lucia Varbanescu, Ahmed Musaafir, Chris Lemaire, Alexandru Iosup

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

229 Downloads (Pure)

Abstract

The question 'Can big data and HPC infrastructure converge?' has important implications for many operators and clients of modern computing. However, answering it is challenging. The hardware is currently different, and fast evolving: big data uses machines with modest numbers of fat cores per socket, large caches, and much memory, whereas HPC uses machines with larger numbers of (thinner) cores, non-trivial NUMA architectures, and fast interconnects. In this work, we investigate the convergence of big data and HPC infrastructure for one of the most challenging application domains, the highly irregular graph processing. We contrast through a systematic, experimental study of over 300,000 core-hours the performance of a modern multicore, Intel Knights Landing (KNL) and of traditional big data hardware, in processing representative graph workloads using state-of-the-art graph analytics platforms. The experimental results indicate KNL is convergence-ready, performance-wise, but only after extensive and expert-level tuning of software and hardware parameters.

Original languageEnglish
Title of host publication2018 IEEE International Conference on Cluster Computing (CLUSTER)
Subtitle of host publication[Proceedings]
PublisherInstitute of Electrical and Electronics Engineers, Inc.
Pages66-77
Number of pages12
ISBN (Electronic)9781538683194
DOIs
Publication statusPublished - 2018
Event2018 IEEE International Conference on Cluster Computing, CLUSTER 2018 - Belfast, United Kingdom
Duration: 10 Sept 201813 Sept 2018

Conference

Conference2018 IEEE International Conference on Cluster Computing, CLUSTER 2018
Country/TerritoryUnited Kingdom
CityBelfast
Period10/09/1813/09/18

Funding

Work supported by Dutch projects Vidi MagnaData, COMMIT/. The infrastructure was kindly provided by Intel as a gift, by the Dutch Supercomputing Center through the HPGraph NWO grant, and by the Dutch DAS5 Supercomputing infrastructure co-sponsored by NWO.

FundersFunder number
COMMIT
Dutch Supercomputing Center
Dutch projects Vidi MagnaData
Intel Corporation
Nederlandse Organisatie voor Wetenschappelijk Onderzoek

    Keywords

    • Big Data
    • Graph Processing
    • HPC
    • HPC Big Data convergence
    • Intel Knights Landing
    • Performance evaluation

    Fingerprint

    Dive into the research topics of 'Exploring HPC and Big Data Convergence: A Graph Processing Study on Intel Knights Landing'. Together they form a unique fingerprint.

    Cite this