Exploring HPC and Big Data Convergence: A Graph Processing Study on Intel Knights Landing

Alexandru Uta, Ana Lucia Varbanescu, Ahmed Musaafir, Chris Lemaire, Alexandru Iosup

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

The question 'Can big data and HPC infrastructure converge?' has important implications for many operators and clients of modern computing. However, answering it is challenging. The hardware is currently different, and fast evolving: big data uses machines with modest numbers of fat cores per socket, large caches, and much memory, whereas HPC uses machines with larger numbers of (thinner) cores, non-trivial NUMA architectures, and fast interconnects. In this work, we investigate the convergence of big data and HPC infrastructure for one of the most challenging application domains, the highly irregular graph processing. We contrast through a systematic, experimental study of over 300,000 core-hours the performance of a modern multicore, Intel Knights Landing (KNL) and of traditional big data hardware, in processing representative graph workloads using state-of-the-art graph analytics platforms. The experimental results indicate KNL is convergence-ready, performance-wise, but only after extensive and expert-level tuning of software and hardware parameters.

Original languageEnglish
Title of host publicationProceedings - 2018 IEEE International Conference on Cluster Computing, CLUSTER 2018
PublisherInstitute of Electrical and Electronics Engineers, Inc.
Pages66-77
Number of pages12
ISBN (Electronic)9781538683194
DOIs
Publication statusPublished - 1 Nov 2018
Event2018 IEEE International Conference on Cluster Computing, CLUSTER 2018 - Belfast, United Kingdom
Duration: 10 Sep 201813 Sep 2018

Conference

Conference2018 IEEE International Conference on Cluster Computing, CLUSTER 2018
CountryUnited Kingdom
CityBelfast
Period10/09/1813/09/18

Fingerprint

Landing
Processing
Hardware
Oils and fats
Tuning
Data storage equipment
Big data

Keywords

  • Big Data
  • Graph Processing
  • HPC
  • HPC Big Data convergence
  • Intel Knights Landing
  • Performance evaluation

Cite this

Uta, A., Varbanescu, A. L., Musaafir, A., Lemaire, C., & Iosup, A. (2018). Exploring HPC and Big Data Convergence: A Graph Processing Study on Intel Knights Landing. In Proceedings - 2018 IEEE International Conference on Cluster Computing, CLUSTER 2018 (pp. 66-77). [8514860] Institute of Electrical and Electronics Engineers, Inc.. https://doi.org/10.1109/CLUSTER.2018.00019
Uta, Alexandru ; Varbanescu, Ana Lucia ; Musaafir, Ahmed ; Lemaire, Chris ; Iosup, Alexandru. / Exploring HPC and Big Data Convergence : A Graph Processing Study on Intel Knights Landing. Proceedings - 2018 IEEE International Conference on Cluster Computing, CLUSTER 2018. Institute of Electrical and Electronics Engineers, Inc., 2018. pp. 66-77
@inproceedings{b0f30cc65be143c48dd7ccfe54b984e9,
title = "Exploring HPC and Big Data Convergence: A Graph Processing Study on Intel Knights Landing",
abstract = "The question 'Can big data and HPC infrastructure converge?' has important implications for many operators and clients of modern computing. However, answering it is challenging. The hardware is currently different, and fast evolving: big data uses machines with modest numbers of fat cores per socket, large caches, and much memory, whereas HPC uses machines with larger numbers of (thinner) cores, non-trivial NUMA architectures, and fast interconnects. In this work, we investigate the convergence of big data and HPC infrastructure for one of the most challenging application domains, the highly irregular graph processing. We contrast through a systematic, experimental study of over 300,000 core-hours the performance of a modern multicore, Intel Knights Landing (KNL) and of traditional big data hardware, in processing representative graph workloads using state-of-the-art graph analytics platforms. The experimental results indicate KNL is convergence-ready, performance-wise, but only after extensive and expert-level tuning of software and hardware parameters.",
keywords = "Big Data, Graph Processing, HPC, HPC Big Data convergence, Intel Knights Landing, Performance evaluation",
author = "Alexandru Uta and Varbanescu, {Ana Lucia} and Ahmed Musaafir and Chris Lemaire and Alexandru Iosup",
year = "2018",
month = "11",
day = "1",
doi = "10.1109/CLUSTER.2018.00019",
language = "English",
pages = "66--77",
booktitle = "Proceedings - 2018 IEEE International Conference on Cluster Computing, CLUSTER 2018",
publisher = "Institute of Electrical and Electronics Engineers, Inc.",

}

Uta, A, Varbanescu, AL, Musaafir, A, Lemaire, C & Iosup, A 2018, Exploring HPC and Big Data Convergence: A Graph Processing Study on Intel Knights Landing. in Proceedings - 2018 IEEE International Conference on Cluster Computing, CLUSTER 2018., 8514860, Institute of Electrical and Electronics Engineers, Inc., pp. 66-77, 2018 IEEE International Conference on Cluster Computing, CLUSTER 2018, Belfast, United Kingdom, 10/09/18. https://doi.org/10.1109/CLUSTER.2018.00019

Exploring HPC and Big Data Convergence : A Graph Processing Study on Intel Knights Landing. / Uta, Alexandru; Varbanescu, Ana Lucia; Musaafir, Ahmed; Lemaire, Chris; Iosup, Alexandru.

Proceedings - 2018 IEEE International Conference on Cluster Computing, CLUSTER 2018. Institute of Electrical and Electronics Engineers, Inc., 2018. p. 66-77 8514860.

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - Exploring HPC and Big Data Convergence

T2 - A Graph Processing Study on Intel Knights Landing

AU - Uta, Alexandru

AU - Varbanescu, Ana Lucia

AU - Musaafir, Ahmed

AU - Lemaire, Chris

AU - Iosup, Alexandru

PY - 2018/11/1

Y1 - 2018/11/1

N2 - The question 'Can big data and HPC infrastructure converge?' has important implications for many operators and clients of modern computing. However, answering it is challenging. The hardware is currently different, and fast evolving: big data uses machines with modest numbers of fat cores per socket, large caches, and much memory, whereas HPC uses machines with larger numbers of (thinner) cores, non-trivial NUMA architectures, and fast interconnects. In this work, we investigate the convergence of big data and HPC infrastructure for one of the most challenging application domains, the highly irregular graph processing. We contrast through a systematic, experimental study of over 300,000 core-hours the performance of a modern multicore, Intel Knights Landing (KNL) and of traditional big data hardware, in processing representative graph workloads using state-of-the-art graph analytics platforms. The experimental results indicate KNL is convergence-ready, performance-wise, but only after extensive and expert-level tuning of software and hardware parameters.

AB - The question 'Can big data and HPC infrastructure converge?' has important implications for many operators and clients of modern computing. However, answering it is challenging. The hardware is currently different, and fast evolving: big data uses machines with modest numbers of fat cores per socket, large caches, and much memory, whereas HPC uses machines with larger numbers of (thinner) cores, non-trivial NUMA architectures, and fast interconnects. In this work, we investigate the convergence of big data and HPC infrastructure for one of the most challenging application domains, the highly irregular graph processing. We contrast through a systematic, experimental study of over 300,000 core-hours the performance of a modern multicore, Intel Knights Landing (KNL) and of traditional big data hardware, in processing representative graph workloads using state-of-the-art graph analytics platforms. The experimental results indicate KNL is convergence-ready, performance-wise, but only after extensive and expert-level tuning of software and hardware parameters.

KW - Big Data

KW - Graph Processing

KW - HPC

KW - HPC Big Data convergence

KW - Intel Knights Landing

KW - Performance evaluation

UR - http://www.scopus.com/inward/record.url?scp=85057283772&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85057283772&partnerID=8YFLogxK

U2 - 10.1109/CLUSTER.2018.00019

DO - 10.1109/CLUSTER.2018.00019

M3 - Conference contribution

SP - 66

EP - 77

BT - Proceedings - 2018 IEEE International Conference on Cluster Computing, CLUSTER 2018

PB - Institute of Electrical and Electronics Engineers, Inc.

ER -

Uta A, Varbanescu AL, Musaafir A, Lemaire C, Iosup A. Exploring HPC and Big Data Convergence: A Graph Processing Study on Intel Knights Landing. In Proceedings - 2018 IEEE International Conference on Cluster Computing, CLUSTER 2018. Institute of Electrical and Electronics Engineers, Inc. 2018. p. 66-77. 8514860 https://doi.org/10.1109/CLUSTER.2018.00019