Efficient Embedding of Dynamic Languages in Big-Data Analytics

Luca Salucci; Daniele Bonetta; Walter Binder

doi:10.1109/ICDCSW.2016.40

Efficient Embedding of Dynamic Languages in Big-Data Analytics

Luca Salucci
, Daniele Bonetta
, Walter Binder

Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

Abstract

Over the last years several frameworks have emerged in the field of big-data analytics. Recent frameworks expose a developer-friendly API via dynamic languages such as Python. Unfortunately, the integration of dynamic languages with the parallel and distributed runtime of such frameworks is cumbersome, as it requires the integration of two or more language virtual machines via inter-process communication, introducing communication overheads and reducing the benefits of the shared memory present in modern multicore machines. In this paper we highlight the advantages of hosting multiple language runtimes in a single shared (language) virtual machine, and the possible performance gain of such an approach in the context of the Apache Spark framework.

Original language	English
Title of host publication	Proceedings - 2016 IEEE 36th International Conference on Distributed Computing Systems Workshops, ICDCSW 2016
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	19-24
ISBN (Electronic)	9781509014828
DOIs	https://doi.org/10.1109/ICDCSW.2016.40
Publication status	Published - 23 Nov 2016
Externally published	Yes
Event	36th IEEE International Conference on Distributed Computing Systems Workshops, ICDCSW 2016 - Nara, Japan Duration: 27 Jun 2016 → 30 Jun 2016

Conference

Conference	36th IEEE International Conference on Distributed Computing Systems Workshops, ICDCSW 2016
Country/Territory	Japan
City	Nara
Period	27/06/16 → 30/06/16

Access to Document

10.1109/ICDCSW.2016.40

Persistent URL (handle)

Persistent link

Cite this

@inproceedings{410f022ac5e04278b494019939c69a6d,

title = "Efficient Embedding of Dynamic Languages in Big-Data Analytics",

abstract = "Over the last years several frameworks have emerged in the field of big-data analytics. Recent frameworks expose a developer-friendly API via dynamic languages such as Python. Unfortunately, the integration of dynamic languages with the parallel and distributed runtime of such frameworks is cumbersome, as it requires the integration of two or more language virtual machines via inter-process communication, introducing communication overheads and reducing the benefits of the shared memory present in modern multicore machines. In this paper we highlight the advantages of hosting multiple language runtimes in a single shared (language) virtual machine, and the possible performance gain of such an approach in the context of the Apache Spark framework.",

author = "Luca Salucci and Daniele Bonetta and Walter Binder",

year = "2016",

month = nov,

day = "23",

doi = "10.1109/ICDCSW.2016.40",

language = "English",

pages = "19--24",

booktitle = "Proceedings - 2016 IEEE 36th International Conference on Distributed Computing Systems Workshops, ICDCSW 2016",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

address = "United States",

note = "36th IEEE International Conference on Distributed Computing Systems Workshops, ICDCSW 2016 ; Conference date: 27-06-2016 Through 30-06-2016",

}

Salucci, L, Bonetta, D & Binder, W 2016, Efficient Embedding of Dynamic Languages in Big-Data Analytics. in Proceedings - 2016 IEEE 36th International Conference on Distributed Computing Systems Workshops, ICDCSW 2016. Institute of Electrical and Electronics Engineers Inc., pp. 19-24, 36th IEEE International Conference on Distributed Computing Systems Workshops, ICDCSW 2016, Nara, Japan, 27/06/16. https://doi.org/10.1109/ICDCSW.2016.40

Efficient Embedding of Dynamic Languages in Big-Data Analytics. / Salucci, Luca; Bonetta, Daniele; Binder, Walter.
Proceedings - 2016 IEEE 36th International Conference on Distributed Computing Systems Workshops, ICDCSW 2016. Institute of Electrical and Electronics Engineers Inc., 2016. p. 19-24.

Research output: Chapter in Book / Report / Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Efficient Embedding of Dynamic Languages in Big-Data Analytics

AU - Salucci, Luca

AU - Bonetta, Daniele

AU - Binder, Walter

PY - 2016/11/23

Y1 - 2016/11/23

N2 - Over the last years several frameworks have emerged in the field of big-data analytics. Recent frameworks expose a developer-friendly API via dynamic languages such as Python. Unfortunately, the integration of dynamic languages with the parallel and distributed runtime of such frameworks is cumbersome, as it requires the integration of two or more language virtual machines via inter-process communication, introducing communication overheads and reducing the benefits of the shared memory present in modern multicore machines. In this paper we highlight the advantages of hosting multiple language runtimes in a single shared (language) virtual machine, and the possible performance gain of such an approach in the context of the Apache Spark framework.

AB - Over the last years several frameworks have emerged in the field of big-data analytics. Recent frameworks expose a developer-friendly API via dynamic languages such as Python. Unfortunately, the integration of dynamic languages with the parallel and distributed runtime of such frameworks is cumbersome, as it requires the integration of two or more language virtual machines via inter-process communication, introducing communication overheads and reducing the benefits of the shared memory present in modern multicore machines. In this paper we highlight the advantages of hosting multiple language runtimes in a single shared (language) virtual machine, and the possible performance gain of such an approach in the context of the Apache Spark framework.

UR - https://www.scopus.com/pages/publications/85006789458

UR - https://www.scopus.com/pages/publications/85006789458#tab=citedBy

U2 - 10.1109/ICDCSW.2016.40

DO - 10.1109/ICDCSW.2016.40

M3 - Conference contribution

SP - 19

EP - 24

BT - Proceedings - 2016 IEEE 36th International Conference on Distributed Computing Systems Workshops, ICDCSW 2016

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 36th IEEE International Conference on Distributed Computing Systems Workshops, ICDCSW 2016

Y2 - 27 June 2016 through 30 June 2016

ER -

Efficient Embedding of Dynamic Languages in Big-Data Analytics

Abstract

Conference

Access to Document

Persistent URL (handle)

Other files and links

Fingerprint

Cite this