Graphical spark programming in IoT mashup tools

Tanmaya Mahapatra, Ilias Gerostathopoulos, Christian Prehofer, Shilpa Ghanashyam Gore

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

With the unprecedented rise in the number of IoT devices, the amount of data generated from sensors is huge and often demands an in-depth analysis to acquire suitable insights. Mashup tools, used primarily for intuitive graphical programming of IoT applications, can help both for efficiently prototyping and also data analytics pipelines. In this study, we focus on the tight integration of data analytics capabilities of Spark in IoT mashup tools. The main challenge in this direction is the presence of a wide range of data interfaces and APIs in the Spark ecosystem. In this study, we contribute to current applications by (i) providing a thorough analysis of the Spark ecosystem and selecting suitable data interfaces for use in a graphical flow-based programming paradigm, (ii) devising a novel, generic approach for programming Spark from graphical flows that comprises early-stage validation and code generation of Java Spark programs. The approach is implemented in aFlux, our JVM-based mashup tool and is evaluated in three use cases showcasing the machine learning and stream analytics capabilities of Spark.

Original languageEnglish
Title of host publication2018 5th International Conference on Internet of Things
Subtitle of host publicationSystems, Management and Security, IoTSMS 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages163-170
Number of pages8
ISBN (Electronic)9781538695852
DOIs
Publication statusPublished - 30 Nov 2018
Externally publishedYes
Event5th International Conference on Internet of Things: Systems, Management and Security, IoTSMS 2018 - Valencia, Spain
Duration: 15 Oct 201818 Oct 2018

Publication series

Name2018 5th International Conference on Internet of Things: Systems, Management and Security, IoTSMS 2018

Conference

Conference5th International Conference on Internet of Things: Systems, Management and Security, IoTSMS 2018
Country/TerritorySpain
CityValencia
Period15/10/1818/10/18

Funding

ACKNOWLEDGEMENT This work is part of the TUM Living Lab Connected Mobility (TUM LLCM) project and has been funded by the Bavarian Ministry of Economic Affairs, Energy and Technology (StMWi) through the Center Digitisation.Bavaria, an initiative of the Bavarian State Government.

FundersFunder number
Bavarian Ministry of Economic Affairs, Energy and Technology
Bavarian State Government
Bayerisches Staatsministerium für Wirtschaft, Infrastruktur, Verkehr und Technologie
Technische Universität München

    Keywords

    • data ana-lytics
    • end-users
    • graphical flows
    • Internet of Things
    • IoT applications
    • mashup tools
    • Spark analytics

    Fingerprint

    Dive into the research topics of 'Graphical spark programming in IoT mashup tools'. Together they form a unique fingerprint.

    Cite this