Designing flink pipelines in IoT mashup tools

Tanmaya Mahapatra, Ilias Gerostathopoulos, Federico Alonso Fernández Moreno, Christian Prehofer

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

Internet of Things (IoT) applications are generating increasingly large amounts of data because of continuous activity and periodical sensing capabilities. Processing the data generated by IoT applications is necessary to derive important insights-for example, processing data from CO emissions can help municipal authorities apply traffic restrictions in order to improve a city's air quality. State-of-the-art stream-processing platforms, such as Apache Flink, can be used to process large amounts of data streams from different IoT devices. However, it is difficult to both set-up and write applications for these platforms; this is also manifested in the increasing need for data analysts and engineers. A promising solution is to enable domain experts, who are not necessarily programmers, to develop the necessary stream pipelines by providing them with domain-specific graphical tools. We present our proposal for a state-of-the-art mashup tool, originally developed for wiring IoT applications together, to graphically design streaming data pipelines and deploy them as a Flink application. Our prototype and experimental evaluation show that our proposal is feasible and potentially impactful.

Original languageEnglish
Title of host publication4th Norwegian Big Data Symposium, NOBIDS 2018
Pages41-53
Number of pages13
Volume2316
Publication statusPublished - 1 Jan 2018
Externally publishedYes
Event4th Norwegian Big Data Symposium, NOBIDS 2018 - Trondheim, Norway
Duration: 14 Nov 2018 → …

Publication series

NameCEUR Workshop Proceedings
PublisherCEUR Workshop Proceedings
ISSN (Print)1613-0073

Conference

Conference4th Norwegian Big Data Symposium, NOBIDS 2018
Country/TerritoryNorway
CityTrondheim
Period14/11/18 → …

Funding

This work is part of the TUM Living Lab Connected Mobility (TUM LLCM) project and has been funded by the Bavarian Ministry of Economic Affairs, Energy and Technology (StMWi) through the Center Digitisation.Bavaria, an initiative of the Bavarian State Government.

FundersFunder number
Bavarian Ministry of Economic Affairs, Energy and Technology
Bavarian State Government
Bayerisches Staatsministerium für Wirtschaft, Infrastruktur, Verkehr und Technologie
Technische Universität München

    Keywords

    • Flink pipelines
    • Graphical tool
    • IoT mashup tools
    • Stream analytics

    Fingerprint

    Dive into the research topics of 'Designing flink pipelines in IoT mashup tools'. Together they form a unique fingerprint.

    Cite this