TY - GEN
T1 - Graphical spark programming in IoT mashup tools
AU - Mahapatra, Tanmaya
AU - Gerostathopoulos, Ilias
AU - Prehofer, Christian
AU - Gore, Shilpa Ghanashyam
PY - 2018/11/30
Y1 - 2018/11/30
N2 - With the unprecedented rise in the number of IoT devices, the amount of data generated from sensors is huge and often demands an in-depth analysis to acquire suitable insights. Mashup tools, used primarily for intuitive graphical programming of IoT applications, can help both for efficiently prototyping and also data analytics pipelines. In this study, we focus on the tight integration of data analytics capabilities of Spark in IoT mashup tools. The main challenge in this direction is the presence of a wide range of data interfaces and APIs in the Spark ecosystem. In this study, we contribute to current applications by (i) providing a thorough analysis of the Spark ecosystem and selecting suitable data interfaces for use in a graphical flow-based programming paradigm, (ii) devising a novel, generic approach for programming Spark from graphical flows that comprises early-stage validation and code generation of Java Spark programs. The approach is implemented in aFlux, our JVM-based mashup tool and is evaluated in three use cases showcasing the machine learning and stream analytics capabilities of Spark.
AB - With the unprecedented rise in the number of IoT devices, the amount of data generated from sensors is huge and often demands an in-depth analysis to acquire suitable insights. Mashup tools, used primarily for intuitive graphical programming of IoT applications, can help both for efficiently prototyping and also data analytics pipelines. In this study, we focus on the tight integration of data analytics capabilities of Spark in IoT mashup tools. The main challenge in this direction is the presence of a wide range of data interfaces and APIs in the Spark ecosystem. In this study, we contribute to current applications by (i) providing a thorough analysis of the Spark ecosystem and selecting suitable data interfaces for use in a graphical flow-based programming paradigm, (ii) devising a novel, generic approach for programming Spark from graphical flows that comprises early-stage validation and code generation of Java Spark programs. The approach is implemented in aFlux, our JVM-based mashup tool and is evaluated in three use cases showcasing the machine learning and stream analytics capabilities of Spark.
KW - data ana-lytics
KW - end-users
KW - graphical flows
KW - Internet of Things
KW - IoT applications
KW - mashup tools
KW - Spark analytics
UR - http://www.scopus.com/inward/record.url?scp=85059982001&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85059982001&partnerID=8YFLogxK
U2 - 10.1109/IoTSMS.2018.8554665
DO - 10.1109/IoTSMS.2018.8554665
M3 - Conference contribution
AN - SCOPUS:85059982001
T3 - 2018 5th International Conference on Internet of Things: Systems, Management and Security, IoTSMS 2018
SP - 163
EP - 170
BT - 2018 5th International Conference on Internet of Things
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 5th International Conference on Internet of Things: Systems, Management and Security, IoTSMS 2018
Y2 - 15 October 2018 through 18 October 2018
ER -