On the Use of In-memory Analytics Workflows to Compute eScience Indicators from Large Climate Datasets

Alessandro DAnca, Cosimo Palazzo, Donatello Elia, Sandro Fiore, Ioannis Bistinas, Kristin Bottcher, Victoria Bennett, Giovanni Aloisio

Research output: Chapter in Book / Report / Conference proceedingChapterAcademicpeer-review

Abstract

The need to apply complex algorithms on large volumes of data is boosting the development of technological solutions able to satisfy big data analytics needs in Cloud and HPC environments. In this context Ophidia represents a big data analytics framework for eScience offering a cross-domain solution for managing scientific, multi-dimensional data. It also exploits an in-memory-based distributed data storage and provides support for the submission of complex workflows by means of various interfaces compliant to well-known standards. This paper presents some applications of Ophidia for the computation of climate indicators defined in the CLIPC project, the WPS interface used for the submission and the workflow based approach employed.
Original languageEnglish
Title of host publicationProceedings - 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017
PublisherInstitute of Electrical and Electronics Engineers, Inc.
Pages1035-1043
Number of pages9
ISBN (Print)9781509066100
DOIs
Publication statusPublished - 10 Jul 2017

Publication series

NameProceedings - 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017

Fingerprint

Data storage equipment
Big data

Keywords

  • Big data analytics
  • Climate impact indicator
  • Performance evaluation
  • Scientific workflows

Cite this

DAnca, A., Palazzo, C., Elia, D., Fiore, S., Bistinas, I., Bottcher, K., ... Aloisio, G. (2017). On the Use of In-memory Analytics Workflows to Compute eScience Indicators from Large Climate Datasets. In Proceedings - 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017 (pp. 1035-1043). (Proceedings - 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017). Institute of Electrical and Electronics Engineers, Inc.. https://doi.org/10.1109/CCGRID.2017.132
DAnca, Alessandro ; Palazzo, Cosimo ; Elia, Donatello ; Fiore, Sandro ; Bistinas, Ioannis ; Bottcher, Kristin ; Bennett, Victoria ; Aloisio, Giovanni. / On the Use of In-memory Analytics Workflows to Compute eScience Indicators from Large Climate Datasets. Proceedings - 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017. Institute of Electrical and Electronics Engineers, Inc., 2017. pp. 1035-1043 (Proceedings - 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017).
@inbook{107ab758cd6540ffbd0d3cff6681d8aa,
title = "On the Use of In-memory Analytics Workflows to Compute eScience Indicators from Large Climate Datasets",
abstract = "The need to apply complex algorithms on large volumes of data is boosting the development of technological solutions able to satisfy big data analytics needs in Cloud and HPC environments. In this context Ophidia represents a big data analytics framework for eScience offering a cross-domain solution for managing scientific, multi-dimensional data. It also exploits an in-memory-based distributed data storage and provides support for the submission of complex workflows by means of various interfaces compliant to well-known standards. This paper presents some applications of Ophidia for the computation of climate indicators defined in the CLIPC project, the WPS interface used for the submission and the workflow based approach employed.",
keywords = "Big data analytics, Climate impact indicator, Performance evaluation, Scientific workflows",
author = "Alessandro DAnca and Cosimo Palazzo and Donatello Elia and Sandro Fiore and Ioannis Bistinas and Kristin Bottcher and Victoria Bennett and Giovanni Aloisio",
year = "2017",
month = "7",
day = "10",
doi = "10.1109/CCGRID.2017.132",
language = "English",
isbn = "9781509066100",
series = "Proceedings - 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017",
publisher = "Institute of Electrical and Electronics Engineers, Inc.",
pages = "1035--1043",
booktitle = "Proceedings - 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017",

}

DAnca, A, Palazzo, C, Elia, D, Fiore, S, Bistinas, I, Bottcher, K, Bennett, V & Aloisio, G 2017, On the Use of In-memory Analytics Workflows to Compute eScience Indicators from Large Climate Datasets. in Proceedings - 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017. Proceedings - 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017, Institute of Electrical and Electronics Engineers, Inc., pp. 1035-1043. https://doi.org/10.1109/CCGRID.2017.132

On the Use of In-memory Analytics Workflows to Compute eScience Indicators from Large Climate Datasets. / DAnca, Alessandro; Palazzo, Cosimo; Elia, Donatello; Fiore, Sandro; Bistinas, Ioannis; Bottcher, Kristin; Bennett, Victoria; Aloisio, Giovanni.

Proceedings - 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017. Institute of Electrical and Electronics Engineers, Inc., 2017. p. 1035-1043 (Proceedings - 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017).

Research output: Chapter in Book / Report / Conference proceedingChapterAcademicpeer-review

TY - CHAP

T1 - On the Use of In-memory Analytics Workflows to Compute eScience Indicators from Large Climate Datasets

AU - DAnca, Alessandro

AU - Palazzo, Cosimo

AU - Elia, Donatello

AU - Fiore, Sandro

AU - Bistinas, Ioannis

AU - Bottcher, Kristin

AU - Bennett, Victoria

AU - Aloisio, Giovanni

PY - 2017/7/10

Y1 - 2017/7/10

N2 - The need to apply complex algorithms on large volumes of data is boosting the development of technological solutions able to satisfy big data analytics needs in Cloud and HPC environments. In this context Ophidia represents a big data analytics framework for eScience offering a cross-domain solution for managing scientific, multi-dimensional data. It also exploits an in-memory-based distributed data storage and provides support for the submission of complex workflows by means of various interfaces compliant to well-known standards. This paper presents some applications of Ophidia for the computation of climate indicators defined in the CLIPC project, the WPS interface used for the submission and the workflow based approach employed.

AB - The need to apply complex algorithms on large volumes of data is boosting the development of technological solutions able to satisfy big data analytics needs in Cloud and HPC environments. In this context Ophidia represents a big data analytics framework for eScience offering a cross-domain solution for managing scientific, multi-dimensional data. It also exploits an in-memory-based distributed data storage and provides support for the submission of complex workflows by means of various interfaces compliant to well-known standards. This paper presents some applications of Ophidia for the computation of climate indicators defined in the CLIPC project, the WPS interface used for the submission and the workflow based approach employed.

KW - Big data analytics

KW - Climate impact indicator

KW - Performance evaluation

KW - Scientific workflows

U2 - 10.1109/CCGRID.2017.132

DO - 10.1109/CCGRID.2017.132

M3 - Chapter

SN - 9781509066100

T3 - Proceedings - 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017

SP - 1035

EP - 1043

BT - Proceedings - 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017

PB - Institute of Electrical and Electronics Engineers, Inc.

ER -

DAnca A, Palazzo C, Elia D, Fiore S, Bistinas I, Bottcher K et al. On the Use of In-memory Analytics Workflows to Compute eScience Indicators from Large Climate Datasets. In Proceedings - 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017. Institute of Electrical and Electronics Engineers, Inc. 2017. p. 1035-1043. (Proceedings - 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017). https://doi.org/10.1109/CCGRID.2017.132