User transparent data and task parallel multimedia computing with Pyxis-DT

T.P. van Kessel, B. van Werkhoven, N. Drost, J. Maassen, H.E. Bal, F.J. Seinstra

Research output: Contribution to JournalArticleAcademicpeer-review

Abstract

The research area of Multimedia Content Analysis (MMCA) considers all aspects of the automated extraction of knowledge from multimedia archives and data streams. To satisfy the increasing computational demands of emerging MMCA problems, there is an urgent need to apply High Performance Computing (HPC) techniques. As most MMCA researchers are not also HPC experts, however, there is a demand for programming models and tools that are both efficient and easy to use. Existing user transparent parallelization tools generally use a data parallel approach in which data structures (e.g. video frames) are scattered among the available nodes in a compute cluster. For certain MMCA applications a data parallel approach induces intensive communication, however, which significantly decreases performance. In these situations, we can benefit from applying alternative approaches. We present Pyxis-DT, a user transparent parallel programming model for MMCA applications that employs both data and task parallelism. Hybrid parallel execution is obtained by run-time construction and execution of a task graph consisting of strictly defined building block operations. Results show that for realistic MMCA applications the concurrent use of data and task parallelism can significantly improve performance compared to using either approach in isolation. Extensions for GPU clusters are also presented. © 2013 Elsevier B.V. All rights reserved.
Original languageEnglish
Pages (from-to)2252-2261
Number of pages10
JournalFuture Generation Computer Systems
Volume29
DOIs
Publication statusPublished - 2013

Fingerprint

Parallel programming
Data structures
Communication
Graphics processing unit

Cite this

van Kessel, T.P. ; van Werkhoven, B. ; Drost, N. ; Maassen, J. ; Bal, H.E. ; Seinstra, F.J. / User transparent data and task parallel multimedia computing with Pyxis-DT. In: Future Generation Computer Systems. 2013 ; Vol. 29. pp. 2252-2261.
@article{9bfe54bd2a6343efb5ec6647364cb7b6,
title = "User transparent data and task parallel multimedia computing with Pyxis-DT",
abstract = "The research area of Multimedia Content Analysis (MMCA) considers all aspects of the automated extraction of knowledge from multimedia archives and data streams. To satisfy the increasing computational demands of emerging MMCA problems, there is an urgent need to apply High Performance Computing (HPC) techniques. As most MMCA researchers are not also HPC experts, however, there is a demand for programming models and tools that are both efficient and easy to use. Existing user transparent parallelization tools generally use a data parallel approach in which data structures (e.g. video frames) are scattered among the available nodes in a compute cluster. For certain MMCA applications a data parallel approach induces intensive communication, however, which significantly decreases performance. In these situations, we can benefit from applying alternative approaches. We present Pyxis-DT, a user transparent parallel programming model for MMCA applications that employs both data and task parallelism. Hybrid parallel execution is obtained by run-time construction and execution of a task graph consisting of strictly defined building block operations. Results show that for realistic MMCA applications the concurrent use of data and task parallelism can significantly improve performance compared to using either approach in isolation. Extensions for GPU clusters are also presented. {\circledC} 2013 Elsevier B.V. All rights reserved.",
author = "{van Kessel}, T.P. and {van Werkhoven}, B. and N. Drost and J. Maassen and H.E. Bal and F.J. Seinstra",
year = "2013",
doi = "10.1016/j.future.2013.06.005",
language = "English",
volume = "29",
pages = "2252--2261",
journal = "Future Generation Computer Systems",
issn = "0167-739X",
publisher = "Elsevier",

}

User transparent data and task parallel multimedia computing with Pyxis-DT. / van Kessel, T.P.; van Werkhoven, B.; Drost, N.; Maassen, J.; Bal, H.E.; Seinstra, F.J.

In: Future Generation Computer Systems, Vol. 29, 2013, p. 2252-2261.

Research output: Contribution to JournalArticleAcademicpeer-review

TY - JOUR

T1 - User transparent data and task parallel multimedia computing with Pyxis-DT

AU - van Kessel, T.P.

AU - van Werkhoven, B.

AU - Drost, N.

AU - Maassen, J.

AU - Bal, H.E.

AU - Seinstra, F.J.

PY - 2013

Y1 - 2013

N2 - The research area of Multimedia Content Analysis (MMCA) considers all aspects of the automated extraction of knowledge from multimedia archives and data streams. To satisfy the increasing computational demands of emerging MMCA problems, there is an urgent need to apply High Performance Computing (HPC) techniques. As most MMCA researchers are not also HPC experts, however, there is a demand for programming models and tools that are both efficient and easy to use. Existing user transparent parallelization tools generally use a data parallel approach in which data structures (e.g. video frames) are scattered among the available nodes in a compute cluster. For certain MMCA applications a data parallel approach induces intensive communication, however, which significantly decreases performance. In these situations, we can benefit from applying alternative approaches. We present Pyxis-DT, a user transparent parallel programming model for MMCA applications that employs both data and task parallelism. Hybrid parallel execution is obtained by run-time construction and execution of a task graph consisting of strictly defined building block operations. Results show that for realistic MMCA applications the concurrent use of data and task parallelism can significantly improve performance compared to using either approach in isolation. Extensions for GPU clusters are also presented. © 2013 Elsevier B.V. All rights reserved.

AB - The research area of Multimedia Content Analysis (MMCA) considers all aspects of the automated extraction of knowledge from multimedia archives and data streams. To satisfy the increasing computational demands of emerging MMCA problems, there is an urgent need to apply High Performance Computing (HPC) techniques. As most MMCA researchers are not also HPC experts, however, there is a demand for programming models and tools that are both efficient and easy to use. Existing user transparent parallelization tools generally use a data parallel approach in which data structures (e.g. video frames) are scattered among the available nodes in a compute cluster. For certain MMCA applications a data parallel approach induces intensive communication, however, which significantly decreases performance. In these situations, we can benefit from applying alternative approaches. We present Pyxis-DT, a user transparent parallel programming model for MMCA applications that employs both data and task parallelism. Hybrid parallel execution is obtained by run-time construction and execution of a task graph consisting of strictly defined building block operations. Results show that for realistic MMCA applications the concurrent use of data and task parallelism can significantly improve performance compared to using either approach in isolation. Extensions for GPU clusters are also presented. © 2013 Elsevier B.V. All rights reserved.

U2 - 10.1016/j.future.2013.06.005

DO - 10.1016/j.future.2013.06.005

M3 - Article

VL - 29

SP - 2252

EP - 2261

JO - Future Generation Computer Systems

JF - Future Generation Computer Systems

SN - 0167-739X

ER -