User Transparent Data and Task Parallel Multimedia Computing with Pyxis-DT

T.P. van Kessel, N. Drost, J. Maassen, H.E. Bal, F.J. Seinstra

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

The research area of Multimedia Content Analysis (MMCA) considers all aspects of the automated extraction of knowledge from multimedia archives and data streams. To satisfy the increasing computational demands of emerging MMCA problems, there is an urgent need to apply High Performance Computing (HPC) techniques. However, as most MMCA researchers are not also HPC experts, in the field there is a demand~for~programming models and tools that are both efficient and easy~to~use. Today several user transparent library-based parallelization tools exist that aim to satisfy both these requirements. Such tools generally use a data parallel approach in which data structures (e.g. video frames) are scattered among the available nodes in a compute cluster. However, for certain MMCA applications a data parallel approach induces intensive communication, which significantly decreases performance. In these situations, we can benefit from applying alternative approaches. This paper presents Pyxis-DT: a user transparent parallel programming model for MMCA applications that employs both data and task parallelism. Hybrid parallel execution is obtained by run-time construction and execution of a task graph consisting of strictly defined building block operations. Each of these building block operations can be executed in data parallel fashion. Results show that for realistic MMCA applications the concurrent use of data and task parallelism can significantly improve performance compared to using either approach in isolation.
Original languageEnglish
Title of host publicationProceedings of the 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2012)
PublisherIEEE Computer Society
Pages17-24
Number of pages8
Publication statusPublished - 13 May 2012

Fingerprint

Parallel programming
Data structures
Communication

Cite this

van Kessel, T. P., Drost, N., Maassen, J., Bal, H. E., & Seinstra, F. J. (2012). User Transparent Data and Task Parallel Multimedia Computing with Pyxis-DT. In Proceedings of the 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2012) (pp. 17-24). IEEE Computer Society.
van Kessel, T.P. ; Drost, N. ; Maassen, J. ; Bal, H.E. ; Seinstra, F.J. / User Transparent Data and Task Parallel Multimedia Computing with Pyxis-DT. Proceedings of the 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2012). IEEE Computer Society, 2012. pp. 17-24
@inproceedings{18f1f2a0a9bc46588d232592f3cdf8b7,
title = "User Transparent Data and Task Parallel Multimedia Computing with Pyxis-DT",
abstract = "The research area of Multimedia Content Analysis (MMCA) considers all aspects of the automated extraction of knowledge from multimedia archives and data streams. To satisfy the increasing computational demands of emerging MMCA problems, there is an urgent need to apply High Performance Computing (HPC) techniques. However, as most MMCA researchers are not also HPC experts, in the field there is a demand~for~programming models and tools that are both efficient and easy~to~use. Today several user transparent library-based parallelization tools exist that aim to satisfy both these requirements. Such tools generally use a data parallel approach in which data structures (e.g. video frames) are scattered among the available nodes in a compute cluster. However, for certain MMCA applications a data parallel approach induces intensive communication, which significantly decreases performance. In these situations, we can benefit from applying alternative approaches. This paper presents Pyxis-DT: a user transparent parallel programming model for MMCA applications that employs both data and task parallelism. Hybrid parallel execution is obtained by run-time construction and execution of a task graph consisting of strictly defined building block operations. Each of these building block operations can be executed in data parallel fashion. Results show that for realistic MMCA applications the concurrent use of data and task parallelism can significantly improve performance compared to using either approach in isolation.",
author = "{van Kessel}, T.P. and N. Drost and J. Maassen and H.E. Bal and F.J. Seinstra",
year = "2012",
month = "5",
day = "13",
language = "English",
pages = "17--24",
booktitle = "Proceedings of the 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2012)",
publisher = "IEEE Computer Society",

}

van Kessel, TP, Drost, N, Maassen, J, Bal, HE & Seinstra, FJ 2012, User Transparent Data and Task Parallel Multimedia Computing with Pyxis-DT. in Proceedings of the 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2012). IEEE Computer Society, pp. 17-24.

User Transparent Data and Task Parallel Multimedia Computing with Pyxis-DT. / van Kessel, T.P.; Drost, N.; Maassen, J.; Bal, H.E.; Seinstra, F.J.

Proceedings of the 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2012). IEEE Computer Society, 2012. p. 17-24.

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - User Transparent Data and Task Parallel Multimedia Computing with Pyxis-DT

AU - van Kessel, T.P.

AU - Drost, N.

AU - Maassen, J.

AU - Bal, H.E.

AU - Seinstra, F.J.

PY - 2012/5/13

Y1 - 2012/5/13

N2 - The research area of Multimedia Content Analysis (MMCA) considers all aspects of the automated extraction of knowledge from multimedia archives and data streams. To satisfy the increasing computational demands of emerging MMCA problems, there is an urgent need to apply High Performance Computing (HPC) techniques. However, as most MMCA researchers are not also HPC experts, in the field there is a demand~for~programming models and tools that are both efficient and easy~to~use. Today several user transparent library-based parallelization tools exist that aim to satisfy both these requirements. Such tools generally use a data parallel approach in which data structures (e.g. video frames) are scattered among the available nodes in a compute cluster. However, for certain MMCA applications a data parallel approach induces intensive communication, which significantly decreases performance. In these situations, we can benefit from applying alternative approaches. This paper presents Pyxis-DT: a user transparent parallel programming model for MMCA applications that employs both data and task parallelism. Hybrid parallel execution is obtained by run-time construction and execution of a task graph consisting of strictly defined building block operations. Each of these building block operations can be executed in data parallel fashion. Results show that for realistic MMCA applications the concurrent use of data and task parallelism can significantly improve performance compared to using either approach in isolation.

AB - The research area of Multimedia Content Analysis (MMCA) considers all aspects of the automated extraction of knowledge from multimedia archives and data streams. To satisfy the increasing computational demands of emerging MMCA problems, there is an urgent need to apply High Performance Computing (HPC) techniques. However, as most MMCA researchers are not also HPC experts, in the field there is a demand~for~programming models and tools that are both efficient and easy~to~use. Today several user transparent library-based parallelization tools exist that aim to satisfy both these requirements. Such tools generally use a data parallel approach in which data structures (e.g. video frames) are scattered among the available nodes in a compute cluster. However, for certain MMCA applications a data parallel approach induces intensive communication, which significantly decreases performance. In these situations, we can benefit from applying alternative approaches. This paper presents Pyxis-DT: a user transparent parallel programming model for MMCA applications that employs both data and task parallelism. Hybrid parallel execution is obtained by run-time construction and execution of a task graph consisting of strictly defined building block operations. Each of these building block operations can be executed in data parallel fashion. Results show that for realistic MMCA applications the concurrent use of data and task parallelism can significantly improve performance compared to using either approach in isolation.

M3 - Conference contribution

SP - 17

EP - 24

BT - Proceedings of the 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2012)

PB - IEEE Computer Society

ER -

van Kessel TP, Drost N, Maassen J, Bal HE, Seinstra FJ. User Transparent Data and Task Parallel Multimedia Computing with Pyxis-DT. In Proceedings of the 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2012). IEEE Computer Society. 2012. p. 17-24