PipesFS: Fast Linux I/O in the Unix Tradition

W.J. de Bruijn, H.J. Bos

Research output: Contribution to JournalArticleAcademicpeer-review

Abstract

This paper presents PipesFS, an I/O architecture for Linux 2.6 that increases I/O throughput and adds support for heterogeneous parallel processors by (1) collapsing many I/O interfaces onto one: the Unix pipeline, (2) increasing pipe efficiency and (3) exploiting pipeline modularity to spread computation across all available processors. PipesFS extends the pipeline model to kernel I/O and communicates with applications through a Linux virtual filesystem (VFS), where directory nodes represent operations and pipe nodes export live kernel data. Users can thus interact with kernel I/O through existing calls like mkdir, tools like grep, most languages and even shell scripts. To support performance critical tasks, PipesFS improves pipe throughput through copy, context switch and cache miss avoidance. To integrate heterogeneous processors (e.g., the Cell) it transparently moves operations to the most efficient type of core.
LanguageEnglish
Pages55-63
JournalACM SIGOPS Operating Systems Review
Volume42
Issue number5
DOIs
StatePublished - 2008

Fingerprint

Pipelines
Pipe
Throughput
Switches
Linux

Cite this

@article{a7ad2cb737dc4520ba08b3dd36f74c0f,
title = "PipesFS: Fast Linux I/O in the Unix Tradition",
abstract = "This paper presents PipesFS, an I/O architecture for Linux 2.6 that increases I/O throughput and adds support for heterogeneous parallel processors by (1) collapsing many I/O interfaces onto one: the Unix pipeline, (2) increasing pipe efficiency and (3) exploiting pipeline modularity to spread computation across all available processors. PipesFS extends the pipeline model to kernel I/O and communicates with applications through a Linux virtual filesystem (VFS), where directory nodes represent operations and pipe nodes export live kernel data. Users can thus interact with kernel I/O through existing calls like mkdir, tools like grep, most languages and even shell scripts. To support performance critical tasks, PipesFS improves pipe throughput through copy, context switch and cache miss avoidance. To integrate heterogeneous processors (e.g., the Cell) it transparently moves operations to the most efficient type of core.",
author = "{de Bruijn}, W.J. and H.J. Bos",
note = "pipesfs:osr08",
year = "2008",
doi = "10.1145/1400097.1400104",
language = "English",
volume = "42",
pages = "55--63",
journal = "ACM SIGOPS Operating Systems Review",
issn = "0163-5980",
publisher = "Association for Computing Machinery (ACM)",
number = "5",

}

PipesFS: Fast Linux I/O in the Unix Tradition. / de Bruijn, W.J.; Bos, H.J.

In: ACM SIGOPS Operating Systems Review, Vol. 42, No. 5, 2008, p. 55-63.

Research output: Contribution to JournalArticleAcademicpeer-review

TY - JOUR

T1 - PipesFS: Fast Linux I/O in the Unix Tradition

AU - de Bruijn,W.J.

AU - Bos,H.J.

N1 - pipesfs:osr08

PY - 2008

Y1 - 2008

N2 - This paper presents PipesFS, an I/O architecture for Linux 2.6 that increases I/O throughput and adds support for heterogeneous parallel processors by (1) collapsing many I/O interfaces onto one: the Unix pipeline, (2) increasing pipe efficiency and (3) exploiting pipeline modularity to spread computation across all available processors. PipesFS extends the pipeline model to kernel I/O and communicates with applications through a Linux virtual filesystem (VFS), where directory nodes represent operations and pipe nodes export live kernel data. Users can thus interact with kernel I/O through existing calls like mkdir, tools like grep, most languages and even shell scripts. To support performance critical tasks, PipesFS improves pipe throughput through copy, context switch and cache miss avoidance. To integrate heterogeneous processors (e.g., the Cell) it transparently moves operations to the most efficient type of core.

AB - This paper presents PipesFS, an I/O architecture for Linux 2.6 that increases I/O throughput and adds support for heterogeneous parallel processors by (1) collapsing many I/O interfaces onto one: the Unix pipeline, (2) increasing pipe efficiency and (3) exploiting pipeline modularity to spread computation across all available processors. PipesFS extends the pipeline model to kernel I/O and communicates with applications through a Linux virtual filesystem (VFS), where directory nodes represent operations and pipe nodes export live kernel data. Users can thus interact with kernel I/O through existing calls like mkdir, tools like grep, most languages and even shell scripts. To support performance critical tasks, PipesFS improves pipe throughput through copy, context switch and cache miss avoidance. To integrate heterogeneous processors (e.g., the Cell) it transparently moves operations to the most efficient type of core.

U2 - 10.1145/1400097.1400104

DO - 10.1145/1400097.1400104

M3 - Article

VL - 42

SP - 55

EP - 63

JO - ACM SIGOPS Operating Systems Review

T2 - ACM SIGOPS Operating Systems Review

JF - ACM SIGOPS Operating Systems Review

SN - 0163-5980

IS - 5

ER -