FastLanes: A Next-Gen File Format

This thesis presents the design and implementation of FastLanes, a next-generation file format for OLAP workloads on modern CPUs and GPUs. It redesigns lightweight encodings to be more data-parallel, fully exploiting SIMD and GPU parallelism, achieving decoding speeds of over 100 billion integers per second in scalar execution. It also introduces ALP (Adaptive Lossless Floating-Point Compression)—a novel, adaptive, and SIMD-friendly floating-point compressor that surpasses state-of-the-art methods such as ZSTD. GPU extensions of FastLanes adapt these techniques to thread-level parallelism, addressing shared-memory and warp-divergence bottlenecks to accelerate analytical workloads on GPU-based engines such as Crystal. The dissertation integrates these innovative encodings into a fully functional file format, also called FastLanes, which combines multiple lightweight codecs through Expression Encoding—a composable representation that merges encoding strategies (e.g., FOR, RLE, DICT) to achieve compression ratios comparable to heavyweight compressors while maintaining exceptional decompression speed. Furthermore, FastLanes is released as open-source software, ensuring reproducibility and enabling future research in hardware-optimized data storage and analytics.