TypeSan: Practical type confusion detection

Istvan Haller, Yuseok Jeon, Hui Peng, Mathias Payer, Cristiano Giuffrida, Herbert Bos, Erik Van Der Kouwe

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

The low-level C++ programming language is ubiquitously used for its modularity and performance. Typecasting is a fundamental concept in C++ (and object-oriented programming in general) to convert a pointer from one object type into another. However, downcasting (converting a base class pointer to a derived class pointer) has critical security implications due to potentially different object memory layouts. Due to missing type safety in C++, a downcasted pointer can violate a programmer's intended pointer semantics, allowing an attacker to corrupt the underlying memory in a type-unsafe fashion. This vulnerability class is receiving increasing attention and is known as type confusion (or badcasting). Several existing approaches detect different forms of type confusion, but these solutions are severely limited due to both high run-time performance overhead and low detection coverage. This paper presents TypeSan, a practical type-confusion detector which provides both low run-time overhead and high detection coverage. Despite improving the coverage of state-of-the-art techniques, TypeSan significantly reduces the type-confusion detection overhead compared to other solutions. TypeSan relies on an efficient per-object metadata storage service based on a compact memory shadowing scheme. Our scheme treats all the memory objects (i.e., globals, stack, heap) uniformly to eliminate extra checks on the fast path and relies on a variable compression ratio to minimize run-time performance and memory overhead. Our experimental results confirm that TypeSan is practical, even when explicitly checking almost all the relevant typecasts in a given C++ program. Compared to the state of the art, TypeSan yields orders of magnitude higher coverage at 4-10 times lower performance overhead on SPEC and 2 times on Firefox. As a result, our solution offers superior protection and is suitable for deployment in production software. Moreover, our highly efficient metadata storage back-end is potentially useful for other defenses that require memory object tracking.

Original languageEnglish
Title of host publicationCCS 2016 - Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security
PublisherAssociation for Computing Machinery (ACM)
Pages517-528
Number of pages12
Volume24-28-October-2016
ISBN (Electronic)9781450341394
DOIs
Publication statusPublished - 24 Oct 2016
Event23rd ACM Conference on Computer and Communications Security, CCS 2016 - Vienna, Austria
Duration: 24 Oct 201628 Oct 2016

Conference

Conference23rd ACM Conference on Computer and Communications Security, CCS 2016
CountryAustria
CityVienna
Period24/10/1628/10/16

Keywords

  • Downcasting
  • Type confusion
  • Type safety
  • Typecasting

Fingerprint Dive into the research topics of 'TypeSan: Practical type confusion detection'. Together they form a unique fingerprint.

Cite this