Enviral: Fuzzing the Environment for Evasive Malware Analysis

Research output: Chapter in Book / Report / Conference proceedingConference contributionAcademicpeer-review

Abstract

Analyzing malicious behavior is vital to effectively safeguard computer systems against malware. However, contemporary malware frequently contains evasive behavior, which allows it to hide its malicious intent from analysis. More specifically, if the malware detects it is being executed in an analysis environment, it resorts to evasive routines that exhibit benign behavior. Manually deactivating evasive checks requires significant effort, and is therefore not a scalable technique with regards to the increasing amount of evasive malware. Unfortunately, the existing systems that automatically analyze evasive malware are impractical, computationally inefficient, or incomplete by design. In this paper, we introduce Enviral, an automatic evasive malware analysis framework that proposes a novel method to analyze evasive malware, combining the best elements of existing approaches. We achieve this by applying fuzzing techniques to repeatedly adapt the view of the execution environment, thereby iteratively defeating the evasive checks in the target application. We realize these adaptations by applying mutations to the outcomes of environment queries, which in turn leads to the exploration of multiple execution paths. Our experimental results demonstrate that Enviral can detect and overcome evasive behavior and thereby exposes previously hidden activity in malware. We evaluate our system against a similar framework, and conclude that Enviral can expose 39% more interesting hidden system call activity on average, and achieves productive explorations where previously unseen behavior is discovered in 67% more malware samples.

Original languageEnglish
Title of host publicationEUROSEC 2023
Subtitle of host publicationProceedings of the 2023 European Workshop on System Security
PublisherAssociation for Computing Machinery, Inc
Pages8-14
Number of pages7
ISBN (Electronic)9798400700859
DOIs
Publication statusPublished - May 2023
Event16th European Workshop on Systems Security, EUROSEC 2023 - Rome, Italy
Duration: 8 May 2023 → …

Conference

Conference16th European Workshop on Systems Security, EUROSEC 2023
Country/TerritoryItaly
CityRome
Period8/05/23 → …

Bibliographical note

Publisher Copyright:
© 2023 Copyright held by the owner/author(s).

Keywords

  • evasive malware analysis
  • fuzzing
  • system call hooks

Fingerprint

Dive into the research topics of 'Enviral: Fuzzing the Environment for Evasive Malware Analysis'. Together they form a unique fingerprint.

Cite this