Abstract
Analyzing malicious behavior is vital to effectively safeguard computer systems against malware. However, contemporary malware frequently contains evasive behavior, which allows it to hide its malicious intent from analysis. More specifically, if the malware detects it is being executed in an analysis environment, it resorts to evasive routines that exhibit benign behavior. Manually deactivating evasive checks requires significant effort, and is therefore not a scalable technique with regards to the increasing amount of evasive malware. Unfortunately, the existing systems that automatically analyze evasive malware are impractical, computationally inefficient, or incomplete by design. In this paper, we introduce Enviral, an automatic evasive malware analysis framework that proposes a novel method to analyze evasive malware, combining the best elements of existing approaches. We achieve this by applying fuzzing techniques to repeatedly adapt the view of the execution environment, thereby iteratively defeating the evasive checks in the target application. We realize these adaptations by applying mutations to the outcomes of environment queries, which in turn leads to the exploration of multiple execution paths. Our experimental results demonstrate that Enviral can detect and overcome evasive behavior and thereby exposes previously hidden activity in malware. We evaluate our system against a similar framework, and conclude that Enviral can expose 39% more interesting hidden system call activity on average, and achieves productive explorations where previously unseen behavior is discovered in 67% more malware samples.
Original language | English |
---|---|
Title of host publication | EUROSEC 2023 |
Subtitle of host publication | Proceedings of the 2023 European Workshop on System Security |
Publisher | Association for Computing Machinery, Inc |
Pages | 8-14 |
Number of pages | 7 |
ISBN (Electronic) | 9798400700859 |
DOIs | |
Publication status | Published - May 2023 |
Event | 16th European Workshop on Systems Security, EUROSEC 2023 - Rome, Italy Duration: 8 May 2023 → … |
Conference
Conference | 16th European Workshop on Systems Security, EUROSEC 2023 |
---|---|
Country/Territory | Italy |
City | Rome |
Period | 8/05/23 → … |
Bibliographical note
Publisher Copyright:© 2023 Copyright held by the owner/author(s).
Keywords
- evasive malware analysis
- fuzzing
- system call hooks