Enterprise Organization of National Program of Excellence in Software

The paper titled "Fool Me If You Can: On the Robustness of Binary Code Similarity Detection Models against Semantics-preserving Transformations," co-authored by Ji-yong Eom (Ph.D. candidate), Min-seok Kim (M.S. candidate), both from the SecAI Lab (supervised by Hyungjoon Koo, https://secai.skku.edu/), and Michalis Polychronakis from Stony Brook University, has been accepted for publication at the prestigious Foundations of Software Engineering 2026 (FSE '26). The paper will be presented in July 2026.

Software reverse engineering is a critical process in the security field, including vulnerability analysis and malware detection, but it requires a high level of expertise. However, relying solely on such methods presents limitations in effectively addressing the rapidly increasing modern threats. To overcome this challenge, recent approaches have actively proposed techniques to assist reverse engineering with artificial intelligence, especially models that extract contextual information from machine code (assembly language). Similar to how natural language can convey meaning through context-preserving transformations, assembly language also has techniques for transforming code while maintaining the same semantics (semantics-preserving code transformations). However, there has been a lack of in-depth analysis on how robust artificial intelligence models are against these types of transformations. This study systematically analyzes the impact of eight transformation techniques on the performance of six representative AI-based binary similarity detection models. It also introduces how models can lead to incorrect judgments, such as false positives and false negatives. For this, a dataset consisting of 9,565 transformed binaries from 620 original binaries was built for experimentation. The results show that the robustness to transformations varies based on the architecture and preprocessing methods of the models, and that even slight transformations can significantly degrade model performance, especially if the attacker designs the transformation precisely. This research emphasizes that, when designing AI models for supporting reverse engineering, model robustness against binary transformations should be considered as crucial as performance metrics.

Abstract: Binary code analysis plays an essential role in cybersecurity, facilitating reverse engineering to reveal the inner workings of programs in the absence of source code. Traditional approaches, such as static and dynamic analysis, extract valuable insights from stripped binaries, but often demand substantial expertise and manual effort. Recent advances in deep learning have opened promising opportunities to enhance binary analysis by capturing latent features and disclosing underlying code semantics. Despite the growing number of binary analysis models based on machine learning, their robustness to adversarial code transformations at the binary level remains underexplored to date. In this work, we evaluate the robustness of deep learning models for the task of binary code similarity detection (BCSD) under semantics-preserving transformations. The unique nature of machine instructions presents distinct challenges compared to the typical input perturbations found in other domains. To achieve our goal, we introduce asmFooler, a system that evaluates the resilience of BCSD models using a diverse set of adversarial code transformations that preserve functional semantics. We construct a dataset of 9,565 binary variants from 620 baseline samples by applying eight semantics-preserving transformations across six representative BCSD models. Our major findings highlight several key insights: i) model robustness highly relies on the design of the processing pipeline, including code pre-processing, model architecture, and internal feature selection, which collectively determine how code semantics are captured; ii) the effectiveness of adversarial transformations is bounded by a transformation budget, shaped by model-specific constraints such as input size limits and the expressive capacity of semantically equivalent instructions; iii) well-crafted adversarial transformations can be highly effective, even when introducing minimal perturbations; and iv) such transformations efficiently disrupt the model's decision (e.g., misleading to false positives or false negatives) by focusing on semantically significant instructions.

| Professor Hyungjoon Koo | kevin.koo@skku.edu, kevinkoo001.github.io

| SecAI Lab | secai.skku.edu