Yan Lin 0003

dblp:27/586-3 · DBLP profile ↗
← Back
18ranked-venue papers
7as first author
10since 2021 · last 2026
0000-0002-6509-9131ORCID · conflict

Domains — the database's venue-derived domains; a paper can count in several

Security and privacy · 16 · 6 first-author · 8 since 2021Software engineering, systems software and programming languages · 2 · 1 first-author · 2 since 2021
YearPublicationVenuePosition
2026 MalElves: Reinforcement Learning-Driven Adversarial Example Generation for Evading Cross-Platform ELF Malware Detection
abstract
Adversarial Example (AE) generation is a key instrument for stress-testing and hardening malware detectors, yet most existing techniques target Portable Executable (PE) files and do not transfer cleanly to Executable and Linkable Format (ELF) binaries prevalent in Internet of Things (IoT) environments. We proposeMalElves, a reinforcement learning-driven AE generation framework for cross-platform ELF malware.MalElvesmakes three core technical contributions. First, a code-data-aware manipulation framework unifies obfuscation and rewriting across ARM, ×86, and ×64 architectures while preserving functionality. Second, a sample-efficient state design reduces 2,350 raw ELF features to a compact 21-dimensional input. Third, a shaped multi-detector reward uses fully disclosed PPO settings for full reproducibility. We evaluateMalElveson 161,414 malware samples and 74,260 benign samples. We test against four static detectors and a behavioral-ensemble stress test. The method achieves average ASRs of 89.9%, 85.3%, 63.8%, 60.6%, and 24.7% across detectors. The overall average ASR reaches 64.8%. Each successful evasion requires 2.24 interaction rounds on average.
Zhangbo Long, Letian Sha, Yan Lin 0003, Peijie Sun, Haiping Huang, Fu Xiao 0001, Zhiquan Liu 0001
IEEE Trans. Software Eng.3
2025 Beyond Tag Collision: Cluster-based Memory Management for Tag-based Sanitizers
Mengfei Xie, Yan Lin 0003, Jianming Fu, Chenke Luo, Guojun Peng
CCS2
2025 PatchFuzz: An Efficient Way to Incorporate Patching With Hybrid Fuzzing
abstract
Hybrid and patching-based fuzzing methods offer promise in uncovering software bugs using concolic execution and program transformation techniques. However, current implementations face efficiency challenges from three main factors. First, the efficacy of hybrid fuzzing can be compromised due to disruptions introduced during the mutation of inputs generated by concolic execution, hindering vulnerability discovery. Second, the speed and throughput of the underlying fuzzer significantly impact the effectiveness of both methods. Third, patching-based fuzzing has an inefficient patching system and high analysis costs. Nonetheless, hybrid and patching-based fuzzing offer complementary strengths that can enhance overall efficacy. For instance, patching can maintain exploration depth by reducing the likelihood of input structure disruption, while concolic execution can authenticate detected crashes. In this paper, we present PATCHFUZZ, which integrates fuzzing and patching at a fundamental level, leveraging concolic execution to augment the fuzzing process. Specifically, PATCHFUZZ binds patching addresses with each seed rather than the program itself, promoting frequent interactions between patching and fuzzing. Additionally, we've devised efficient methodologies for patching information management and patched program switching. Evaluation on LAVA-M, CGC and OSS-Fuzz datasets shows PATCHFUZZ surpasses state-of-the-art fuzzers like QSYM, SYMQEMU, AFL++ CmpLog and T-Fuzz. Deployed on industrial devices, PATCHFUZZ uncovered 9 new vulnerabilities.
Letian Sha, Luheng Zhang, Yan Lin 0003, Fu Xiao 0001, Jiaye Pan
IEEE Trans. Dependable Secur. Comput.4
2025 Egalitarian Randomization for Multi-Language Applications on ARM64
abstract
Due to the inevitable information loss during IR lowering, compile-time metadata collection can provide more precise auxiliary information than binary analysis to achieve reliable fine-grained randomization. However, existing schemes build on deep modifications of compilers, making it challenging to provide consistent randomization protection for different high-level languages. Additionally, they are inadequate for securing widely used smartphones and embedded devices, since only ×86-64 applications are currently supported. In this paper, we present MLARandom, a compiler-assisted function-level randomization scheme designed for Multi-Language ARM64 applications. MLARandom employs a lightweight compilation standardization strategy that allows for uniform information collection at the assembly level, regardless of the high-level language or compiler used. Further, it combines ARM64 architecture specifications and collected relocation types to accurately repair all ARM64 pointers after randomization. Our experimental results show that MLARandom can equally randomize modules developed in different languages (e.g., C/C++, Rust, Fortran, Cangjie) with negligible runtime overhead (0.51%), to effectively counter against traditional Code Reuse Attacks as well as advanced Cross-Language Attacks. Although randomization approaches based on reassembly can achieve similar goals, our empirical evaluation highlights the imprecise pointer identification as a major obstacle to their practical deployment.
Mengfei Xie, Yan Lin 0003, Jianming Fu, Chenke Luo, Guojun Peng
IEEE Trans. Dependable Secur. Comput.2
2024 Peep With A Mirror: Breaking The Integrity of Android App Sandboxing via Unprivileged Cache Side Channel
Yan Lin 0003, Joshua Wong, Debin Gao
USENIX Security Symposium1
2024 Analyzing and revivifying function signature inference using deep learning
Yan Lin 0003, Trisha Singhal, Debin Gao, David Lo 0001
Empir. Softw. Eng.1
2023 BinAlign: Alignment Padding Based Compiler Provenance Recovery
Maliha Ismail, Yan Lin 0003, DongGyun Han, Debin Gao
ACISP2
2023 PointerScope: Understanding Pointer Patching for Code Randomization
abstract
Various fine-grained randomization schemes have been designed to increase the entropy of process space, while none of them can rise from an academic exercise to industrial deployment like Address Space Layout Randomization (ASLR). One of the critical reasons is the incorrectness of randomization caused by the mismatch between their pointer collection capabilities and the high accuracy requirements of the pointer patching task. In this article, we present PointerScope, an accurate compile-time pointer collection scheme deriving from a group of novel observations. The success of PointerScope relies on the complete tracing of the pointer generation process, including the compilation chain from compiler to static linker and the interface specification between them. From this view, PointerScope identifies four types of pointer-related static linker behaviors and clarifies five types of inherent addressing modes in the x86-64 architecture. The vague understanding of them causes the Compiler-assisted Code Randomization (CCR) to incorrectly collect pointers and patch them to the wrong values after randomization. Further, we measure the pointer collection capability of augmented binary analysis, the experimental results show that they can mitigate challenges from the traditional binary analysis by the given premises, but additional heuristics still need to be designed to support the fine-grained randomization.
Mengfei Xie, Yan Lin 0003, Chenke Luo, Guojun Peng, Jianming Fu
IEEE Trans. Dependable Secur. Comput.2
2022 ReSIL: Revivifying Function Signature Inference using Deep Learning with Domain-Specific Knowledge
abstract
Function signature recovery is important for binary analysis and security enhancement, such as bug finding and control-flow integrity enforcement. However, binary executables typically have crucial information vital for function signature recovery stripped off during compilation. To make things worse, recent studies show that many compiler optimization strategies further complicate the recovery of function signatures with intended violations to function calling conventions.
Yan Lin 0003, Debin Gao, David Lo 0001
CODASPY1
2021 When Function Signature Recovery Meets Compiler Optimization
abstract
Matching indirect function callees and callers using function signatures recovered from binary executables (number of arguments and argument types) has been proposed to construct a more fine-grained control-flow graph (CFG) to help control-flow integrity (CFI) enforcement. However, various compiler optimizations may violate calling conventions and result in unmatched function signatures. In this paper, we present eight scenarios in which compiler optimizations impact function signature recovery, and report experimental results with 1,344 real-world applications of various optimization levels. Most interestingly, our experiments show that compiler optimizations have both positive and negative impacts on function signature recovery, e.g., its elimination of redundant instructions at callers makes counting of the number of arguments more accurate, while it hurts argument type matching as the compiler chooses the most efficient (but potentially different) types at callees and callers. To better deal with these compiler optimizations, we propose a set of improved policies and report our more accurate CFG models constructed from the 1,344 applications. We additionally compare our results recovered from binary executables with those extracted from program source and reveal scenarios where compiler optimization makes the task of accurate function signature recovery undecidable.
Yan Lin 0003, Debin Gao
SP1
2019 DynOpVm: VM-Based Software Obfuscation with Dynamic Opcode Mapping
Xiaoyang Cheng, Yan Lin 0003, Debin Gao, Chunfu Jia
ACNS2
2019 Control-Flow Carrying Code
abstract
Control-Flow Integrity~(CFI) is an effective approach in mitigating control-flow hijacking attacks including code-reuse attacks. Most conventional CFI techniques use memory page protection mechanism, Data Execution Prevention~(DEP), as an underlying basis. For instance, CFI defenses use read-only address tables to avoid metadata corruption. However, this assumption has shown to be invalid with advanced attacking techniques, such as Data-Oriented Programming, data race, and Rowhammer attacks. In addition, there are scenarios in which DEP is unavailable, e.g., bare-metal systems and applications with dynamically generated code. We present the design and implementation of Control-Flow Carrying Code~(C^3), a new CFI enforcement without depending on DEP, which makes the CFI policies embedded safe from being overwritten by attackers. C3 embeds the Control-Flow Graph (CFG) and its enforcement into instructions of the program by encrypting each basic block with a key derived from the CFG. The "proof-carrying" code ensures that only valid control flow transfers can decrypt the corresponding instruction sequences, and that any unintended control flow transfers or overwritten code segment would cause program crash with high probability due to the wrong decryption key and the corresponding random code bytes obtained. We implement C3 on top of an instrumentation platform and apply it to many popular programs. Our security evaluation shows that C3 is capable of enforcing strong CFI policies and is able to defend against most control-flow hijacking attacks while suffering from moderate runtime overhead.
Yan Lin 0003, Xiaoyang Cheng, Debin Gao
AsiaCCS1
2018 Towards Dynamically Monitoring Android Applications on Non-rooted Devices in the Wild
abstract
Dynamic analysis is an important technique to reveal sensitive behavior of Android apps. Current works require access to the code-level and system-level events (e.g., API calls and system calls) triggered by the running apps and consequently they can only be conducted on in-lab running environments (e.g., emulators and modified OS). The strict requirement of running environment hinders their deployment in scale and makes them vulnerable to anti-analysis techniques. Furthermore, current dynamic analysis of Android apps exploits input generators to invoke app behavior, which, however, cannot provide sufficient code coverage.
Xiaoxiao Tang, Yan Lin 0003, Daoyuan Wu, Debin Gao
WISEC2
2017 SafeStack ^+ : Enhanced Dual Stack to Combat Data-Flow Hijacking
Yan Lin 0003, Xiaoxiao Tang, Debin Gao
ACISP (2)1
2017 FRProtector: Defeating Control Flow Hijacking Through Function-Level Randomization and Transfer Protection
Jianming Fu, Yan Lin 0003
SecureComm3
2016 Control Flow Integrity Enforcement with Dynamic Code Optimization
Yan Lin 0003, Xiaoxiao Tang, Debin Gao, Jianming Fu
ISC1
2016 Impact of Environment on Branch Transfer of Software
Jianming Fu, Yan Lin 0003, Xu Zhang 0008
SecureComm2
2014 Computation Integrity Measurement Based on Branch Transfer
abstract
Tasks are selectively migrated to the cloud with the widespread adoption of the cloud computing platform, but the user cannot know whether the tasks are tampered in the cloud, so it is an urgent demand for cloud users to verify the execution integrity of the program in the cloud. The computation integrity measurement based on behavior is difficult to detect carefully crafted shell code. According to the property of shell code, this paper proposes a computation integrity measurement based on branch transfer called CIMB, which is a fine-grained instruction-level integrity measurement. In this approach, all branches in the user-level have been recorded, which effectively cover all execution control flow of a program, and CIMB can detect control-flow hijacking attacks without the support of source code, such as Return-oriented Programming (ROP) and Jump-oriented Programming (JOP). Meanwhile, distance between two instruction addresses and machine code of instruction can mask the measurement inconsistency derived from address space layout randomization of program and shared libraries. Finally, we have implemented CIMB with a dynamic binary instrumentation tool Pin on x86 32-bit version of ubuntu12.04. Its experimental results show that CIMB is feasible and it has a relatively stable measurement result, and the advantages of CIMB and factors affecting the results of measurement are analyzed and discussed.
Jianming Fu, Yan Lin 0003, Xu Zhang 0008, Pengwei Li
TrustCom2