Figure 1: Synthetic Audio Forensics Evaluation Challenge Round 1 Results.Performance (circle markers) from top five teams and the detection vs. false alarm curves are shown for three tasks of increasing difficulty: detection of (1) synthetic voice audio, (2) synthetic voice audio post-processed with various compression and resampling and (3) laundered to evade detection.

Lin Ai, Tharindu Sandaruwan Kumarage, Amrita Bhattacharjee, Zizhou Liu, Zheng Hui, Michael S. Davinroy, James Cook, Laura Cassani, Kirill Trapeznikov, Matthias Kirchner, Arslan Basharat, Anthony Hoogs, Joshua Garland, Huan Liu, Julia Hirschberg. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.

Binary analyses based on deep neural networks (DNNs), or neural binary analyses (NBAs), have become a hotly researched topic in recent years. DNNs have been wildly successful at pushing the performance and accuracy envelopes in the natural language and image processing domains. Thus, DNNs are highly promising for solving binary analysis problems that are hard due to a lack of complete information resulting from the lossy compilation process. Despite this promise, it is unclear that the prevailing strategy of repurposing embeddings and model architectures originally developed for other problem domains is sound given the adversarial contexts under which binary analysis often operates.