This paper proposes a novel, resource-efficient approach to Visual Speech Recognition (VSR) leveraging speech representations produced by any trained Automatic Speech Recognition (ASR) model. Moving away from the resource-intensive trends prevalent in recent literature, our method distills knowledge from a trained Conformer-based ASR model, achieving competitive performance on standard VSR benchmarks with significantly less resource utilization. Using unlabeled audio-visual data only, our baseline model achieves a word error rate (WER) of 47.4% and 54.7% on the LRS2 and LRS3 test benchmarks, respectively. After fine-tuning the model with limited labeled data, the word error rate reduces to 35% (LRS2) and 45.7% (LRS3). Our model can be trained on a single consumer-grade GPU within a few days and is capable of performing real-time end-to-end VSR on dated hardware, suggesting a path towards more accessible and resource-efficient VSR methodologies.

This work addresses the problem of 3D-localizing and enhancing the speech of one main speaker in noisy multi-speaker hospital environments using a multi-channel microphone array. In our model, we propose conducting speaker localization using a machine learning model based on convolutional recurrent neural networks (CRNN) followed by minimum variance distortionless response (MVDR) beamforming. In addition, to ensure that our speech enhancement module is adaptive when deployed in different environments, we trained a meta learning model. Firstly, in the localization step, an estimation of the direction of arrival (DOA) in the elevation and azimuth planes is executed. This is conducted in a 3D space with the presence of noise, reverberation, and up to two more speakers. Using estimated DOA, the MVDR beamformer then enhances the speech of the main speaker. In order to test our model, we adopted and simulated a real-world problem where the objective was to enhance the speech of a clinician in a noisy intensive care unit (ICU) with the presence of other speakers. Furthermore, in order to validate our model, we adopted a speech-to-text module to evaluate the word error rate. Moreover, we implemented our algorithm on hardware using commercially available components and tested it in a real environment. Results showed that our model outperforms other machine learning and non-machine learning algorithms. Finally, we used our trained Meta Learning model to show that our model can adapt to new environments while maintaining high performance after retraining with only a few-shot recordings.

Logic locking is a prominent technique to protect the integrity of hardware designs throughout the integrated circuit design and fabrication flow. However, in recent years, the security of locking schemes has been thoroughly challenged by the introduction of various deobfuscation attacks. As in most research branches, deep learning is being introduced in the domain of logic locking as well. Therefore, in this article we present SnapShot, a novel attack on logic locking that is the first of its kind to utilize artificial neural networks to directly predict a key bit value from a locked synthesized gate-level netlist without using a golden reference. Hereby, the attack uses a simpler yet more flexible learning model compared to existing work. Two different approaches are evaluated. The first approach is based on a simple feedforward fully connected neural network. The second approach utilizes genetic algorithms to evolve more complex convolutional neural network architectures specialized for the given task. The attack flow offers a generic and customizable framework for attacking locking schemes using machine learning techniques. We perform an extensive evaluation of SnapShot for two realistic attack scenarios, comprising both reference combinational and sequential benchmark circuits as well as silicon-proven RISC-V core modules. The evaluation results show that SnapShot achieves an average key prediction accuracy of 82.60% for the selected attack scenario, with a significant performance increase of 10.49 percentage points compared to the state of the art. Moreover, SnapShot outperforms the existing technique on all evaluated benchmarks. The results indicate that the security foundation of common logic locking schemes is built on questionable assumptions. Based on the lessons learned, we discuss the vulnerabilities and potentials of logic locking uncovered by SnapShot. The conclusions offer insights into the challenges of designing future logic locking schemes that are resilient to machine learning attacks.

Efficient development of parallel software is one of the biggest hurdles to exploit the advantages of heterogeneous multi-core architectures. Fast and accurate compiler technology is required for determining the trade-off between multiple objectives, such as power and performance. To tackle this problem, the paper at hand proposes the novel heuristic TONPET. Furthermore, it is integrated into the SLX tool suite for a detailed evaluation and an applicability study. TOPNET is tested against representative benchmarks on three different platforms and compared to a state-of-the-art Evolutionary Multi Objective Algorithm (EMOA). On average, TONPET produces 6% better Pareto fronts, while being 18X faster in the worst case.