Multimodal question answering (QA) often requires identifying which video, audio, or sensor tokens are relevant to the question. Yet modality disagreements are common: off-camera speech, background noise, or motion outside the field of view often mislead fusion models that weight all streams equally. We present RAVEN, a unified QA architecture whose core is QuART, a query-conditioned cross-modal gating module that assigns scalar relevance scores to each token across modalities, enabling the model to amplify informative signals and suppress distractors before fusion. RAVEN is trained through a three-stage pipeline comprising unimodal pretraining, query-aligned fusion, and disagreement-oriented fine-tuning - each stage targeting a distinct challenge in multi-modal reasoning: representation quality, cross-modal relevance, and robustness to modality mismatch. To support training and evaluation, we release AVS-QA, a dataset of 300K synchronized Audio-Video-Sensor streams paired with automatically generated question-answer pairs. Experimental results on seven multi-modal QA benchmarks - including egocentric and exocentric tasks - show that RAVEN achieves up to 14.5% and 8.0% gains in accuracy compared to state-of-the-art multi-modal large language models, respectively. Incorporating sensor data provides an additional 16.4% boost, and the model remains robust under modality corruption, outperforming SOTA baselines by 50.23%. Our code and dataset are available at https://github.com/BASHLab/RAVEN.

Modern perception models, particularly those designed for multisensory egocentric tasks, have achieved remarkable performance but often come with substantial computational costs. These high demands pose challenges for real-world deployment, especially in resource-constrained environments. In this paper, we introduce EgoAdapt, a framework that adaptively performs cross-modal distillation and policy learning to enable efficient inference across different egocentric perception tasks, including egocentric action recognition, active speaker localization, and behavior anticipation. Our proposed policy module is adaptable to task-specific action spaces, making it broadly applicable. Experimental results on three challenging egocentric datasets EPIC-Kitchens, EasyCom, and Aria Everyday Activities demonstrate that our method significantly enhances efficiency, reducing GMACs by up to 89.09%, parameters up to 82.02%, and energy up to 9.6x, while still on-par and in many cases outperforming, the performance of corresponding state-of-the-art models.

Spoken Language Understanding (SLU) systems must balance performance and efficiency, particularly in resource-constrained environments. Existing methods apply distillation and quantization separately, leading to suboptimal compression as distillation ignores quantization constraints. We propose QUADS, a unified framework that optimizes both through multi-stage training with a pre-tuned model, enhancing adaptability to low-bit regimes while maintaining accuracy. QUADS achieves 71.13\% accuracy on SLURP and 99.20\% on FSC, with only minor degradations of up to 5.56\% compared to state-of-the-art models. Additionally, it reduces computational complexity by 60--73$\times$ (GMACs) and model size by 83--700$\times$, demonstrating strong robustness under extreme quantization. These results establish QUADS as a highly efficient solution for real-world, resource-constrained SLU applications.

Most existing speech disfluency detection techniques only rely upon acoustic data.In this work, we present a practical multimodal disfluency detection approach that leverages available video data together with audio.We curate an audiovisual dataset and propose a novel fusion technique with unified weight-sharing modality-agnostic encoders to learn the temporal and semantic context.Our resilient design accommodates real-world scenarios where the video modality may sometimes be missing during inference.We also present alternative fusion strategies when both modalities are assured to be complete.In experiments across five disfluency-detection tasks, our unified multimodal approach significantly outperforms Audio-only unimodal methods, yielding an average absolute improvement of 10% (i.e., 10 percentage point increase) when both video and audio modalities are always available, and 7% even when video modality is missing in half of the samples.

Partial computation offloading offers lower latency, privacy (by processing data close to the source), and higher performance. In partial offloading, some of the processing is performed on the less capable Internet-of-Things (IoTs), and more complex computation is performed in more capable cloud computers. Though many well-established security protocols are available to address external intrusion, it introduces significant overhead for large data sample. This overhead is becoming a challenging problem with the recent trend and promises of on-device deep neural network (DNN) inference. This paper studies how much prior data distribution knowledge is required by an intruder to retrieve the input from a partially computed vector. We explore simple and complex DNN tasks, which are unknown to the intruder. We show that for simple datasets, Auto-Encoder(AE) and Variational Auto Encoder(VAE) can retrieve the original input image with a 4% accuracy drop on average. However, the same is not valid for complex data distribution.

This article presents a new convolutional neural network (CNN) topology using wavelet kernels to detect and discriminate single or multiple partial discharge (PD) locations in high voltage power apparatus with increased accuracy. The method is tested on an electrical equipment model with acoustic PD sensors. A cubical tank has been emulated in the laboratory representing the equipment under test and partial discharge sources have been placed at different positions along with the required data acquisition hardware. The present scheme eliminates the requirement of separate algorithms for feature extraction and classification of the acquired PD signals. Wavelet kernels of the CNN play a crucial role in feature learning, and the proposed CNN architecture as a whole, can classify the features in a supervised manner. The performance of the proposed scheme is compared with other existing methods using the same data set. It is found that an overall accuracy of 97.64% is achieved by the proposed method, outperforming other existing methods by a significant margin of at least 5% in terms of accuracy. The developed module is a generic one and can be adapted for different high voltage electrical apparatus with similar topological structures; hence, it can be used in various ways in power industry.