In this study, we investigate multimodal foundation models (MFMs) for emotion recognition from non-verbal sounds. We hypothesize that MFMs, with their joint pre-training across multiple modalities, will be more effective in non-verbal sounds emotion recognition (NVER) by better interpreting and differentiating subtle emotional cues that may be ambiguous in audio-only foundation models (AFMs). To validate our hypothesis, we extract representations from state-of-the-art (SOTA) MFMs and AFMs and evaluated them on benchmark NVER datasets. We also investigate the potential of combining selected foundation model (FM) representations to enhance NVER further inspired by research in speech recognition and audio deepfake detection. To achieve this, we propose a framework called MATA (Intra-Modality Alignment through Transport Attention). Through MATA coupled with the combination of MFMs: LanguageBind and ImageBind, we report the topmost performance with accuracies of 76.47%, 77.40%, 75.12% and F1-scores of 70.35%, 76.19%, 74.63% for ASVP-ESD, JNV, and VIVAE datasets against individual FMs and baseline fusion techniques and report SOTA on the benchmark datasets.

Deep learning-based models are vulnerable to adversarial examples crafted with different adversarial attack techniques. Numerous attack methods have been proposed that utilize gradient information of deep model to craft an adversarial examples. Amongst the existing defense mechanisms, adversarial training has gained considerable attention in building robust deep models that remain effective against different adversarial attacks. However, adversarial training demands high computational cost during the development of a robust deep model. In this paper, we present a simple yet effective defense mechanism against adversarial attacks. The proposed defense mechanism uses the concept of bit plane slicing for de-noising of an input image. The efficacy of the proposed defense technique has been evaluated on two benchmark image datasets, viz. MNIST and Fashion-MNIST datasets. The experiments and results show that the proposed defence technique yields comparable and competitive performance to state-of-the-art defense techniques against adversarial attacks.

Deep learning techniques have been widely applied for performing intelligent fault diagnosis (IFD) for applications such as bearing fault diagnosis, wind turbines and drilling operations. Notwithstanding the huge success of deep learning-based models for intelligent fault diagnosis, their vulnerability against adversarial attacks have been neglected to a great extent. The adversarial attacks aim to craft an adversarial sample by addition of imperceptible perturbation to its clean data sample. In this paper, we investigate the performance of different deep models against four state-of-the-art adversarial attacks. A total of four deep models have been tested against untargeted white-box adversarial attacks. Moreover, analysis of transferability of adversarial examples across different deep models is also inspected. Experiments and results reveal that deep models for IFD are highly susceptible to adversarial examples crafted with four state-of-the-art adversarial attacks. The proposed work presents an extensive insight on adversarial samples of machinery vibration signals of the CWRU dataset. Additionally, we are also releasing a first ever adversarial attack dataset for IFD, i.e. Adv-IFD. The code used in this work and adversarial attack datasets are available at: https://github.com/achyutmani/ADV-IFD

Classification of environmental sounds becomes an essential unit for several applications such as robot audition, audio-visual and ambient intelligent systems. Recent years have witnessed remarkable success of adversarial attacks for misguiding deep models with a high fooling rate. Adversarial examples crafted with adversarial techniques pose serious security threats for deployment of deep models to practical scenarios. In this paper, we show that adversarial signals crafted by performing search in the opposite direction of the search direction of the perturbations could be utilized while training of deep classifiers of environmental sound classification (ESC). The proposed reverse adversarial attack (RAA) technique can be used as data augmentation tool to enhance the accuracy of deep model that performs ESC. The deep network trained with the crafted adversarial signal shows high robustness against adversarial attacks. The efficacy of the proposed method is evaluated on two benchmark ESC datasets viz. ESC-10 and DCASE-2019 Task-1A datasets. The experiments and results show that the model trained with the proposed technique shows comparable and competitive performance to state-of-the-art (SOTA) deep ESC models.

Classification of environmental sound is an equally challenging task to classification of human speech or music. In recent years, attention mechanism has gained considerable attention in learning representative and prototypical features to resolve various research challenges from domains such as computer vision, text mining and sound classification. Advancements in the computer vision can be used for improving the performance of deep models designed to perform environmental sound classification as well. In this paper, we investigated the performance of deep network when twelve SOTA visual attention mechanisms are incorporated in the training of the deep network. The performance of the deep model is evaluated on two benchmark environmental sound classification datasets, viz. ESC-10 and DCASE-2019 task-1(A) datasets.

Knowledge of strengths and weaknesses of players is the key for team selection and strategy planning in any team sport such as Cricket. Computationally, this problem is mostly unexplored. Existing methods focus only on aggregate and macroscopic statistics that ignore many details. The central idea of our paper is to mine strength and weakness rules using short text commentary data. This dataset is compact, semi-structured, accurate, and yet ignored by the machine learning community until now. We collect fine-grained information about each player from the short text commentary dataset and represent it using domain-specific features identified by us. We employ a dimensionality reduction method specific to discrete random variable case, namely correspondence analysis and construct semantic relation between bowler and batsman. This relation is plotted using biplots. Human readable strength and weakness rules are extracted from the biplots. We have performed experiments using a large dataset that describes over one million deliveries. We validate our extracted rules using both intrinsic and extrinsic validation.