In this paper, we introduce a conditional transformer-based U-net architecture for speech emotion recognition (SER). The proposed architecture consists of convolutional transformer (CTR)-based encoder, feature decoder and emotion decoder. The CTR-based U-net encoder is designed to extract compressed bottleneck features for recognition. The emotion is then predicted by the CTR-based emotion decoder, where we use additional trainable emotion query to better capture the characteristics of different emotional states. The CTR-based feature decoder reconstructs the given input features from the bottleneck features. This auxiliary decoder allows us to use additional information while training the model, which further improves the recognition performance. Specifically, the proposed feature decoder is performed through sophisticated embedding of the predicted emotional states and the features of the encoder layers. In the proposed framework, we consider the channel attention for each time frame in the CTRs to better enable a real-time processing of emotion recognition. Experimental results showed that the proposed emotion classification method provided better recognition performance than the selected benchmark algorithms.

Robustness remains a fundamental challenge in real-world speech recognition, particularly under adverse acoustic conditions. In this paper, we introduce a robust speech emotion recognition (SER) approach using a conditional transformer-based architecture, specifically designed to maintain consistent performance across a wide range of noisy acoustic environments. The proposed convolutional transformer (CTr)-based architecture consists of three primary modules: a noise-classification module (NCM), a speech feature enhancement module (SFEM) and a speech emotion recognition module (SERM). The NCM first identifies the noise type presented in the given noisy speech features, producing embeddings that condition both the SFEM and SERM to better capture the characteristics of different noise types. The front-end SFEM enhances noisy speech features to improve the robustness of the SERM in adverse acoustic conditions and finally, the SERM predicts the emotional states. Experimental results show that that the proposed architecture performs robust under various noisy acoustic environment and provide better results than the selected benchmark methods.

Precision livestock farming (PLF) has been transformed by machine learning (ML), enabling more precise and timely interventions that enhance overall farm productivity, animal welfare, and environmental sustainability. However, despite the availability of various sensing technologies, few datasets leverage multiple modalities, which are crucial for developing more accurate and efficient monitoring devices and ML models. To address this gap, we present MmCows, a multimodal dataset for dairy cattle monitoring. This dataset comprises a large amount of synchronized, high-quality measurement data on behavioral, physiological, and environmental factors. It includes two weeks of data collected using wearable and implantable sensors deployed on ten milking Holstein cows, such as ultra-wideband (UWB) sensors, inertial sensors, and body temperature sensors. In addition, it features 4.8 million frames of high-resolution image sequences from four isometric view cameras, as well as temperature and humidity data from environmental sensors. We also gathered milk yield data and outdoor weather conditions. One full day’s worth of image data is annotated as ground truth, totaling 20,000 frames with 213,000 bounding boxes of 16 cows, along with their 3D locations and behavior labels. An extensive analysis of MmCows is provided to evaluate the modalities individually and their complementary benefits. The release of MmCows and its benchmarks will facilitate research on multimodal monitoring of dairy cattle, thereby promoting sustainable dairy farming. The dataset and the code for benchmarks are available at https://github.com/neis-lab/mmcows.

Heat stress, caused by a warming climate and the increasingly high milk-producing dairy cattle, is one of the major threats to the well-being of dairy cattle as well as the economic, environmental, and social sustainability of dairy farming around the world. Timely identification of cows under heat stress is crucial to improving animal welfare, preventing milk production losses, and preserving water and energy for cooling.

In this paper, we introduce a discriminative training algorithm of the non-negative matrix factorization (NMF) model for single-channel enhancement of convolutive noisy speech. The basis vectors for the clean speech and noises are estimated simultaneously during the training stage by incorporating the concept of classification from machine learning. Specifically, we employ the probabilistic generative model (PGM) of classification, specified by an inverse Gaussian distribution, as a priori structure for the basis vectors. Both the NMF and classification parameters are obtained by using the expectation-maximization (EM) algorithm, which guarantees convergence to a stationary point. Experimental results show that the proposed algorithm provides better enhancement performance than the benchmark algorithms.

In this paper, we propose a basis compensation algorithm for non-negative matrix factorization (NMF) models as applied to supervised single-channel speech enhancement. In the proposed framework, we use extra free basis vectors for both the clean speech and noise during the enhancement stage in order to capture the features which are not included in the training data. Specifically, the free basis vectors of the clean speech are obtained by exploiting a priori knowledge based on a Gamma distribution. The free bases of the noise are estimated using a regularization approach, which enforces them to be orthogonal to the clean speech and noise basis vectors estimated during the training stage. Experimental results show that the proposed NMF algorithm with basis compensation provides better performance in speech enhancement than the benchmark algorithms.

In this letter, we introduce a discriminative training algorithm of the basis vectors in the nonnegative matrix factorization (NMF) model for single-channel speech enhancement. The basis vectors for the clean speech and noises are estimated simultaneously during the training stage by incorporating the concept of classification from machine learning. Specifically, we consider the probabilistic generative model (PGM) of classification, which is specified by class-conditional densities, along with the NMF model. The update rules of the NMF are jointly obtained with the parameters of the class-conditional densities using the expectation-maximization (EM) algorithm, which guarantees convergence. Experimental results show that the proposed algorithm provides better performance in speech enhancement than the benchmark algorithms.