Deciphering sentiments or emotions in face-to-face human interactions is an inherent capability of human intelligence, and thus a natural goal of artificial intelligence. The proliferation of multimedia data in video sites gives rise to multimodal sentiment analysis in various applications and research fields such as movie and product review, opinion polling, and affective computing. In order to improve the performance of multimodal sentiment analysis task, this paper proposes a novel neural network with multiple stacked attention mechanism (MSAM) on multimodal data containing texts, video, and audio at an utterance level. We conduct experiments using two benchmark datasets, namely CMU Multi-modal Opinion-level Sentiment Intensity (CMU-MOSI) corpus, and CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) corpus. Compared with a comprehensive set of state-of-the-art baselines, the evaluation results demonstrate the effectiveness of our proposed MSAM network.

Sleep stage classification is a critical concern in sleep quality assessment and disease diagnosis. Graph network based studies for sleep stages classification have achieved promising performance. However, these studies still ignored the importance of learning morphological feature information with the spatial-temporal relationship among multi-modal physiological signals. To address this issue, we propose a Spatial-temporal Adaptive Graph Network named StAGN for sleep stage classification. The main advantage of StAGN is to adaptively learn the time-dependent and channel-wise interdependent waveform morphological features in multimodal physiological signals. Such features will be extracted by a modified 1-dimensional ResNet with a projection shortcut connection and adjusted by a joint spatial-temporal attention, thereby best serving the followed brain topological connection graph network for sleep stage classification. Meanwhile, we leverage the contrastive learning scheme with label information to further improve classification accuracy without changing the signal morphology. Experiment results on two publicly available sleep datasets of ISRUC-S1 and ISRUC-S3 show that the proposed StAGN can achieve a competitive performance for sleep stage classification, which is superior to the state-of-the-art counterparts.