Depression is a severe mental health disorder that constitutes a significant global health challenge, necessitating efficient and accurate automated diagnostic tools. Traditional approaches to automated depression assessment using multi-modal data often face three significant difficulties: small sample sizes in available datasets, high computational cost, and difficulties in effectively integrating different modalities. In this paper, we first mitigate the problem of dataset quality by utilizing an estimated imputation strategy for missing values based on a known data augmentation method. Additionally, we introduce a multi-modal depression detection model named MDDMamba, designed to address the latter two challenges by integrating features from pre-trained emotion models with the memory-efficient selective state space model named Mamba. The MDDMamba model combines features from pre-trained emotion models for both text and audio, enhancing the extraction of emotional information. The core of the MDDMamba model is the CrossMamba module, which replaces the traditional Transformer’s quadratic computational complexity with a linear complexity alternative, facilitating efficient multi-modal fusion. Experimental results on CMDC and EATD-Corpus datasets show that the MDDMamba model achieves better classification and regression performance and stronger generalization capabilities in depression detection. Notably, the MDDMamba model achieves an F1 score of 0.94 and a Pearson correlation coefficient of 0.86 on the CMDC dataset while maintaining lower memory cost, outperforming most advanced multi-modal depression detection methods across various experimental scenarios.

Abstract Video‐based emotion recognition has been a long‐standing research topic for computer scientists and psychiatrists. In contrast to traditional discrete emotional models, emotion recognition based on continuous emotional models can better describe the progression of emotions. Quantitative analysis of emotions will have crucial impacts on promoting the development of intelligent products. The current solutions to continuous emotion recognition still have many issues. The original continuous emotion dataset contains incomplete data annotations, and the existing methods often ignore temporal information between frames. The following measures are taken in response to the above problems. Initially, aiming at the problem of incomplete video labels, the correlation between discrete and continuous video emotion labels is used to complete the dataset labels. This correlation is used to propose a mathematical model to fill the missing labels of the original dataset without adding data. Moreover, this paper proposes a continuous emotion recognition network based on an optimized temporal convolutional network, which adds a feature extraction submodule and a residual module to retain shallow features while improving the feature extraction ability. Finally, validation experiments on the Aff‐wild2 dataset achieved accuracies of 0.5159 and 0.65611 on the valence and arousal dimensions, respectively, by adopting the above measures.

In recent years, small object detection has been widely applied to aerial scenes. Although existing small object detection algorithms have achieved significant success, it remains challenging to ensure an acceptable processing speed and detection accuracy simultaneously in high-resolution images. In this work, we propose a novel framework called SliNet, a slicingaided learning network with a SPPFCSPC Block and a ContentAware ReAssembly of Features (CARAFE) Block merged. Since aerial images usually have high resolutions and small detection targets, we slice images into smaller overlapping patches and integrate SPPFCSPC and CARAFE to find attention region with larger receptive field in dense-object scenes. Experimental results on dataset VisDrone-DET2019 show that the SliNet achieves competitive performance and obtains faster detection speed due to the decrease of computational cost. On the VisDrone2019- DET-val dataset, we attain a mAP score of 46.4% with a mAP50 score of 67.1%, which is better than the state-of-the-art models, demonstrating the superiority of our approach

Video‐based person re‐identification aims to verify the pedestrian identity from image sequences. The sequences are captured by cameras located in different directions at different times. Existing studies have certain limitations in the case of occlusions and pose variations. To solve the aforementioned problems, this study proposes a new two‐stage framework, from which the key‐image‐based fusion spatial–temporal feature (KISTF) of the pedestrian can be extracted from the video. The image‐level features at all timestamps are aggregated into the sequence‐level feature representation of the video by using an long short‐term memory network. Additionally, the concept of key image is defined for the image sequence, and the frame‐level feature of the pedestrian is extracted from these key images. The proposed spatial–temporal feature, KISTF, is obtained by fusing the sequence‐level feature and the frame‐level feature. It aims to solve the problem of pedestrian representation in small video data sets. Experiments are conducted on the iLIDS‐VID and PRID2011 data sets. The results demonstrate that the proposed approach outperforms state‐of‐the‐art video‐based re‐identification methods.

Learning‐based image interpolation methods have been proved to be effective in image interpolation. In this study, the authors propose an accurate image interpolation with adaptive k ‐nearest neighbour searching and non‐linear regression. The proposed method aims to find k ‐nearest neighbours of the input image patch and use them to learn the non‐linear mapping between low‐resolution and high‐resolution image patches. To be specific, they first divide the training image patches into many subspaces, then they utilise an adaptive robust and precise k nearest neighbour searching scheme with proposed normalised Gaussian similarity to find the k nearest neighbours in the matched subspace. The selected k image patch pairs are then used to learn the non‐linear regression model through an extreme learning machine. Furthermore, the proposed interpolation method is a cascade framework that consists of two stages. Stage 2 takes the results of Stage 1 as input to further improve the performance. Extensive experimental results on commonly used test images and image datasets indicate that their proposed algorithm obtains competitive performance against the state‐of‐the‐art methods both in terms of objective evaluation values and the subjective effect of reconstructed images.

With the rapid development of social media, visual sentiment analysis from image or video has become a hot spot in visual understanding researches. In this work, we propose an effective approach using visual and textual fusion for sentiment analysis of short GIF videos with textual descriptions. We extract both sequence-level and frame-level visual features for each given GIF video. Next, we build a visual sentiment classifier by using the extracted features. We also define a mapping function, which converts the sentiment probability from the classifier to a sentiment score used in our fusion function. At the same time, for the accompanied textual annotations, we employ the Synset forest to extract the sets of the meaningful sentiment words and utilize the SentiWordNet3.0 model to obtain the textual sentiment score. Then, we design a joint visual-textual sentiment score function weighted with visual sentiment component and textual sentiment one. To make the function more robust, we introduce a noticeable difference threshold to further process the fused sentiment score. Finally, we adopt a grid search technique to obtain relevant model hyper-parameters by optimizing a sentiment aware score function. Experimental results and analysis extensively demonstrate the effectiveness of the proposed sentiment recognition scheme on three benchmark datasets including T-GIF dataset, GSO-2016 dataset and Adjusted-GIFGIF dataset.

Person re-identification has gradually become a hot research topic in many fields, such as security, criminal investigation and video analysis. In this paper, we propose a novel feature extraction framework for video-based person re-identification, namely, the partial attribute-driven network (PADNet). The proposed method is based on the deep-learning architecture and incorporates the attribute and identity learning of the pedestrian. Existing attribute research always focuses on the feature representation at the global-level. Unlike them, first, the pedestrian is automatically partitioned to several body parts in our work. Then the pedestrian and his/her body parts are annotated by the global and partial attributes, respectively. Finally, we employ a four-branch multi-label network to explore the spatial-temporal cues of videos by utilizing these labeled samples. Extensive experiments are conducted on two video-based datasets, including PRID2011 and iLIDS-VID. The experimental results demonstrate the superiority and effectiveness of the proposed PADNet over the state-of-the-art approaches.

Images captured under low‐light conditions are often of low visibility. To improve visualisation, a novel low light image enhancement method is presented based on the non‐uniform illumination prior model. First, the k ‐means method is used to process the value channel in the hue‐saturation‐value (HSV) colour space after space conversion of the input image. Then, the initial illumination of segmented scenes is estimated by an improved maximum red–green–blue method. Next, an illumination preservation method is presented to maintain the naturalness of the enhanced image. Furthermore, the non‐uniform illumination prior model is proposed to enhance the textural details in the enhanced image. Fast Fourier transformation is used to accelerate the optimisation. Since an adaptive weight is assigned, the proposed method can preserve the edges and textures at the bright and edge areas. Experimental analysis shows that the results using the proposed method have less noise, better illumination, improved contrast, and satisfactory naturalness. In addition, the proposed method can provide better quality images in terms of subjective and objective assessments.