ABSTRACT Visible‐Infrared Cross‐Modal Person Re‐Identification (VI‐ReID) confronts significant challenges arising from feature misalignment across spectra and structural differences within the same identity, especially under low‐light scenarios. To address these, we propose the Frequency‐Spatial‐Channel Collaborative Enhancement Network (FSC‐Net), leveraging a spatial‐semantic collaborative mechanism and a multi‐dimensional Frequency‐Spatial‐Channel feature enhancement framework. FSC‐Net employs a Frequency Semantic Attention module for cross‐modal semantic alignment, a Gated Channel Attention module for discriminative feature enhancement, and a Spatial Transformer for improved local structural perception. Extensive experiments demonstrate the superiority of FSC‐Net, achieving state‐of‐the‐art Rank‐1/mAP accuracy of 95.07%/91.18% on RegDB, 76.90%/74.26% on SYSU‐MM01, and 56.69%/63.37% on the low‐light LLCM dataset, respectively. Especially, on the RegDB dataset, our model achieves an absolute gain of 5.2% and 8.1% in terms of Rank‐1 and mAP in infrared‐to‐visible mode.

ABSTRACT Visible‐infrared person re‐identification (VI‐ReID) is a cross‐modal retrieval task characterized by significant challenges, with the objective of precisely identifying and matching pedestrian instances across different spectral modalities, namely visible‐light and infrared imagery. The primary difficulties stem from substantial inter‐modal discrepancies and intra‐modal feature variations, which complicate effective cross‐modal matching. While existing approaches generally focus on embedding heterogeneous modal data into a unified feature space to extract shared representations, they often overlook the discriminative identity information embedded within modality‐specific features. To overcome this inherent limitation, we propose a novel pinwheel‐guided dynamic representation network (PDRNet), designed to mine and enhance the directional structural cues and scale‐sensitive discriminative features inherent in modality‐specific representations. Specifically, we integrate direction‐aware pinwheel convolution (PConv) into a two‐stream architecture to strengthen local structural representation and guide the learning of shared semantic features, thereby improving both the discriminability and structural modeling of modality‐specific information. Furthermore, to accommodate scale disparities across modalities and individuals, we incorporate a scale‐based dynamic loss (SD loss), which adaptively adjusts the loss weights related to scale and positional information. This mechanism mitigates the error amplification often observed in small‐scale samples and enhances both the discriminative power and robustness of cross‐modal matching across varying scales. We perform extensive experiments on multiple well‐established public benchmarks. The results consistently show that the proposed PDRNet achieves superior performance compared to existing methods in both recognition accuracy and cross‐modal matching effectiveness.

ABSTRACT Cross‐resolution person re‐identification (CR‐ReID) seeks to overcome the challenge of retrieving and matching specific person images across cameras with varying resolutions. Numerous existing studies utilize established CNNs and ViTs to resize captured low‐resolution (LR) images and align them with high‐resolution (HR) image features or construct common feature spaces to match between images of different resolutions. However, these methods ignore the potential feature connection between the LR and HR images of the same pedestrian identity. Besides, the CNNs or ViTs usually obtain outliers within the attention maps of LR images; this inclination to excessively concentrate on anomalous information may obscure the genuine and anticipated characteristics between images, which makes it challenging to extract meaningful information from the images. In this work, we propose the abnormal feature elimination and reconfiguration Transformer (ARTransformer), a novel network architecture for robust cross‐resolution person re‐identification tasks. This method uses a resolution feature discriminator to learn resolution‐invariant features and output feature matrices of images with different resolutions. It then calculates the potential feature relationships between images of pedestrians with the same identity but different resolutions through a new cross‐resolution landmark agent attention (CR‐LAA) mechanism. Conclusively, it utilizes output feature matrices to model LR and HR image interactions by mitigating abnormal image features and prioritizing attention on the target person by learning representations from input images of various resolutions. Experimental results show that ARTransformer performs well in matching images with different resolutions, even with unseen resolution, and extensive evaluations on four real‐world datasets confirm the excellent results of our approach.

ABSTRACT Successfully training a model requires substantial computational power, excellent model design, and high training costs, which implies that a well‐trained model holds significant commercial value. Protecting a trained Deep Neural Network (DNN) model from Intellectual Property (IP) infringement has become a matter of intense concern recently. Particularly, embedding and verifying watermarks in black‐box models without accessing internal model parameters, while ensuring the robustness and invisibility of the watermark, remains a challenging issue. Unlike many existing methods, we propose a cascade ownership verification framework based on invisible watermarks, with a focus on how to effectively protect the copyright of black‐box watermark models and detect unauthorized users' infringement behaviors. This framework consists of two parts: watermark generation and copyright verification. In the watermark generation phase, watermarked samples are generated from key samples and label images. The difference between watermarked samples and key samples is imperceptible, while a specific identifier has been injected into the watermarked samples, leaving a backdoor as an entry point for copyright verification. The copyright verification phase employs hypothesis testing to enhance the confidence level of verification. In image classification tasks based on MNIST, CIFAR‐10, and CIFAR‐100 datasets, experiments were conducted on several popular deep learning models. The experimental results show that this framework offers high security and effectiveness in protecting model copyrights and demonstrates strong robustness against pruning and fine‐tuning attacks.

ABSTRACT Currently, artificial intelligence technology is rapidly penetrating into various fields of socioeconomic development with increasing depth and breadth, becoming an important force driving innovation and development, empowering thousands of industries, while also bringing challenges such as security governance. The application of deep neural network models must implement hierarchical access based on user permissions to prevent unauthorized users from accessing and abusing the model, and to prevent malicious attackers from tampering or damaging the model, thereby reducing its vulnerabilities and security risks. To address this issue, the model provider must implement a hierarchical authorization policy for the model, which can grant users access to the model based on their specific needs, while ensuring that unauthorized users cannot use the model. Common methods for implementing hierarchical authorization of models include pruning and encryption, but existing technologies require high computational complexity and have unclear hierarchical effects. In this article, we propose a sparsity polarization pruning approach for layered authorization, which combines sparsity regularization to filter insignificant channels and a polarization technique to cluster critical channels into distinct intervals. By pruning channels based on polarized scaling factors from the batch normalization (BN) layer, our method dynamically adjusts model precision to match user authorization levels. Initially, we extract the scaling factor of the BN layer to assess the importance of each channel. A sparsity regularizer is then applied to filter out irrelevant scaling factors. To enhance the clarity and rationality of pruning intervals, we use a polarization technique to induce clustering of scaling factors. So we proposed multiuser hierarchical authorization using sparsity polarization pruning for model active protection. Based on the grading requirements, we prune channels corresponding to varying numbers of significant scaling factors. Access is granted at different levels depending on the precision key provided by the user, thereby ensuring a secure and efficient means of accessing the model's resources. Experimental results demonstrate that our approach achieves superior grading performance across three datasets and two different neural networks, showcasing its broad applicability. Moreover, our method achieves effective grading just by pruning a small portion of the channels, offering a high level of efficiency.

Summary In an agnostic space environment, aerial person re‐identification (Re‐ID) is a task that the query person may not occur in the gallery set, it is considered a subordinate task within the domain of open‐world person Re‐ID, and is a more challenging and practical application research. The aerial person images, captured by unmanned aerial vehicles, present more significant challenges such as weak appearance features, fewer individual person samples and occlusion due to variations in camera height and viewing angles compared to ground‐level images. Most state‐of‐the‐arts person Re‐ID methods developed for open‐world datasets rely heavily on local convolutional neural networks but exhibit suboptimal performance when directly applied to aerial person Re‐ID tasks. In this article, a parameter instance learning based on vision transformers (ViT) model is introduced for the design of aerial person Re‐ID. Initially, we employ a self‐supervised paradigm grounded in parameter instance discrimination, aiming to capture feature alignment and instance similarity. Subsequently, using labeled training data, we optimize the network model through the calculation of two types of loss functions. Finally, we employ a feature enhancement strategy utilizing zero‐padding and displacement techniques. This strategy effectively and directly enhances the robustness of the ViT model against issues such as occlusion and misalignment. We conducted experiments on a Re‐ID dataset to validate the effectiveness of the method. Our approach achieves a mean average precision of 57.31% and a Rank‐1 accuracy of 65.29% on the aerial person Re‐ID dataset PRAI‐1581.

Summary The ability of facial expression recognition, the ability to decipher human emotions from facial features, and we can judge the emotional state of human beings by analysing facial expressions. Empowered by this technology, the related research of student sentiment analysis has become a focal point in the realm of educational technology. Teachers can now read the emotional state of their students and estimate the effectiveness of their teaching strategies, enabling them to implement appropriate intervention techniques to enhance teaching outcomes. However, facial expression recognition techniques are currently limited by their shortcomings. Network performance degradation and loss of feature information are key issues that hinder the effectiveness of sentiment analysis of student feedback. To overcome these limitations, in this paper, we propose RFMNet, a novel network model based on deep learning theory. RFMNet, with its higher extraction capability, is designed to accurately analyse student expressions. It introduces a relation‐aware global attention (RGA) module, which facilitates the integration of more discriminative expression features in the image, leading to more refined sentiment analysis. This innovative model also employs a Mish activation function and a focal loss function. The Mish activation function is of particular interest because it helps to avoid the loss of feature information due to neuron deactivation when the ReLU gradient is . This approach results in a more robust and accurate facial expression recognition model. Our model was tested on the FERPlus public dataset. The model achieved an average recognition accuracy of 89.62%, which is a testament to its performance in decoding the facial expressions of students. This precision allows for intelligent processing of teaching information and real‐time feedback, enabling teachers to adapt their teaching strategies to better suit the needs of their students.

Summary Various deep neural network (DNN) model watermarks have been proposed by researchers to verify copyrights for deep neural networks DNN. However, most DNN watermarking methods cannot prevent attackers from stealing and using the model. Unlike many existing approaches, this paper uses a channel pruning algorithm to protect DNN models, which verifies DNN models copyrights but also prevents the illegal use of DNN models. In this work, the pruning threshold or pruning rate is used as the secret key of a DNN model. After the secret key is distributed to multiple users, they prune the DNN model with the secret key, and the pruned and fine‐tuned model is provided to the users. The users can verify ownership of the model according to the pruning accuracy and fine‐tuning accuracy. If the secret key is incorrect, the accuracy of the model after fine‐tuning will be very low, and users will be unable to use the reasoning function of the fine‐tuned model. Based on the CIFAR‐10 and CIFAR‐100 datasets, we conducted experiments on five popular DNN models. The experimental results show that we can authorize multiple users by pruning very few channels in the convolution layers of the DNN model.

Summary Person re‐identification (Re‐ID) aims to retrieve a person of interest across multiple nonoverlapping cameras. In recent years, to enable person Re‐ID technology to play out its application value in real‐world scenarios, visual surveillance through unmanned aerial vehicle (UAV) platforms has received intense attention, and aerial person datasets have been constructed. However, the pedestrian images captured by ground cameras and those captured by UAVs exhibit great differences. Person Re‐ID methods based on ground person images have difficulty performing Re‐ID on aerial person images. In this paper, we first use a meta‐transfer method to learn to generalize the aerial person Re‐ID task. Specifically, combining the ideas of meta‐learning and transfer learning, a meta‐learning strategy is introduced to study a feature extractor, and a transfer learning strategy is introduced to utilize and further improve the acquired meta‐knowledge. To prevent the catastrophic forgetting and overfitting problems caused by large‐scale model parameters, we freeze the lower‐layer neurons with powerful generalization and fine‐tune the higher‐layer neurons with strong specialization to transfer and represent the feature extractor. In addition, during the model training process, it is observed that the presence of difficult categories in the given dataset significantly affects the convergence speed and recognition accuracy of the utilized meta‐learning method, and the loss function based on the general Euclidean distance measure tends to mislead the model to optimize in a suboptimal direction. Therefore, we introduce a curriculum sampling based learning strategy that is harmonized with our meta‐transfer learning framework and a new metric formulation of sample similarity based on the Mahalanobis distance to improve the model. In the experimental part, when our method is adopted, a Rank‐1 accuracy of 63.63% and a mean average precision (mAP) of 38.02% are achieved on an aerial Re‐ID dataset, demonstrating its potential for completing person Re‐ID with aerial images. The results obtained on two commonly used ground pedestrian datasets show the generalization of the proposed method. Ablation studies also validate that each component contributes to improving the performance of the model.

Summary Facial expression is the basis of human emotion recognition, and we can infer the emotional state of human beings by analysing facial expressions. As the main method of human emotional expression, facial expression contains much information about inner emotional changes. In recent years, with the wide application of artificial intelligence in education, related research on student sentiment analysis has become a hot topic in the educational technology field. Analysing affective data is helpful for understanding students' learning status and provides an important basis for the effective implementation of learning interventions, which is of great significance for the evaluation of teaching effects and changes in teaching methods. However, traditional learner emotion recognition methods have some problems, such as low recognition rates, complex algorithms, poor robustness, and easy loss of the key information of facial expression features. Therefore, this article proposes a learner emotion recognition method based on NAGNet. The network model is composed of Res2Net, nonlocal attention and GeM pooling, which can fuse global expression feature information to realize fine‐grained sentiment analysis. Additionally, we conducted training and experiments on the large‐scale learner emotion dataset FERPlus. The NAGNet model trained by the public emotion dataset FERPlus has a recognition accuracy of 89.3% for eight kinds of student emotions. The experimental results show that the method can quickly and accurately identify learner emotional states. We conduct experiments in a real classroom scenario, and use the NAGNet model proposed in this article to analyse and detect students' real‐time sentiment. Then teachers improve teaching methods by understanding the feedback of students' classroom status. Therefore, this method has important reference significance in the construction of wisdom classrooms and wisdom learning environments.

Visible–thermal person reidentification (VT-ReID) is an emerging cross-modality matching problem, which aims to identify the same person across the daytime visible modality and nighttime thermal modality in the Internet of Things. Existing cutting-edge approaches consistently attempt to exploit image generation technique to generate cross-modality images or design various feature-level constraints to align feature distribution of heterogeneous data. However, color variations originating from the different imaging processes of spectrum cameras remain unsolved, which leads to suboptimal feature representations. In this article, we present a simple but very effective data augmentation method named Random Gray for the cross-modality matching task. Given a training sample, Random Gray randomly selects a rectangular region and translates it to grayscale. In this process, training images with fusing various levels of visible and grayscale information are generated, thereby reducing the risk of overfitting and making the model robust to color variations. Besides, we introduce a novel pooling method called softpooling to retain more information in the reduced activation maps. With softpooling layer, the network can learn more discriminative person features and further boost its retrieval performance. We conduct extensive experiments on publicly available cross-modality Re-ID data sets (SYSU-MM01 and RegDB) to demonstrate the effectiveness of our proposed method. Experimental results show that Random Gray and softpooling strategies yield significant accuracy improvement, and they can be utilized as training tricks for further VT-ReID research.

Visible-Infrared person reidentification (VI-ReID) is a challenging matching problem due to large modality variations between visible and infrared images. Existing approaches usually bridge the modality gap with only feature-level constraints, ignoring pixel-level variations. Some methods employ a generative adversarial network (GAN) to generate style-consistent images, but it destroys the structure information and incurs a considerable level of noise. In this article, we explicitly consider these challenges and formulate a novel spectrum-aware feature augmentation network named SFANet for cross-modality matching problem. Specifically, we put forward to employ grayscale-spectrum images to fully replace RGB images for feature learning. Learning with the grayscale-spectrum images, our model can apparently reduce modality discrepancy and detect inner structure relations across the different modalities, making it robust to color variations. At feature level, we improve the conventional two-stream network by balancing the number of specific and sharable convolutional blocks, which preserve the spatial structure information of features. Additionally, a bidirectional tri-constrained top-push ranking loss (BTTR) is embedded in the proposed network to improve the discriminability, which efficiently further boosts the matching accuracy. Meanwhile, we further introduce an effective dual-linear with batch normalization identification (ID) embedding method to model the identity-specific information and assist BTTR loss in magnitude stabilizing. On SYSU-MM01 and RegDB datasets, we conducted extensively experiments to demonstrate that our proposed framework contributes indispensably and achieves a very competitive VI-ReID performance.

Abstract Sentiment analysis has a wide application prospect in business, medicine, security and other fields, which provides a new perspective for the development of education. Students' sentiment data play an important role in the evaluation of teachers' teaching quality and students' learning effect, and provide a basis for the implementation of effective learning intervention. However, most of the research is to obtain the real‐time learning status of students in the classroom through teachers' naked eye observation and students' text feedback, which will lead to some problems such as incomplete feedback content and delayed feedback analysis. Based on the mini‐Xception framework, this article implements the real‐time identification and analysis of student sentiment in classroom teaching, and the degree of student engagement is analyzed according to the teaching events triggered by teacher to provide reasonable suggestions for subsequent teaching progress. The experimental results show that the mini‐Xception model trained by FER2013 data sets has high recognition accuracy for the real‐time detection of seven student sentiments, and the average accuracy is 76.71%. Compared with text feedback, it can assist teachers in understanding student learning states in time so that they can take corresponding actions, and realize the real‐time performance of wisdom classroom teaching information feedback, the high efficiency of information transmission, and the intelligence of information processing.

Summary Person re‐identification (Re‐ID) aims at identifying the same person across multiple non‐overlapping camera views. A number of existing methods have been presented for this task in a fully‐supervised manner that requires a large amount of training annotations. However, obtaining high quality labels is extremely time consuming and expensive. In this article, we focus on the semi‐supervised person Re‐ID and propose a one‐shot stepwise learning method to address the above issue. It exploits only one labeled data along with additional unlabeled samples to gradually but steadily improving the discriminative capability of the feature representation. Specifically, we first construct labeled data portion to train Re‐ID model. Then we fine‐tune the overall system by the following two steps iteratively: (1) assigning the estimated labels to the unlabeled portion; (2) updating the network parameters according to the selected data. During the propagation process, different from conventional sampling method, we propose a novel dynamic sampling strategy to enlarge the pseudo‐labeled subset step by step to make the pseudo labels more reliable. On Market‐1501, DukeMTMC‐ReID and MARS datasets, we conducted extensively experiments to demonstrate that our proposed method contributes indispensably and achieves a very competitive Re‐ID performance.

We propose a rotation and scale invariant method to locate people lying on the ground. Unlike conventional human-shape detection methods which assume that all human shapes are in upright position, a person lying on the ground can have arbitrary orientation and pose. Accounting for every possible body configuration would thus require a huge training dataset that would be challenging to gather. In this paper, we propose a method which increases the size of a small training dataset and allows to detect multiple body poses. To do so, our method increases the size of the dataset with a geometric distortion method followed by a rejection sampling method. Then, it automatically identifies K body configurations in the training set, realign it in upright position and trains K SVM classifiers, one for each body configuration. Lying pose detection is then performed by considering a max pooling strategy across all K SVM classifiers.