Motor imagery-based brain-computer interfaces have been increasingly applied in post-stroke motor rehabilitation. Their effectiveness depends on accurate electroencephalography (EEG)-based motor imagery (MI) classification to provide reliable feedback. However, differences in MI-related EEG patterns between left- and right-sided hemiparesis result in heterogeneity among stroke patients, complicating MI classification. Most classification models are developed for healthy subjects and fail to capture stroke-specific neurophysiology. To address this, we propose HSAMoE, a hemiparetic-side-aware mixture of experts framework. HSAMoE employs a router to identify the hemiparetic side and directs data to the corresponding specialized expert, which specifically models the non-lesioned hemisphere where MI patterns are reliably preserved. Experiments on the Stroke MI 50 and Stroke19-HS datasets demonstrate that HSAMoE achieves superior performance over competing methods.

Speech Referring Expression Comprehension (SREC) aims to localize the object in an image referred to by a spoken natural language query. However, raw speech is continuous and noisy, and prior ASR-free methods that align full utterances with transcripts using only global supervision can overfit to spurious correlations, limiting fine-grained grounding. To address this issue, we propose SliceCSRef, a robust SREC framework that improves generalization via dual-level semantic alignment. Beyond utterance-level speech–text alignment, SliceCSRef introduces slice-wise alignment that pairs randomly sampled speech segments with weakly matched transcript spans based on their relative temporal positions, providing fine-grained supervision without additional annotations. Experiments on six benchmarks show that SliceCSRef achieves state-of-the-art performance under standard settings and consistently improves robustness under truncated speech and playback-speed variations.

Although deep learning models have shown promise in Electroencephalogram (EEG) - based emotion recognition, their precision and robustness are often limited by domain-specific bias and low signal-to-noise ratio of EEG. Among the challenges encountered, the disparities in EEG data between subjects have been observed to potentially exceed those observed between different states of the task. The integration of identity domain information with task domain information may cause the model's learning of subject-dependent information, thereby hindering the ability to discern differences between tasks. This paper proposes a novel deep learning framework that improves the performance of EEG-based emotion classification models through the removal of the identity domain. Specifically, the proposed framework utilizes Auto-encoder (AE) based model as denoising module to extract globally informative features from the EEG signals. The domain information extraction module extracts subject-specific features from EEG samples by minimizing the differences between the network's output and the subject's identity labels while ensuring consistency of intermediate features across samples from the same subject. By combining domain-specific and globally informative features, an additional domain removal network is introduced before the final classification network to mitigate domain-specific biases by stripping away identity domain information from globally informative features through pairwise contrastive learning. The proposed EEG-based emotion recognition framework, supported by extensive experimental evidence, shows that incorporating the suggested domain removal technique significantly improves the robustness of the classification model and achieves state-of-the-art results in emotion classification.

In the rapidly advancing field of deep learning, a significant yet often overlooked challenge is the protection of intellectual property (IP) for models based on electroencephalography (EEG). These models, which handle sensitive and private physiological information, have not received as much attention for IP protection as their counterparts in more mainstream areas like computer vision (CV) and natural language processing (NLP). This paper introduces an innovative fingerprinting method for the first time, targeting IP protection of EEG-based models, a domain where conventional watermarking techniques fall short. We design a novel conditional diffusion model, tailored to a universal EEG format, which is the first application of diffusion models in model IP protection. Furthermore, our retrieval strategy, characterized by three distinct conditions, facilitates the construction of the fingerprint validation set from synthesized EEG samples. Experiments demonstrate that our method not only outperforms existing state-of-the-art (SOTA) protection techniques in robustness against various IP attacks but also excels in generating high-quality and high-diversity EEG samples.

Deep neural networks (DNNs) are widely employed across various domains, with their training costs making them crucial assets for model owners.However, the rise of Machine Learning as a Service has made models more accessible, but also increases the risk of leakage.Attackers can successfully steal models through internal leaks or API access, emphasizing the critical importance of protecting intellectual property.Several watermarking methods have been proposed, embedding secret watermarks of model owners into models.However, watermarking requires tampering with the model's training process to embed the watermark, which may lead to a decrease in utility.Recently, some fingerprinting techniques have emerged to generate fingerprint samples near the classification boundary to detect pirated models.Nevertheless, these methods lack distance constraints and suffer from high training costs.To address these issues, we propose to utilize conditional generative network to generate fingerprint data points, enabling a better exploration of the model's decision boundary.By incorporating margin loss during GAN training, we can control the distance between generated data points and classification boundary to ensure the robustness and uniqueness of our method.Moreover, our method does not require additional training of proxy models, enhancing the efficiency of fingerprint acquisition.To validate the effectiveness of our approach, we evaluate it on CIFAR-10 and Tiny-ImageNet, considering three types of model extraction attacks, fine-tuning, pruning, and transfer learning attacks.The results demonstrate that our method achieves ARUC values of 0.186 and 0.153 on CIFAR-10 and Tiny-ImageNet datasets, respectively, representing a remarkable improvement of 400% and 380% compared to the current leading baseline.The source code is available at https://github.com/wason981/MarginFinger.

Knowledge Tracing (KT) focuses on estimating students' knowledge states and predicting their future performances, which is a crucial task for online education platforms. In light of the advancements in educational big data and deep neural networks, numerous KT models have been proposed and promising outcomes have been achieved. Nevertheless, we have noted that current methods possess certain evident constraints. Thus, we propose a Knowledge Tracing model with Multi-Relational Integration (MRIKT): (1) we consider the more sophisticated relations between questions and skills, which can reveal deeper patterns of students' learning; (2) we emphasize the forgetfulness nature of students and the value of inter-exercises relations by incorporating a retentive module. Specifically, we choose graph convolutional networks to construct the advanced-relation between questions and skills, named graph representation module. Additionally, by linking different exercises, our novel retentive module, inspired by RetNet, can acquire valuable insights. We extensively evaluate the performance of MRIKT on three real-world datasets. The results demonstrate that MRIKT achieves outstanding performance, which improves at least 8.44% compared to baseline models.

Simultaneous Localization and Mapping (SLAM) has developed as a fundamental method for intelligent robot perception over the past decades. Most of the existing feature-based SLAM systems relied on traditional hand-crafted visual features and a strong static world assumption, which makes these systems vulnerable in complex dynamic environments. In this paper, we propose a robust monocular SLAM system by combining geometry-based methods with two convolutional neural networks. Specifically, a lightweight deep local feature detection network is proposed as the system front-end, which can efficiently generate keypoints and binary descriptors robust against variations in illumination and viewpoint. Besides, we propose a motion segmentation and depth estimation network for simultaneously predicting pixel-wise motion object segmentation and depth map, so that our system can easily discard dynamic features and reconstruct 3D maps without dynamic objects. The comparison against state-of-the-art methods on publicly available datasets shows the effectiveness of our system in highly dynamic environments.

Given a user’s historical interaction sequence, online novel recommendation suggests the next novel the user may be interested in. Online novel recommendation is important but underexplored. In this paper, we concentrate on recommending online novels to new users of an online novel reading platform, whose first visits to the platform occurred in the last seven days. We have two observations about online novel recommendation for new users. First, repeat novel consumption of new users is a common phenomenon. Second, interactions between users and novels are informative. To accurately predict whether a user will reconsume a novel, it is crucial to characterize each interaction at a fine-grained level. Based on these two observations, we propose a neural network for online novel recommendation, called NovelNet. NovelNet can recommend the next novel from both the user’s consumed novels and new novels simultaneously. Specifically, an interaction encoder is used to obtain accurate interaction representation considering fine-grained attributes of interaction, and a pointer network with a pointwise loss is incorporated into NovelNet to recommend previously-consumed novels. Moreover, an online novel recommendation dataset is built from a well-known online novel reading platform and is released for public use as a benchmark. Experimental results on the dataset demonstrate the effectiveness of NovelNet 1.

Target-oriented opinion words extraction (TOWE) is a subtask of aspect-based sentiment analysis (ABSA). Given a sentence and an aspect term occurring in the sentence, TOWE extracts the corresponding opinion words for the aspect term. TOWE has two types of instance. In the first type, aspect terms are associated with at least one opinion word, while in the second type, aspect terms do not have corresponding opinion words. However, previous researches trained and evaluated their models with only the first type of instance, resulting in a sample selection bias problem. Specifically, TOWE models were trained with only the first type of instance, while these models would be utilized to make inference on the entire space with both the first type of instance and the second type of instance. Thus, the generalization performance will be hurt. Moreover, the performance of these models on the first type of instance cannot reflect their performance on entire space. To validate the sample selection bias problem, four popular TOWE datasets containing only aspect terms associated with at least one opinion word are extended and additionally include aspect terms without corresponding opinion words. Experimental results on these datasets show that training TOWE models on entire space will significantly improve model performance and evaluating TOWE models only on the first type of instance will overestimate model performance.

As a key component in simultaneous localization and mapping (SLAM) system, loop closure detection (LCD) eliminates the accumulated errors by recognizing previously visited places. In recent years, deep learning methods have been proved effective in LCD. However, most of the existing methods do not make good use of the useful information provided by monocular images, which tends to limit their performance in challenging dynamic scenarios with partial occlusion by moving objects. To this end, we propose a novel workflow, which is able to combine multiple information provided by images. We first introduce semantic information into LCD by developing a local-aware Class Activation Maps (CAMs) weighting method for extracting features, which can reduce the adverse effects of moving objects. Compared with previous methods based on semantic segmentation, our method has the advantage of not requiring additional models or other complex operations. In addition, we propose two effective temporal constraint strategies, which utilize the relationship of image sequences to improve the detection performance. Moreover, we propose to use the keypoint matching strategy as the final detector to further refuse false positives. Experiments on four publicly available datasets indicate that our approach can achieve higher accuracy and better robustness than the state-of-the-art methods.

Steganographer detection task is to identify criminal users, who attempt to conceal confidential information by steganography methods, among a large number of innocent users. The significant challenge of the task is how to collect the evidences to identify the guilty user with suspicious images, which are embedded with secret messages generating by unknown steganography and payload. Unfortunately, existing methods for steganalysis were served for the binary classification. It makes them harder to classify the images with different kinds of payloads, especially when the payloads of images in test dataset have not been provided in advance. In this paper, we propose a novel steganographer detection method based on multiclass deep neural networks. In the training stage, the networks are trained to classify the images with six types of payloads. The networks can preserve even strengthen the weak stego signals from secret messages in much larger receptive filed by virtue of residual and dilated residual learning. In the inference stage, the learnt model is used to extract the discriminative features, which can capture the difference between guilty users and innocent users. A series of empirical experimental results demonstrate that the proposed method achieves good performance in spatial and frequency domains even though the embedding payload is low. The proposed method achieves a higher level of robustness of inter-steganographic algorithms and can provide a possible solution to address the payload mismatch problem

According to consciousness involvement, human’s learning can be roughly classified into explicit learning and implicit learning. Contrasting strongly to explicit learning with clear targets and rules, such as our school study of mathematics, learning is implicit when we acquire new information without intending to do so. Research from psychology indicates that implicit learning is ubiquitous in our daily life. Moreover, implicit learning plays an important role in human visual perception. But in the past 60 years, most of the well-known machine-learning models aimed to simulate explicit learning while the work of modeling implicit learning was relatively limited, especially for computer vision applications. This article proposes a novel unsupervised computational model for implicit visual learning by exploring dissipative system, which provides a unifying macroscopic theory to connect biology with physics. We test the proposed Dissipative Implicit Learning Model (DILM) on various datasets. The experiments show that DILM not only provides a good match to human behavior but also improves the explicit machine-learning performance obviously on image classification tasks.

The large-scale video data on the web contain a lot of semantics, which are an important part of semantic web. Video descriptors can usually represent somewhat the semantics. Thus, they play a very important role in web multimedia content analysis, such as Scale-invariant feature transform (SIFT) feature. In this paper, we proposed a new video descriptor, called a temporal-compress and shorter SIFT(TC-S-SIFT) which can efficiently and effectively represent the semantics of web videos. By omitting the least discriminability orientation in three stages of standard SIFT on every representative frame, the dimensions of the shorter SIFT are reduced from 128-dimension to 96-dimension to save space storage. Then, the SIFT can be compressed by tracing SIFT features on video temporal domain, which highly compress the quantity of local features to reduce visual redundancy, and keep basically the robustness and discrimination. Experimental results show our method can yield comparable accuracy and compact storage size.

Image recognition with incomplete data is a well-known hard problem in multimedia content analysis. This paper proposes a novel deep learning technique called semiconducting bilinear deep belief networks (SBDBN) by referencing human's visual cortex and intelligent perception. Inheriting from deep models, SBDBN simulates the laminar structure of human's cerebral cortex and the neural loop in human's visual areas. To address the special difficulties of image recognition with incomplete data, we design a novel second-order deep architecture with semiconducting restricted boltzmann machines. Moreover, two peaks activation of human's perception is implemented by three learning stages of semiconducting bilinear discriminant initialization, greedy layer-wise reconstruction, and global fine-tuning. Owing to exploiting the embedding information according to the reliable features rather than any completion of missing features, the proposed SBDBN has demonstrated outstanding recognition ability on two standard datasets and one constructed dataset, comparing with both incomplete image recognition techniques and existing deep learning models.

Detection and description of local features are a classical problem in image processing and multimedia content analysis. Based on the in homogeneity of visual orientation in human visual system, we propose a novel algorithm S-SIFT to detect and describe local image features. In three stages of S-SIFT, the information from the least discriminability orientation is omitting. Compared with the standard SIFT algorithm, S-SIFT has lower dimension and provides a faster key point matching. Experiments on the standard dataset demonstrate that our algorithm yields comparable or even better results for feature detection and matching tasks.

Water reflection, a kind of typical imperfect reflection symmetry problem, plays an important role in image content analysis. However, existing techniques of symmetry recognition cannot recognize water reflection images correctly because of the complex and various distortions caused by water wave. To address this difficulty, we construct a novel feature space which is composed of motion blur invariant moments. Moreover, we propose an efficient detection algorithm to determine the reflection axis in images with water reflection. By experimenting on real image dataset with different tasks, the proposed techniques demonstrate impressive results in the water reflection image classification, the reflection axis detection, and the retrieval of the images with water reflection.