Preference-Based reinforcement learning (PBRL) learns directly from the preferences of human teachers regarding agent behaviors without needing meticulously designed reward functions. However, existing PBRL methods often learn primarily from explicit preferences, neglecting the possibility that teachers may choose equal preferences. This neglect may hinder the understanding of the agent regarding the task perspective of the teacher, leading to the loss of important information. To address this issue, we introduce the Equal Preference Learning Task, which optimizes the neural network by promoting similar reward predictions when the behaviors of two agents are labeled as equal preferences. Building on this task, we propose a novel PBRL method, Multi-Type Preference Learning (MTPL), which allows simultaneous learning from equal preferences while leveraging existing methods for learning from explicit preferences. To validate our approach, we design experiments applying MTPL to four existing state-of-the-art baselines across ten locomotion and robotic manipulation tasks in the DeepMind Control Suite. The experimental results indicate that simultaneous learning from both equal and explicit preferences enables the PBRL method to more comprehensively understand the feedback from teachers, thereby enhancing feedback efficiency. Project page: https://github.com/FeiCuiLengMMbb/paper_MTPL

Preference-based Reinforcement Learning (PBRL) relies on the efficient collection and use of preference data to train accurate reward functions, enabling agents to learn directly from human preferences. This process allows agents to better understand human intentions while effectively reducing biases inherent in AI systems. The pairwise comparison method gathers diverse preference data, and Seqrank expands preference datasets through transitivity, both fail to establish preference relationships across different rounds of labeling. This limitation can result in fragmented signals and slow convergence toward the optimal policy. To address this, we propose the Global Tree (GTree), a method built on the Seqrank framework that integrates trajectory preferences across multiple rounds, providing a unified representation of global preferences. Moreover, we posit that different trajectory comparison methods offer distinct advantages depending on the task and the stage of training. To fully exploit these strengths, we introduce FLIP. This adaptive strategy dynamically selects either the pairwise method or GTree based on historical performance, optimizing method use for each task and training stage. Our evaluations demonstrate that integrating cross-round preferences accelerates the convergence of the reward function, while the FLIP strategy further enhances learning efficiency and overall performance, thereby enabling agents to better understand human intentions.

Few-shot class-incremental learning (FSCIL) aims to learn novel concepts using limited examples without forgetting. However, most studies focus on solving the catastrophic forgetting problem of FSCIL, while neglecting the poor robustness of these algorithms. For example, introducing just a 10% FGSM attack can result in a decrease of more than 30% in accuracy. To tackle this challenge, we propose the Vital Importance Playback for Robust FSCIL (VIP-FSCIL) method. The effectiveness of this approach stems from its consideration of sample importance to construct a replay exemplar set, as well as the reduction of overfitting risks to adversarial examples through knowledge distillation. Our approach thus enhances the robustness of FSCIL models while preserving their generalization. The experimental results indicate that VIP-FSCIL is able to improve the accuracy of the model in perturbed scenarios by 5-9% on CIFAR100 and CUB200 datasets, compared to other methods.

In a real-world setting, machine needs to continuously recognize new categories without forgetting. However, the number of new categories may be small. For some difficult categories, even humans cannot recognize only based on few-shot examples. To address the above issues, an innovative uncertainty-aware few-shot class incremental learning method (UACL) is proposed, which allows the model to continuously recognize new classes with few-shot examples and identify the classes it cannot recognize currently. Besides, in order to imitate the cognitive way of human beings and improve the continuous representation ability, we propose a pseudo-incremental task construction mechanism based on uncertainty estimation, where the machine learn to recognize from simple to difficult. Further, a large-scale pre-training model is used as an expert system to guide the model to recognize difficult classes. We evaluate our method on three popular benchmark datasets, showing that UACL is state-of-the-art.

Traditional entity and relation extraction (ERE) task aims at extracting (subject, object, predicate) triplets from natural language texts. However, this ignores the prerequisites for some knowledge. CHIP2022 TASK2 first uses condition spans to support causal triplets. We expand it to the conditional entity and relation extraction task (Cond-ERE), the goal of which is to extract condition spans for conditional triplets on the basis of traditional ERE tasks. Currently, the one-step joint extraction model is almost unable to extract condition spans. Therefore, based on the existing models, we present a three-stage pipeline: first use one model for ERE to jointly extract triplets with all unconditional relations, then for each conditional one use a separate model to extract triplets, and finally use the Machine Reading Comprehension (MRC) model to judge whether these triplets with conditional relations are actually conditional or not and extract condition spans at the same time. Experiment results show that our three-stage pipeline can better extract triplets for multiple relations, and has significant advantages to extract condition spans. Besides, The MRC query template we designed further improves the performance of condition span extraction.

Nowadays, with the explosive growth of multimodal reviews on social media platforms, multimodal sentiment analysis has recently gained popularity because of its high relevance to these social media posts. Although most previous studies design various fusion frameworks for learning an interactive representation of multiple modalities, they fail to incorporate sentimental knowledge into inter-modality learning. This pa-per proposes a Multi-channel Attentive Graph Convolutional Network (MAGCN), consisting of two main components: cross-modality interactive learning and sentimental feature fusion. For cross-modality interactive learning, we exploit the self-attention mechanism combined with densely connected graph convolutional networks to learn inter-modality dynamics. For sentimental feature fusion, we utilize multi-head self-attention to merge sentimental knowledge into inter-modality feature representations. Extensive experiments are conducted on three widely-used datasets. The experimental results demonstrate that the proposed model achieves competitive performance on accuracy and F1 scores compared to several state-of-the-art approaches.

Conversational recommender systems discover users' preferences through dialog and make proper recommendations. Previous works fall into task-oriented and sociable conversational recommender systems. However, these works are not interpretable and sociable simultaneously. To address this problem, we propose a conversational recommender system with topic-related preference graph (CRTPG), consisting of a topic-related preference graph (TP-Graph) construction module, a key entity prediction module, and a dialog generation module. The TP-Graph recognizes the user's entity-level preference and keeps preference information for the recent topics. The key entity prediction module provides key entities as explicit content guidance for dialog generation based on TP-Graph. The dialog generation module generates appropriate responses based on the TP-Graph and knowledge related to key entities. TP-Graph and key entity help humans understand the precise information the system makes decisions based on, improving the interpretability of the system. We conduct automatic and human evaluations on the DuRecDial dataset. Experimental results show that CRTPG achieves state-of-the-art results on recommendation and dialog generation.

Building a good feature space is essential for the metric-based few-shot algorithms to recognize a novel class with only a few samples. The feature space is often built by Convolutional Neural Networks (CNNs). However, CNNs primarily focus on local information with the limited receptive field, and the global information generated by distant pixels is not well used. Meanwhile, having a global understanding of the current task and focusing on distinct regions of the same sample for different queries are important for the few-shot classification. To tackle these problems, we propose the Cross Non-Local Neural Network (CNL) for capturing the long-range dependency of the samples and the current task. CNL extracts the task-specific and context-aware features dynamically by strengthening the features of the sample at a position via aggregating information from all positions of itself and the current task. To reduce losing important information, we maximize the mutual information between the original and refined features as a constraint. Moreover, we add a task-specific scaling to deal with multi-scale and task-specific features extracted by CNL. We conduct extensive experiments for validating our proposed algorithm, which achieves new state-of-the-art performances on two public benchmarks.

Few-shot learning can recognize a novel category based on only a few samples because it learns to learn from a lot of labeled samples during the training process. When data is insufficient, the performance is affected. And it is expensive to obtain a large-scale finegrained dataset with annotation. In this paper, we adopt domain- specific knowledge to fill the gap of insufficient annotated data. We propose a cross-modal knowledge distillation (CMKD) framework to do fine-grained one-shot classification and propose the Spatial Relation Loss (SRL) to transfer cross-modal information, which can tackle the semantic gap between multimodal features. The teacher network distills the spatial relationship of the samples as a soft target for training a unimodal student network. Notably, the student network makes predictions only based on a few samples without any external knowledge in the application. This model-agnostic framework will be well adapted to other few-shot models. Extensive experimental results on benchmarks demonstrate that CMKD can make full use of cross-modal knowledge in image and text few-shot classification. CKMD improves the performances of the student networks significantly, even if it is a state-of-the-art student network.

The document layout analysis (DLA) aims to split the document image into different interest regions and understand the role of each region, which has wide application such as optical character recognition (OCR) systems and document retrieval. However, it is a challenge to build a DLA system because the training data is very limited and lacks an efficient model. In this paper, we propose an end-to-end united network named Dynamic Residual Fusion Network (DRFN) for the DLA task. Specifically, we design a dynamic residual feature fusion module which can fully utilize low-dimensional information and maintain high-dimensional category information. Besides, to deal with the model overfitting problem that is caused by lacking enough data, we propose the dynamic select mechanism for efficient fine-tuning in limited train data. We experiment with two challenging datasets and demonstrate the effectiveness of the proposed module.

Automatic analysis of highly crowded people has attracted extensive attention from computer vision research. Previous approaches for crowd counting have already achieved promising performance across various benchmarks. However, to deal with the real situation, we hope the model run as fast as possible while keeping accuracy. In this paper, we propose a compact convolutional neural network for crowd counting which learns a more efficient model with a small number of parameters. With three parallel filters executing the convolutional operation on the input image simultaneously at the front of the network, our model could achieve nearly real-time speed and save more computing resources. Experiments on two benchmarks show that our proposed method not only takes a balance between performance and efficiency which is more suitable for actual scenes but also is superior to existing light-weight models in speed.

The small inter-class variance and the large intra-class variance make the few-shot and fine-grained image classification more difficult because the machine cannot obtain enough information from only a few images. The external knowledge contains more semantics and can support the model to extract important features, while most of existing few-shot learning algorithms only focus on leveraging the visual features from images, little attention has been paid to the cross-modal external knowledge. In this paper, we propose a knowledge-based fine-grained classification mechanism for few-shot learning, which can overcome the difficulty of only obtaining limited and discriminative features from unimodal samples. We extract the visual features and the knowledge features from textual descriptions and a domain-specific knowledge graph at global and local levels to build the semantic space. To tackle the gap between multimodal features, we propose a mirror framework, named Mirror Mapping Network (MMN), to map the multimodal features into the same semantic space with two directions. Extensive experimental results show that our method outperforms the state-of-the-art.

Segmentation-based approaches have enabled state-of-the-art performance in long or curved text detection tasks. However, false detection still is a challenge when two text instances are close to each other. To address this problem, in this paper, we propose a Text Contour Attention Text Detector (TCATD), which can locate scene text with arbitrary orientation and shape accurately. Different from previous work, TCATD focus on text contour map (TC), text center intensity map (TCI) and text kernel maps (TK). The TC can introduce text contour information, the TCI can help to learn the accurate text segmentation and the TK can generate the complete shape of text instances. Besides, we propose a Text Contour Attention Module to deal with contour information. After the Text Contour Attention Module, TC, TCI and TK will be obtained. Extensive experiments on ICDAR2015, CTW1500 and Total-Text demonstrate that the proposed method achieves the state-of-the-art performance.

Crowd counting, i.e., estimation number of the pedestrian in crowd images, is emerging as an essential research problem with the public security applications. The density-based method of crowd counting still has some challenges, such as lack of perspective information in density map and background noise. Current models often misjudge background noise as a person and the ground truth density map widely used now is not so accurate. In this paper, we present a novel approach to help generate a higher quality density map. On the one hand, we eliminate the apparent mistakes in the density map with the help of a semantic segmentation model, which provides more information about fine-granted negative samples. On the other hand, we modify the density map to make sure it maintains a natural attribute. The experimental results prove the effectiveness of our method for crowd counting models, especially in uneven distribution monitoring scenario.

Crowd counting, i.e. count the number of people in a crowded visual space, is emerging as an essential research problem with public security. A key in the design of the crowd counting system is to create a stable and accurate robust model, which requires to process on the feature channels of the counting network. In this study, the authors present a featured channel enhancement (FCE) block for crowd counting. First, they use a feature extraction unit to obtain the information of each channel and encodes the information of each channel. Then use a non‐linear variation unit to deal with the encoded channel information, finally, normalise the data and affixed to each channel separately. With the use of the FCE, the positive characteristic channel can be enhanced and weak or negative channel information can be suppressed. The authors successfully incorporate the FCE with two compact networks on the standard benchmarks and prove that the proposed FCE achieves promising results.

Crowd counting, i.e., estimation number of the pedestrian in crowd images, is emerging as an important research problem with the public security applications. A key component for the crowd counting systems is the construction of counting models which are robust to various scenarios under facts such as camera perspective and physical barriers. In this paper, we present an adaptive scenario discovery framework for crowd counting. The system is structured with two parallel pathways that are trained with different sizes of the receptive field to represent different scales and crowd densities. After ensuring that these components are present in the proper geometric configuration, a third branch is designed to adaptively recalibrate the pathway-wise responses by discovering and modeling the dynamic scenarios implicitly. Our system is able to represent highly variable crowd images and achieves state-of-the-art results in two challenging benchmarks.

The accuracy of OCR is usually affected by the quality of the input document image and different kinds of marred document images hamper the OCR results. Among these scenarios, the low-resolution image is a common and challenging case. In this paper, we propose the cascaded networks for document image super-resolution. Our model is composed by the Detail-Preserving Networks with small magnification. The loss function with perceptual terms is designed to simultaneously preserve the original patterns and enhance the edge of the characters. These networks are trained with the same architecture and different parameters and then assembled into a pipeline model with a larger magnification. The low-resolution images can upscale gradually by passing through each Detail-Preserving Network until the final high-resolution images. Through extensive experiments on two scanning document image datasets, we demonstrate that the proposed approach outperforms recent state-of-the-art image super-resolution methods, and combining it with standard OCR system lead to signification improvements on the recognition results.

Math expression image retrieval concerns not only visual features but also high-level semantic understanding. Considering math expression image retrieval as traditional content-based image retrieval may suffer the layout misunderstanding, as math expressions with same symbols but different layouts may be interpreted as different meaning. In this paper, we propose a novel retrieval indexing framework for math expression retrieval, namely Scanner-Recognizer-Embedding (SRE) framework. The math expression images passed through SRE are projected into a low dimension semantic space. Retrieval based on embedded semantic vectors is fast and accurate. Experiments on a math expression database demonstrate that the SRE framework outperforms state-of-the-art image-based features.

In relation extraction tasks, distant supervision methods expand dataset by aligning entity pairs in different knowledge bases and completing the relations between two entities. However, these methods ignore the fact that sentences labels generated by distant supervision methods with high confidence are often incorrect in the real world called Unknown Unknowns (UUs). To deal with this challenge, we propose a crowdsourcing based human-in-the-loop denoising framework which iteratively discovers UUs and corrects them by crowdsourcing to better extract relations. During each epoch of iterations, we choose one sentence bag and repeat two steps: Firstly, attention based Long Short-Term Memory network is applied as a selector to discover potential UUs. Secondly, these UUs are annotated by crowdsourcing with two answer collecting strategies and fed back into selector as positive samples. Until the accuracy of selector reaches a threshold, all annotated samples are added into relation classifier as cleaned train set and framework moves on to next epoch with new sentence bags. The experiments on the New York Times dataset and analysis of potential UUs demonstrate that our framework denoise the dataset and outperforms all the baselines on distant supervision relation extraction tasks.

Crowdsourcing is quite cheap and effective to get a data set labeled by multiple annotators in a short amount of time. Although there are many traditional methods focusing on quality control of crowdsourcing, little pays close attention to the intrinsic ambiguity in dataset, which is hard for workers to make the right decision. In this paper, we introduce DCRB, a collaborative crowdsourcing framework that can obtain high quality output even for ambiguous tasks. We enable users' disagreement and encourage them to provide in-depth explanations when the disagreement occurs. We use these explanations to analyze the ambiguous tasks and design collaborative crowdsourcing to improve the output. In addition, we also design “reward brave” incentive mechanism to encourage users' valuable explanations. The experimental results show that our method significantly improves the accuracy of crowdsourcing, especially for those ambiguous crowdsourcing tasks.

In micro-task crowdsourcing markets such as Amazon's Mechanical Turk, how to obtain high quality result without exceeding the limited budgets is one main challenge. The existing theory and practice of crowdsourcing suggests that uneven task difficulty plays a crucial role to task quality. Yet, it lacks a clear identifying method to task difficulty, which hinders effective and efficient execution of micro-task crowdsourcing. This paper explores the notion of task difficulty and its influence to crowdsourcing, and presents a difficulty-based crowdsourcing method to optimize the crowdsourcing process. We firstly identify task difficulty feature based on a local estimation method in the real crowdsourcing context, followed by proposing an optimization method to improve the accuracy of results, while reducing the overall cost. We conduct a series of experimental studies to evaluate our method, which show that our difficulty-based crowdsourcing method can accurately identify the task difficulty feature, improve the quality of task performance and reduce the cost significantly, and thus demonstrate the effectiveness of task difficulty as task modeling property.

In this paper, we designed and implemented a lightweight mobile crowdsourcing platform called WeCrowd. What makes it distinct from other mobile crowdsourcing applications is that, it is based on WeChat and enables users to post and work on crowdsourcing tasks without setup process, which can greatly save storage space, speed up crowd work and also be convenient for users to take part in tasks. Based on this platform, we conducted a preliminary study to understand how this lightweight crowdsourcing platform worked in reality. The experiment results affirm that WeCrowd provides meaningful insights for mobile crowdsourcing, and enables mobile users to take part in tasks with more freedom, enthusiastic and high quality users experience. We end by some design implications for mobile crowdsourcing.

Many of the recent algorithms have been developed to improve the various aspects of collaborative filtering recommender systems, however, most of them do not take the sectional data of users and items information or characteristic into account. This paper, we present a new improved collaborative filtering based on item similarity modified and item common ratings which take full advantage of the sectional data of item-user matrix information to modify the similarity calculation and rating prediction. Extensive experiments have been conducted on two different dataset to analyze our proposal approach. The results show that our approach can improve the prediction accuracy of the item-based collaborative filtering not only on different neighbors, but also on different training ratio data set.