The Few-Shot Fine-Grained Multimodal Named Entity Recognition task (FewFMNER) aims to recognize the named entities and their fine-grained types in the text based on the social multimedia content of images and texts. With limited labeled data, FewFMNER can more accurately understand the semantics of multimodal content and support downstream applications in social networks. However, as the number of fine-grained types increases, two challenges will arise: (1) Entity boundary ambiguity and concept misjudgment. (2) Semantic confusion exists in fine-grained types. Therefore, we propose a novel Multi-agent Concept Check and Dual-path Reasoning (MCCDR) framework: (1) Entity boundary detection: Recognize named entities and filter concepts to obtain candidate named entity boundaries. (2) Entity type classification: Use dual-path reasoning and type definition to alleviate semantic confusion, and utilize a self-consistency strategy to fuse (entity, fine-grained type) pairs from two perspectives. The experiment results on the Twitter FMNERG dataset show that MCCDR is 6% to 10% higher than the baseline and achieves state-of-the-art results on the FewFMNER task.

Sketch-based 3D shape retrieval (SBSR) aims to retrieve 3D shapes using hand-drawn sketches as query inputs. Although existing SBSR methods have achieved promising results, several challenges still require further investigation. First, most existing approaches usually leverage simple aggregation schemes, often failing to capture the intrinsic relationships between views, which limits the effectiveness of 3D shape feature extraction. Second, conventional SBSR primarily focuses on instance-level alignment while ignoring multi-level alignment, which may neglect complex hierarchical relationships. To address these limitations, we propose a novel Multi-level Encoding with Hierarchical Alignment (MEHA) method for SBSR. Specifically, we adopt spatial encoding and view encoding for multiple views of 3D shapes. The proposed aggregation scheme then integrates these multi-level embedded local features to enhance the representation of 3D shape features. Considering the complexity of 3D shapes, MEHA adopts a two-stage training process: the first stage focuses on learning 3D shape features, while the second stage emphasizes modality alignment. Furthermore, we introduce a hierarchical alignment strategy that bridges the modality gap through instance-level, prototype-level, and centre-level alignment. Extensive experiments on two public benchmark datasets demonstrate the superiority of our method, showing that MEHA outperforms the state-of-the-art baselines.

Multi-view clustering aims at partitioning data into their underlying categories by mining shared and complementary information conveyed by different views. Although the integration of deep learning and disentanglement learning has markedly improved clustering performance, our analysis reveals two fundamental limitations in existing approaches: inadequate separation between view-shared and view-exclusive features; and the negative effects of clustering-irrelevant information on feature decoupling. To tackle these issues, we present a novel Disentangled Feature Learning Network (DFL-Net), which utilizes a progressive learning framework to systematically disentangle features. DFL-Net initially establishes view-shared representations through semantic disparity minimization, followed by the construction of orthogonal feature subspaces using cross-view and intra-view independence constraints to isolate view-specific features. Subsequently, DFL-Net enforces clustering consistency across views to adaptively eliminate irrelevant information, thus enhancing the overall effectiveness of disentanglement learning. The framework introduces two significant innovations: a comprehensive feature independence criterion that concurrently reduces intra-view and cross-view feature dependencies, and an irrelevance filtering mechanism that ensures cross-view clustering consistency. Extensive experiments on benchmark datasets demonstrate the superior performance of DFL-Net compared to state-of-the-art methods.

Heuristic algorithms have been developed to find approximate solutions for high-utility itemset mining (HUIM) problems that compensate for the performance bottlenecks of exact algorithms. However, heuristic algorithms still face the problem of long runtime and insufficient mining quality, especially for large transaction datasets with thousands to tens of thousands of items and up to millions of transactions. To solve these problems, a novel GPU-based efficient parallel heuristic algorithm for HUIM (PHA-HUIM) is proposed in this paper. The iterative process of PHA-HUIM consists of three main steps: the search strategy, fitness evaluation, and ring topology communication. The search strategy and ring topology communication are designed to run in constant time on GPU. The parallelism of fitness evolution helps to substantially accelerate the algorithm. To improve the mining quality, a multi-start strategy with an unbalanced allocation strategy is employed in the search process. Ring topology communication is adopted to maintain population diversity. A load balancing strategy is introduced to reduce the thread divergence to improve the parallel efficiency. The experimental results on nine large datasets show that PHA-HUIM outperforms state-of-the-art HUIM algorithms in terms of speedup performance, runtime, and mining quality.

Bayesian network structure learning (BNSL) from data is an NP-hard problem. Genetic algorithms are powerful for solving combinatorial optimization problems, but the lack of effective guidance results in slow convergence and low accuracy regarding BNSL. To address this problem, we propose a mutual information (MI) guided genetic algorithm (MIGA) for BNSL in this paper, which uses MI to effectively search BN structures. In the initialization phase of MIGA, the population is generated by adding additional constraints based on MI to reach a higher score without losing diversity. By employing normalized MI and defining the population support, the potential dominance in the population can be identified and then used to design a novel crossover operator in order to preserve the dominant genes with a higher probability. Moreover, with the guidance of MI for removing loops from the structures, infeasible solutions can be handled in a straightforward and practical way. The proposed MIGA is evaluated on eleven well-known benchmark datasets and compared with four GA-based methods and four other state-of-the-art BNSL algorithms. Experimental results show that MIGA outperforms the compared algorithms in convergence and learning accuracy.

Video action recognition aims at classifying the action category in given videos. In general, semantic-relevant video frame pairs reflect significant action patterns such as object appearance variation and abstract temporal concepts like speed, rhythm, and so on. However, existing action recognition approaches tend to holistically extract spatiotemporal features. Though effective, there is still a risk of neglecting the crucial action features occurring across frames with a long-term temporal span. Motivated by this, in this article, we propose to perceive actions via frame pairs directly and devise a novel Nest Structure with frame pairs as basic units. Specifically, we decompose a video sequence into all possible frame pairs and hierarchically organize them according to temporal frequency and order, thus transforming the original video sequence into a Nest Structure. Through naturally decomposing actions, the proposed structure can flexibly adapt to diverse action variations such as speed or rhythm changes. Next, we devise a Temporal Pair Analysis module (TPA) to extract discriminative action patterns based on the proposed Nest Structure. The designed TPA module consists of a pair calculation part to calculate the pair features and a pair fusion part to hierarchically fuse the pair features for recognizing actions. The proposed TPA can be flexibly integrated into existing backbones, serving as a side branch to capture various action patterns from multi-level features. Extensive experiments show that the proposed TPA module can achieve consistent improvements over several typical backbones, reaching or updating CNN-based SOTA results on several challenging action recognition benchmarks.

Heuristic algorithms have been developed to find approximate solutions for high-utility itemset mining (HUIM) problems that compensate for the performance bottlenecks of exact algorithms. However, heuristic algorithms still face the problem of long runtime and insufficient mining quality, especially for large transaction datasets with thousands to tens of thousands of items and up to millions of transactions. To solve these problems, a novel GPU-based efficient parallel heuristic algorithm for HUIM (PHA-HUIM) is proposed in this paper. The iterative process of PHA-HUIM consists of three main steps: the search strategy, fitness evaluation, and ring topology communication. The search strategy and ring topology communication are designed to run in constant time on GPU. The parallelism of fitness evolution helps to substantially accelerate the algorithm. A new data structure with a sort-mapping strategy is proposed to enhance the search ability and reduce memory usage. To improve the mining quality, a multi-start strategy with an unbalanced allocation strategy is employed in the search process. Ring topology communication is adopted to maintain population diversity. A load balancing strategy is introduced to reduce the thread divergence to improve the parallel efficiency. The experimental results on nine large datasets show that PHA-HUIM outperforms state-of-the-art HUIM algorithms in terms of speedup performance, runtime, and mining quality.

Most studies on pattern mining have considered only one pattern, such as frequent pattern or high-utility pattern, which is difficult to meet the increasingly diverse needs of users. In this paper, a novel multi-objective problem model for high quality pattern mining (HQPM) is proposed, where the objectives are support, occupancy, and utility. In order to solve the proposed three-objective problem efficiently, an improved multi-objective evolutionary algorithm for HQPM (MOEA-PM) is proposed with two kinds of population initialization strategies and an auxiliary tool. Experimental results on real-world datasets show that the proposed three-objective problem model with the MOEA-PM algorithm can discover patterns that are both frequently occurring and has a high utility in the transaction datasets, while at the same time being relatively complete. The proposed algorithm outperforms the state-of-the-art in terms of efficiency, quality, and convergence speed.

Cross-modal hashing (CMH) has recently received increasing attention with the merit of speed and storage in performing large-scale cross-media similarity search. However, most existing cross-media approaches utilize the batch-based mode to update hash functions, without the ability to efficiently handle the online streaming multimedia data. Online hashing can effectively address the preceding issue by using the online learning scheme to incrementally update the hash functions. Nevertheless, the existing online CMH approaches still suffer from several challenges, such as (1) how to efficiently and effectively utilize the supervision information, (2) how to learn more powerful hash functions, and (3) how to solve the binary constraints. To mitigate these limitations, we present a novel online hashing approach named ONION ( ON line semant I c aut O encoder hashi N g). Specifically, it leverages the semantic autoencoder scheme to establish the correlations between binary codes and labels, delivering the power to obtain more discriminative hash codes. Besides, the proposed ONION directly utilizes the label inner product to build the connection between existing data and newly coming data. Therefore, the optimization is less sensitive to the newly arriving data. Equipping a discrete optimization scheme designed to solve the binary constraints, the quantization errors can be dramatically reduced. Furthermore, the hash functions are learned by the proposed autoencoder strategy, making the hash functions more powerful. Extensive experiments on three large-scale databases demonstrate that the performance of our ONION is superior to several recent competitive online and offline cross-media algorithms.

Bayesian network structure learning (BNSL) from data is an NP-hard problem. Genetic algorithms are powerful for solving combinatorial optimization problems, but the lack of effective guidance results in slow convergence and low accuracy regarding BNSL. To address this problem, we propose a mutual information (MI) guided genetic algorithm (MIGA) for BNSL in this paper, which uses MI to effectively search BN structures. In the initialization phase of MIGA, the population is generated by adding additional constraints based on MI to reach a higher score without losing diversity. By employing normalized MI and defining the population support, the potential dominance in the population can be identified and then used to design a novel crossover operator in order to preserve the dominant genes with a higher probability. Moreover, with the guidance of MI for removing loops from the structures, infeasible solutions can be handled in a straightforward and practical way. The proposed MIGA is evaluated on eleven well-known benchmark datasets and compared with four GA-based methods and four other state-of-the-art BNSL algorithms. Experimental results show that MIGA outperforms the compared algorithms in convergence and learning accuracy.

Due to the explosive growth of multimedia data in recent years, cross-media hashing (CMH) approaches have recently received increasing attention. To learn the hash codes, most existing supervised CMH algorithms employ the strict binary label information, which has small margins between the incorrect labels (0) and the true labels (1), increasing the risk of classification error. Besides, most existing CMH approaches are one-stage algorithms, in which the hash functions and binary codes can be learned simultaneously, complicating the optimization. To avoid NP-hard optimization, many approaches utilize a relaxation strategy. However, this optimisation trick may cause large quantization errors. To address this, we present a novel tWo-stAge discreTe Cross-media Hashing method based on smooth matrix factorization and label relaxation, named WATCH. The proposed WATCH controls the margins adaptively by the novel label relaxation strategy. This innovation reduces the quantization error significantly. Besides, WATCH is a two-stage model. In stage 1, we employ a discrete smooth matrix factorization model. Then, the hash codes can be generated discretely, reducing the large quantization loss greatly. In stage 2, we adopt an effective hash function learning strategy, which produces more effective hash functions. Comprehensive experiments on several datasets demonstrate that WATCH outperforms some state-of-the-art methods.

Given the merits in high computational efficiency and low storage cost, hashing techniques have been widely studied in cross-media retrieval. Existing methods usually adopt the equal length encoding scheme to represent the multimedia data. However, the strictly equal length scheme maybe not optimal because the dimension of different modalities is often various. Besides, there exists other challenges in designing a cross-media retrieval system, e.g., how to address the discrete constraints, how to avoid using the n*n similarity matrix, and how to effectively exploit the discriminative label information. To conquer the above challenges, we propose a novel method, i.e., discrete asymmetric hashing (DAH). Specifically, DAH exploits a flexible model, which can seamlessly deal with equal or unequal length encoding scenarios. Moreover, DAH constructs a supervised semantic embedding framework by jointly minimizing the distance-distance difference and label reconstructing error, significantly reducing the computational complexity. An asymmetric strategy is employed to establish the connection between hash codes and the latent subspace. Furthermore, the hash codes can be learned discretely by the designed optimization algorithm. In the training stage2, a semantic intersection scheme is proposed to learn more powerful hash functions. Experiments show that our DAH is effective in equal and unequal scenarios.

Pattern mining (PM) refers to the process of discovering patterns of interest to users from data. However, most studies have considered only one pattern, such as frequent pattern or high-utility pattern. With the continuous requirement of businesses in various industries, the single-objective PM methods are difficult to meet the increasingly diverse needs of users. In this paper, a multi-objective problem model for high quality pattern mining (HQPM) is proposed, where the objectives are support, occupancy, and utility. In order to solve the proposed three-objective problem efficiently, an improved multi-objective evolutionary algorithm for HQPM (MOEA-PM) is proposed. Two kinds of population initialization strategies are designed, which is used to ensure the population is effectively distributed in the feasible solution space. By taking the properties of the model into consideration, an auxiliary tool is proposed to accelerate the convergence of the algorithm. Experimental results on real-world datasets show that the proposed three-objective problem model with the MOEA-PM algorithm can discover patterns that are both frequently occurring and has a high utility in the transaction datasets, while at the same time being relatively complete. Compared with the state-of-the-art MOEA-based HQPM algorithms, MOEA-PM has better performance in terms of efficiency, quality, and convergence speed.