Existing federated clustering methods typically assume that either all clients supply the same type of single-modal/multi-modal data, or that different clients provide various modalities describing the same object. However, a prevalent real-world scenario involves clients contributing data from diverse and unrelated modalities. Addressing the challenge of uncovering clustering patterns from such heterogeneous modality data distributed across distinct clients is crucial. In this paper, we propose a novel cross-modal feature decoupling-based mixed federated clustering model. To address client heterogeneity, we introduce a cross-modal feature decoupling module for each client, designed to decouple modality-agnostic and modality-specific features through distinct encoders.Only the modality-agnostic encoder parameters and clustering centers of each client are transmitted to the server. This enables the server to aggregate various modality-agnostic encoders, effectively discovering the global clustering structure while avoiding interference from modality-specific noise. Moreover, we develop a global consistent complementary clustering module to integrate the complementary clustering centers from various clients. The global clustering centers are then dispatched back to clients to guide and calibrate their local clustering models. Experimental results on three public datasets show the superiority of the proposed model compared to classic federated clustering methods.

Retinal vessel segmentation underpins computer-assisted screening and monitoring of ocular and systemic disease. While encoder–decoder networks such as U-Net are widely used, their behavior is strongly shaped by the training objective. This work presents a controlled empirical study of loss functions for vessel segmentation using a U-Net architecture that employs strided convolutions in the encoder, together with a consistent pre-processing pipeline based on morphological enhancement and principal component analysis. We compare cross-entropy, weighted cross-entropy, and Dice losses on the DRIVE and STARE datasets under identical settings, reporting pixel-wise and overlap-based measures to reflect both detection and spatial agreement. The configuration with weighted cross-entropy provides a balanced outcome, achieving sensitivity and accuracy of 0.873 and 0.969 on DRIVE, and 0.821 and 0.961 on STARE. Rather than proposing architectural novelty, the contribution of this study is a reproducible data-driven comparison that clarifies the tradeoffs each loss imposes on recall, specificity, and boundary fidelity, offering practical guidance for selecting objectives in retinal vessel segmentation.

Abstract Large Language Models (LLMs) perform well in tabular prediction tasks with limited data, using their ability to understand instructions and learn from examples. However, their reliance on training data can perpetuate social biases, leading to unfair outcomes and disproportionately impacting underprivileged groups. Addressing these biases is critical as LLMs see wider adoption in tabular data tasks. Traditional bias mitigation strategies in machine learning, such as balancing datasets or applying fairness constraints, are less effective with LLMs. Our research explores whether bias in LLMs for tabular data classification can be mitigated. Through extensive experiments, we found that using LLMs in a zero-shot setting introduces bias, and in-context learning slightly reduces these disparities. Meanwhile, fine-tuning and retrieval augmented generation show limited effectiveness in bias mitigation. We introduced three instruction-based prompting strategies to enhance fairness: Fair Prompting , Generalised Prompting , and Descriptive Prompting . The results show that combining descriptive prompting with in-context learning, particularly the Equal Samples Across Demographics approach, consistently narrowed fairness gaps across demographic subgroups, and yielded accuracy gains ranging from 3.27% to 15.05% across multiple datasets, underscoring its potential as a promising strategy in the ongoing effort to mitigate bias in LLMs.

Graph neural networks (GNNs) have demonstrated remarkable effectiveness in various graph representation learning tasks. However, most of existing methods achieve the information extraction by the simple fusion of local and global information, which is a coarse-grained and static way. This may hinder the establishment of a dynamic and collaborative interaction between global and local information, which is crucial for comprehensively understanding graph data. To address this challenge, we propose a novel framework called comprehensive graph representation learning (ComGRL). ComGRL integrates local information into global information to derive powerful representations. It achieves this by implicitly smoothing local information through flexible graph contrastive learning, ensuring reliable representations for subsequent global exploration. Then ComGRL transfers the locally derived representations to a multihead self-attention module, enhancing their discriminative ability by uncovering diverse and rich global correlations. To achieve dynamic transformation between global and local information under self-supervision with pseudo-labels, ComGRL employs a triple sampling strategy to construct mixed node pairs and applies reliable Mixup augmentation across attributes and structure for the main frameworks. This approach broadens the receptive field and facilitates coordination between local and global representation learning, enabling them to reinforce each other. Experimental results across six widely used graph datasets demonstrate that ComGRL achieves excellent performance in node classification tasks. The code could be available athttps://github.com/JinluWang1002/ComGRL.

Due to multidimensional heterogeneity, Multimodal Federated Learning (MMFL) confronts fundamental challenges including modality incongruence, modality agnosticism, and modality incompleteness. Existing methods face a trilemma: leveraging external data with privacy risks, isolating features to restrict cross-modal interaction, or incurring high overhead from complex graph-based coordination, all culminating in suboptimal performance. In this paper, we propose a Modality-Agnostic Hybrid Federated Learning (MA-HyFL) framework that synergistically integrates unimodal and multimodal federated processes in modality-agnostic scenarios. Specifically, a bidirectional cross-modal knowledge distillation is employed to promote comprehensive collaboration at inter-client and intra-client levels, enabling robust knowledge transfer among heterogeneous modalities. A reinforcement learning-based aggregation mechanism is further introduced to orchestrate federated workflows through reward-driven policy optimization, dynamically integrating contributive client selection and adaptive aggregation weighting for closed-loop decision-making. Extensive experiments show that MA-HyFL significantly outperforms the other baseline methods in four realistic real-world applications, each exhibiting varying degrees of statistical heterogeneity and missing rates.

Multimodal knowledge graph completion (MMKGC) enhances the precision and breadth of application of knowledge graphs by integrating rich data from various modalities, steadily increasing its appeal in the research community. Prior studies mainly focus on the common representation of different modalities while neglecting the different and complementary features. On the contrary, some works tend to model triples of each modality separately while overlooking the similarities between modalities. It is challenging to associate the heterogeneous modalities effectively for MMKGC. In this article, we introduce a novel MMKGC framework by cross-modal interaction with similarity-enhancing and difference-embracing (CISEDE), which leverages both the similarities and differences among multimodal entities by a proposed cross-modal interaction mechanism. In the cross-modal interaction, multihead attention is employed to enhance similarity information from multimodal entities and embrace different information by linking various modal triples. Through relation-guided fusion, the modal triples are decoded and merged for MMKGC. The experimental results on three commonly used datasets, FB15k-237, WN9, and WN18RR, show that the proposed method achieves state-of-the-art performance.

Radiology report generation (RRG) models typically focus on individual exams, often overlooking the integration of historical visual or textual data, which is crucial for patient follow-ups. Traditional methods usually struggle with long sequence dependencies when incorporating historical information, but large language models (LLMs) excel at in-context learning, making them well-suited for analyzing longitudinal medical data. In light of this, we propose a novel Historical-Constrained Large Language Models (HC-LLM) framework for RRG, empowering LLMs with longitudinal report generation capabilities by constraining the consistency and differences between longitudinal images and their corresponding reports. Specifically, our approach extracts both time-shared and time-specific features from longitudinal chest X-rays and diagnostic reports to capture disease progression. Then, we ensure consistent representation by applying intra-modality similarity constraints and aligning various features across modalities with multimodal contrastive and structural constraints. These combined constraints effectively guide the LLMs in generating diagnostic reports that accurately reflect the progression of the disease, achieving state-of-the-art results on the Longitudinal-MIMIC dataset. Notably, our approach performs well even without historical data during testing and can be easily adapted to other multimodal large models, enhancing its versatility.

Optimal transport (OT) theory has attracted much attention in machine learning and signal processing applications. OT defines a notion of distance between probability distributions of source and target data points. A crucial factor that influences OT-based distances is the ground metric of the embedding space in which the source and target data points lie. In this work, we propose to learn a suitable latent ground metric parameterized by a symmetric positive definite matrix. We use the rich Riemannian geometry of symmetric positive definite matrices to jointly learn the OT distance along with the ground metric. Empirical results illustrate the efficacy of the learned metric in OT-based domain adaptation.

Graph Neural Networks (GNNs) are proficient in graph representation learning and achieve promising performance on versatile tasks such as node classification and link prediction. Usually, a comprehensive hyperparameter tuning is essential for fully unlocking GNN's top performance, especially for complicated tasks such as node classification on large graphs and long-range graphs. This is usually associated with high computational and time costs and careful design of appropriate search spaces. This work introduces a graph-conditioned latent diffusion framework (GNN-Diff) to generate high-performing GNNs based on the model checkpoints of sub-optimal hyperparameters selected by a light-tuning coarse search. We validate our method through 166 experiments across four graph tasks: node classification on small, large, and long-range graphs, as well as link prediction. Our experiments involve 10 classic and state-of-the-art target models and 20 publicly available datasets. The results consistently demonstrate that GNN-Diff: (1) boosts the performance of GNNs with efficient hyperparameter tuning; and (2) presents high stability and generalizability on unseen data across multiple generation runs. The code is available at https://github.com/lequanlin/GNN-Diff.

Graph Neural Networks (GNNs) have emerged as fundamental tools for a wide range of prediction tasks on graph-structured data. Recent studies have drawn analogies between GNN feature propagation and diffusion processes, which can be interpreted as dynamical systems. In this paper, we delve deeper into this perspective by connecting the dynamics in GNNs to modern Koopman theory and its numerical method, Dynamic Mode Decomposition (DMD). We illustrate how DMD can estimate a low-rank, finite-dimensional linear operator based on multiple states of the system, effectively approximating potential nonlinear interactions between nodes in the graph. This approach allows us to capture complex dynamics within the graph accurately and efficiently. We theoretically establish a connection between the DMD-estimated operator and the original dynamic operator between system states. Building upon this foundation, we introduce a family of DMD-GNN models that effectively leverage the low-rank eigenfunctions provided by the DMD algorithm. We further discuss the potential of enhancing our approach by incorporating domain-specific constraints such as symmetry into the DMD computation, allowing the corresponding GNN models to respect known physical properties of the underlying system. Our work paves the path for applying advanced dynamical system analysis tools via GNNs. We validate our approach through extensive experiments on various learning tasks, including directed graphs, large-scale graphs, long-range interactions, and spatial-temporal graphs. We also empirically verify that our proposed models can serve as powerful encoders for link prediction tasks. The results demonstrate that our DMD-enhanced GNNs achieve state-of-the-art performance, highlighting the effectiveness of integrating DMD into GNN frameworks.

Signed-Directed graphs, known for the ability to express complex relationships, has been the most valuable among several graph types. However, real-world data often contains severe noise and complexity, making it challenging to analyze. Graph contrastive learning, a powerful technique for learning discriminative representations, has become an essential tool for various graph mining tasks. Therefore, we explore graph contrastive learning with signed-directed graphs. To provide multi-scale discriminative representations, we apply framelet transforms to create more intricate filtering representations and propose a Magnetic Framelet-Based Graph Contrastive Learning for Signed-Directed graphs (Framelet-Gcl). This approach offers a robust model for complex graph data analysis by employing structure and Laplacian perturbations to construct node-level contrastive loss. The overall architecture performs framelet-based convolution in both real and complex domains for each view, enhancing the basis for signal processing. Experimental results demonstrate that the proposed approach outperforms existing state-of-the-art methods across various evaluation metrics on four real-world datasets.

Graph Neural Networks (GNNs) are effective and popular techniques for representation learning of graph data, significantly relying on message passing mechanism. Most GNNs utilize graph convolution with low-pass filtering to update the representation in a coupled way, ignoring the potential interference between attribute and topology. To solve this challenging issue, this study proposes a Flexible Graph Representation Learning (FGRL) method, which adheres to a decoupling and then fusion framework. Specifically, the FGRL utilizes a two-way representation learning scheme to improve the flexibility of message passing by disentangling information in the attribute space and topology space. Beyond low-pass filtering, high-pass filtering information is crucial for node-specific characteristic preservation and also extracted simultaneously. The FGRL mitigates interference between attribute and topological representations by enhancing complementarity with cross-frequency diversity exploration. A more comprehensive and flexible graph embedding representation could be obtained by adaptively fusing attribute, low-pass, and high-pass information. Experimental results demonstrate that the proposed FGRL achieves superior performance in node classification tasks, verifying its discriminative ability in graph representation learning.

Traffic forecasting, a crucial application of spatio-temporal graph (STG) learning, has traditionally relied on deterministic models for accurate point estimations. Yet, these models fall short of quantifying future uncertainties. Recently, many probabilistic methods, especially variants of diffusion models, have been proposed to fill this gap. However, existing diffusion methods typically deal with individual sensors separately when generating future time series, resulting in limited usage of spatial information in the probabilistic learning process. In this work, we propose SpecSTG, a novel spectral diffusion framework, to better leverage spatial dependencies and systematic patterns inherent in traffic data. More specifically, our method generates the Fourier representation of future time series, transforming the learning process into the spectral domain enriched with spatial information. Additionally, our approach incorporates a fast spectral graph convolution designed for Fourier input, alleviating the computational burden associated with existing models. Compared with state-of-the-arts, SpecSTG achieves up to 8% improvements on point estimations and up to 0.78% improvements on quantifying future uncertainties. Furthermore, SpecSTG’s training and validation speed is 3.33× of the most efficient existing diffusion method for STG forecasting. The source code for SpecSTG is available at https://anonymous.4open.science/r/SpecSTG.

A significant challenge for Graph Neural Networks (GNNs) lies in accurately performing node classification tasks on heterophilous graphs. Traditional GNN models, which largely rely on the assumption of homophily, often struggle under heterophilous conditions due to the inherent low-pass filtering nature during message-passing among neighboring nodes. Motivated by these limitations, our research explores the dendritic structures found in real neurons, which dynamically amplify and inhibit signals. We demonstrate how the biophysical properties of dendrites enable GNNs to adaptively modulate the Dirichlet energy of graph representations based on the specific context, providing a theoretical justification for integrating neural dendrites into GNNs to improve learning on heterophilous graphs. In this work, we introduce the GDendrite framework, which dynamically adjusts GNN's output response to the various contexts. Our empirical studies, conducted across 13 homophilous and heterophilous benchmark datasets, show that our framework significantly enhances the performance of traditional GNNs in heterophilous settings and outperforms state-of-the-art models in most cases.

Network Intrusion Detection Systems face increasing challenges from sophisticated evasion techniques that manipulate traffic timing patterns. This paper presents a Temporal Evasion Generation Algorithm (TEGA) for creating adversarial examples by exploiting temporal vulnerabilities, and an Adaptive Temporal Defense System (ATDS) to counter these attacks. We formalize temporal evasion mathematically and evaluate both systems using the CIC-IDS2018 dataset. TEGA achieves evasion success rates of 72.4%, significantly outperforming conventional techniques such as standard delay injection (45.6%) and burst pattern manipulation (53.2%). Conversely, ATDS demonstrates robust defense capabilities, with detection accuracy reaching 95% after adaptation and false positive rates reduced to 1.5%. Our comparative analysis reveals that sequence-based feature extraction combined with SVM classification provides optimal resilience against temporal evasion. The adaptive framework rapidly responds to new attack patterns, typically requiring only 2-3 update cycles to achieve over 90% detection accuracy. This research contributes to network security by addressing an emerging attack vector while offering promising directions for developing next-generation intrusion detection systems.

This paper proposes a novel approach to adversarial attacks against machine learning-based network intrusion detection systems (NIDS). Unlike conventional methods that apply feature-space perturbations, we implement realistic problem-space attacks by manipulating network traffic through advanced packet modification techniques using the Scapy framework. Our methodology targets multiple attack vectors, including recon-naissance, data exfiltration, and command and control communications. Experiments evaluate four popular machine learning models (GBDT, DNN, Random Forest, and SVM) against both legitimate and adversarially manipulated traffic. Results demonstrate significant detection performance degradation across all models, with evasion rates reaching 72% for reconnaissance traffic. We observe strong transferability of adversarial examples between different model architectures and analyze the underlying mechanisms through feature space visualization. Our findings highlight fundamental vulnerabilities in ML-based intrusion detection and emphasize the need for defensive techniques specifically addressing realistic traffic manipulations rather than abstract feature perturbations.

Multimodal Large Language Models (MLLMs) possess extensive knowledge and strong reasoning capabilities, achieving remarkable performance in knowledge-based visual question answering, significantly surpassing traditional small-scale Vision-Language Models (VLMs). However, the distinct training paradigms of MLLMs and small-scale VLMs result in misaligned feature representation spaces and divergent answer prediction distributions. To bridge this gap, we propose a novel end-to-end large-small model synergy framework, where small VLMs and MLLMs collaborate via synergistic optimization of shared objectives while maintaining their co-evolving complementary specializations. Specifically, multimodal fine-grained heuristics are extracted from well-tuned small VLMs and subsequently projected into the textual space of MLLMs through dedicated visual and textual collaboration modules. This enables cross-modal guidance for both visual and textual inputs. Finally, a dual-objective synergy loss promotes alignment toward shared goals, while a visual discrepancy loss preserves specialization diversity. Extensive experiments demonstrate that our framework achieves state-of-the-art performance on both the OK-VQA and A-OKVQA benchmarks.

Supervised learning relies on high-quality labeled data, but obtaining such data through human annotation is both expensive and time-consuming. Recent work explores using large language models (LLMs) for annotation, but LLM-generated labels still fall short of human-level quality. To address this problem, we propose the Annotation with Critical Thinking (ACT) data pipeline, where LLMs serve not only as annotators but also as judges to critically identify potential errors. Human effort is then directed towards reviewing only the most "suspicious" cases, significantly improving the human annotation efficiency. Our major contributions are as follows: (1) ACT is applicable to a wide range of domains, including natural language processing (NLP), computer vision (CV), and multimodal understanding, by leveraging multimodal-LLMs (MLLMs). (2) Through empirical studies, we derive 7 insights on how to enhance annotation quality while efficiently reducing the human cost, and then translate these findings into user-friendly guidelines. (3) We theoretically analyze how to modify the loss function so that models trained on ACT data achieve similar performance to those trained on fully human-annotated data. Our experiments show that the performance gap can be reduced to less than 2% on most benchmark datasets while saving up to 90% of human costs.

Due to its powerful capability of self-supervised representation learning and clustering, contrastive attributed graph clustering (CAGC) has achieved great success, which mainly depends on effective data augmentation and contrastive objective setting. However, most CAGC methods utilize edges as auxiliary information to obtain node-level embedding representation and only focus on node-level embedding augmentation. This approach overlooks edge-level embedding augmentation and the interactions between node-level and edge-level embedding augmentations across various granularity. Moreover, they often treat all contrastive sample pairs equally, neglecting the significant differences between hard and easy positive-negative sample pairs, which ultimately limits their discriminative capability. To tackle these issues, a novel robust attributed graph clustering (RAGC), incorporating hybrid-collaborative augmentation (HCA) and contrastive sample adaptive-differential awareness (CSADA), is proposed. First, node-level and edge-level embedding representations and augmentations are simultaneously executed to establish a more comprehensive similarity measurement criterion for subsequent contrastive learning. In turn, the discriminative similarity further consciously guides edge augmentation. Second, by leveraging pseudo-label information with high confidence, a CSADA strategy is elaborately designed, which adaptively identifies all contrastive sample pairs and differentially treats them by an innovative weight modulation function. The HCA and CSADA modules mutually reinforce each other in a beneficent cycle, thereby enhancing discriminability in representation learning. Comprehensive graph clustering evaluations over six benchmark datasets demonstrate the effectiveness of the proposed RAGC against several state-of-the-art CAGC methods. The code of RAGC could be available at https://github.com/TianxiangZhao0474/RAGC.git.

This research paper presents an attack framework between NIDS and CAN-based intrusion detection that secures vehicle-to-everything (V2X) environments. The connection of vehicles to external networks exposes critical security concerns because public network attacks can penetrate into vehicle systems. Our methodology performs three steps to enable crossdomain attacks through network security domain and automotive security domain integration: (1) ensemble feature selection discovering common vulnerabilities, (2) V2X-Aware calibrated gradient descent that follows protocol constraints, and (3) CAN-Deep PackGen produces valid network-based CAN frames. Our attacks achieve evasion rates of up to 87.9% during experiments on KDDCup99 and UNSW-NB15 for NIDS and CICIoV2024 for CAN IDS. The transfer success rate for timing-related features surpasses 85% due to their vulnerability. The attack success rate receives 24.7% improvement from ensemble feature selection which exhibits strong correlation with feature importance in predicting attack transferability. The diagnostic CAN messages demonstrate the maximum vulnerability to attacks by achieving a$\text{9 2. 3 \%}$evasion rate. These findings highlight critical security implications for connected vehicles and demonstrate the need for cross-domain defensive strategies in automotive cybersecurity, providing a foundation for understanding and mitigating adversarial threats in interconnected transportation.

Abstract Entity alignment (EA) aims at identifying equivalent entity pairs across different knowledge graphs (KGs) that refer to the same real-world identity. It has been a compelling but challenging task that requires the integration of heterogeneous information from different KGs to expand the knowledge coverage and enhance inference abilities. To circumvent the shortage of prior seed alignments provided for training, recent EA models utilize pseudo-labeling strategies to iteratively add unaligned entity pairs predicted with high confidence to the seed alignments for model training. However, the adverse impact of confirmation bias during pseudo-labeling has been largely overlooked, thus hindering entity alignment performance. To systematically combat confirmation bias, we propose a new Unified Pseudo-Labeling framework for Entity Alignment (UPL-EA) that explicitly alleviates pseudo-labeling errors to boost the performance of entity alignment. UPL-EA achieves this goal through two key innovations: (1) Optimal Transport (OT)-based pseudo-labeling uses discrete OT modeling as an effective means to determine entity correspondences and reduce erroneous matches across two KGs. An effective criterion is derived to infer pseudo-labeled alignments that satisfy one-to-one correspondences; (2) Parallel pseudo-label ensembling refines pseudo-labeled alignments by combining predictions over multiple models independently trained in parallel. The ensembled pseudo-labeled alignments are thereafter used to augment seed alignments to reinforce subsequent model training for alignment inference. The effectiveness of UPL-EA in eliminating pseudo-labeling errors is both theoretically supported and experimentally validated. Our extensive results and in-depth analyses demonstrate the superiority of UPL-EA over 15 competitive baselines and its utility as a general pseudo-labeling framework for entity alignment.

Abstract The core of multi‐view clustering is how to exploit the shared and specific information of multi‐view data properly. The data missing and incompleteness bring great challenges to multi‐view clustering. In this paper, we propose an innovative multi‐view subspace clustering method with incomplete graph information, so‐called incomplete multiple graphs clustering. Specifically, we creatively separate one shared and multiple specific graphs from multiple raw graph data, and exploit the mask fusion strategy and block diagonal regulariser to obtain the inherent category information. To handle the incomplete multiple graph data, we utilise multiple indicator matrices to mark the missing elements existed in each raw graph. In addition, the weight of each raw graph is adaptively learnt according to the graph importance. The alternative direction optimization algorithm is employed to solve our proposed methods. Finally, we also analyse the algorithm convergence and the computation complexity in detail. The clustering results on six real‐world datasets show that our method obviously outperforms a serious of classic incomplete multi‐view clustering methods.

Graph Neural Networks (GNNs) have become a cornerstone in multivariate time series forecasting by addressing the challenge of modeling inter-series dependencies often overlooked by traditional temporal approaches. However, real-world temporal dependencies (inter- and intra-dependencies) are inherently intertwined, making it difficult to treat them as separate processes. Recent pure graph paradigms attempt to capture these dependencies holistically by transforming time series into fully connected graphs. While effective, these methods suffer from prohibitive computational complexity O (( NT ) 2 ), limiting their scalability for large-scale data and long-term forecasting. To address these challenges, we propose UFGTime, a novel framework that leverages spectral signals to construct a "spectral-variate graph," embedding multivariate temporal dependencies in a compact spectral representation and modeling inter- and intra-signal connections through frequency similarities. Empowered by our proposed graph framelet message-passing function, UFGTime efficiently aggregates global information, avoids over-smoothing, and achieves near-linear complexity O ( kNT ). Extensive experiments on diverse datasets demonstrate that UFG-Time consistently outperforms state-of-the-art baselines, offering a scalable, accurate, and resource-efficient pure graph solution for multivariate time series forecasting.

Accurately locating the question-related regions in one given image is crucial for visual question answering (VQA). The current approaches suffer two limitations: (1) Dividing one image into multiple regions may lose parts of semantic information and original relationships between regions; (2) Choosing only one or all image regions to predict the answer may correspondingly result in the insufficiency or redundancy of information. Therefore, how to effectively mine the relationship between image regions and choose the relevant image regions are vital. In this paper, we propose a novelMulti-granularity feature interaction andMulti-region selection-based triplet VQA model (M2TVQA). To tackle the first limitation, we propose the multi-granularity feature interaction strategy that adaptively supplements the global coarse-granularity features with the regional fine-granularity features. To overcome the second limitation, we design the Top-$K$learning strategy to adaptively select$K$most relevant image regions to the question, even if the selected regions are far away in space. Such a strategy can select as many relevant image regions as possible and reduce introducing noise. Finally, we construct the multi-modality triplet to predict the answer of VQA. Extended experiments on two public outside knowledge datasets OK-VQA and KRVQA verify the effectiveness of the proposed model.

Graph neural networks (GNNs) exhibit a robust capability for representation learning on graphs with complex structures, demonstrating superior performance across various applications. Most existing GNNs utilize graph convolution operations that integrate both attribute and structural information through coupled way. And these GNNs, from an optimization perspective, seek to learn a consensus and compromised embedding representation that balances attribute and graph information, selectively exploring and retaining valid information in essence. To obtain a more comprehensive embedding representation, a novel GNN framework, dubbed Decoupled Graph Neural Networks (DGNN), is introduced. DGNN separately explores distinctive embedding representations from the attribute and graph spaces by decoupled terms. Considering that the semantic graph, derived from attribute feature space, contains different node connection information and provides enhancement for the topological graph, both topological and semantic graphs are integrated by DGNN for powerful embedding representation learning. Further, structural consistency between the attribute embedding and the graph embedding is promoted to effectively eliminate redundant information and establish soft connection. This process involves facilitating factor sharing for adjacency matrices reconstruction, which aims at exploring consensus and high-level correlations. Finally, a more powerful and comprehensive representation is achieved through the concatenation of these embeddings. Experimental results conducted on several graph benchmark datasets demonstrate its superiority in node classification tasks.

Graph Neural Networks (GNNs) are powerful tools for graph representation learning, but they face challenges when applied to large-scale graphs due to substantial computational costs and memory requirements. To address scalability limitations, various methods have been proposed, including samplingbased and decoupling-based methods. However, these methods have their limitations: sampling-based methods inevitably discard some link information during the sampling process, while decoupling-based methods require alterations to the model's structure, reducing their adaptability to various GNNs. This paper proposes a novel graph pooling method, Graph Partial Pooling (GPPool), for scaling GNNs to large-scale graphs. GPPool is a versatile and straightforward technique that enhances training efficiency while simultaneously reducing memory requirements. GPPool constructs small-scale pooled graphs by pooling partial nodes into supernodes. Each pooled graph consists of supernodes and unpooled nodes, preserving valuable local and global information. Training GNNs on these graphs reduces memory demands and enhances their performance. Additionally, this paper provides a theoretical analysis of training GNNs using GPPool-constructed graphs from a graph diffusion perspective. It shows that a GNN can be transformed from a large-scale graph into pooled graphs with minimal approximation error. A series of experiments on datasets of varying scales demonstrates the effectiveness of GPPool.

Current multi-modal knowledge graph completion often incorporates simple fusion neural networks to achieve multi-modal alignment and knowledge completion tasks, which face three major challenges: 1) Inconsistent semantics between images and texts corresponding to the same entity; 2) Discrepancies in semantic spaces resulting from the use of diverse uni-modal feature extractors;3) Inadequate evaluation of semantic alignment using only energy functions or basic contrastive learning losses. To address these challenges, we propose the Multi-modal Entity in One Word (MEOW) model. This model ensures alignment at various levels, including text-image match alignment, feature alignment and distribution alignment. Specificially, the entity image filtering module utilizes a visual-language model to exclude unrelated images by aligning their captions with corresponding text descriptions. A pre-trained CLIP-based encoder is utilized for encoding dense semantic relationships, while a graph attention network based structure encoder handles sparse semantic relationships, yielding a comprehensive semantic representation and enhancing convergence speed. Additionally, a diffusion model is integrated to enhance denoising capabilities. The proposed MEOW further includes a distribution alignment module equipped with dense alignment constraint, integrity alignment constraint, and fusion fidelity constraint to effectively align multi-modal representations. Experiments on two public multi-modal knowledge graph datasets show that MEOW significantly improves link prediction performance. The code of the proposed model is available athttps://github.com/yuyuyuger/MEOW.

With the rapid growth of internet content, multimodal long document data has become increasingly prominent, drawing significant attention from researchers. However, most existing methods primarily focus on scenarios where all modalities are present, often overlooking more challenging and realistic cases involving missing image modality. To address this limitation, we propose a robust multimodal long document classification (MLDC) framework that integrates hierarchical modeling and dynamic prompting to handle complex multimodal long document data. Our approach begins by leveraging hierarchical modeling combined with an Adaptive Correlation Multimodal Transformer (ACMT) to effectively capture relationships between text and images at both section and sentence levels. We also introduce a Dynamic Prompt Generation (DPG) module at both levels to enhance the model’s robustness in handling missing image data. By evaluating sample uncertainty, the DPG module dynamically adjusts both the number of prompts and the prompts themselves, allowing the model to better adapt to the varying needs of different samples. Finally, a Hierarchical Heterogeneous Graph (HHG) is introduced to enhance feature interactions across levels, further improving the coherence and accuracy of the model. Extensive experiments on four multi-modal long document datasets demonstrate that our model shows superior performance compared to existing state-of-the-art MLDC classification methods in various conditions.

Graph Transformers have demonstrated outstanding performance across various graph-based applications. Despite their success, applying them to large-scale graphs presents significant scalability challenges, limiting their practical use in industrial environments. Recent studies have attempted to overcome this challenge by focusing on the spatial domain of graphs, leading to the development of various scalable models. However, these approaches neglect the spectral characteristics of graphs, which are crucial for adaptively extracting information from full-frequency bands based on the graph’s inherent properties. As a result, existing scalable Graph Transformers tend to rely heavily on low-frequency features, overlooking valuable mid- and high-frequency information. This article proposes the Graph Filter Transformer (GFformer), a framework designed to effectively extract full-frequency information from large-scale graphs. Unlike existing Graph Transformers, GFformer integrates graph filters into the Transformer architecture, thereby enhancing its ability to model both structural and frequency-related properties. Utilizing the proposed Spectral Token Converter (ST-converter), GFformer generates a unique spectral token sequence for each node by incorporating features from diverse frequencies that act as tokens. This design enables the independent learning of node representations in parallel and supports mini-batch training with flexible batch sizes, making GFformer highly scalable. ST-converter employs spectral graph filters, including low-, mid-, and high-pass filters, to extract features serving as tokens. Consequently, each sequence encompasses features from various frequencies, enabling GFformer to capture comprehensive frequency information effectively. Extensive experiments on datasets of varying scales, including both homophilic and heterophilic graphs, consistently demonstrate that GFformer outperforms existing representative methods.

Long Document Classification (LDC) has gained significant attention recently. However, multi-modal data in long documents such as texts and images are not being effectively utilized. Prior studies in this area have attempted to integrate texts and images in document-related tasks, but they have only focused on short text sequences and images of pages. How to classify long documents with hierarchical structure texts and embedding images is a new problem and faces multi-modal representation difficulties. In this paper, we propose a novel approach called Hierarchical Multi-modal Transformer (HMT) for cross-modal long document classification. The HMT conducts multi-modal feature interaction and fusion between images and texts in a hierarchical manner. Our approach uses a multi-modal transformer and a dynamic multi-scale multi-modal transformer to model the complex relationships between image features, and the section and sentence features. Furthermore, we introduce a new interaction strategy called the dynamic mask transfer module to integrate these two transformers by propagating features between them. To validate our approach, we conduct cross-modal LDC experiments on two newly created and two publicly available multi-modal long document datasets, and the results show that the proposed HMT outperforms state-of-the-art single-modality and multi-modality methods.

Multiview subspace clustering (MSC) maximizes the utilization of complementary description information provided by multiview data and achieves impressive clustering performance. However, most of them are inefficient or even invalid among large-scale scenarios due to expensive computational complexity. Recently, anchor strategy has been developed to address this, which selects a few representative samples as anchor points for representation learning and anchor graph construction. However, most of them only explore single cross-view correlation, i.e., cross-view consistency from the global aspect or cross-view complementarity from the local aspect, which provides insufficient semantic correlation understanding and exploration for complex multiview data. To effectively address this issue, this study proposes a fast multiview subspace clustering (FMSC) with local-global anchor representation collaborative learning. FMSC integrates the discriminative anchor points learning and anchor graph construction with optimal structure into a joint framework. Furthermore, local (view-specific) and global (view-shared) anchor representations are learned collaboratively under two interaction strategies at different levels, providing beneficial guidance from global learning to local learning. Thus, the proposed FMSC can maximize the exploration of the complementarity-consistency among multiview data and capture a more comprehensive semantic correlation. More importantly, an effective algorithm with linear complexity is designed to solve the corresponding optimization problem of FMSC, making it more practical in large-scale clustering tasks. Extensive experimental results confirm the superiority of the proposed FMSC in both clustering performance and computational efficiency.

Knowledge distillation (KD) has shown great potential for transferring knowledge from a complex teacher model to a simple student model in which the heavy learning task can be accomplished efficiently and without losing too much prediction accuracy. Recently, many attempts have been made by applying the KD mechanism to graph representation learning models such as graph neural networks (GNNs) to accelerate the model's inference speed via student models. However, many existing KD-based GNNs utilize multilayer perceptron (MLP) as a universal approximator in the student model to imitate the teacher model's process without considering the graph knowledge from the teacher model. In this work, we provide a KD-based framework on multiscaled GNNs, known as graph framelet, and prove that by adequately utilizing the graph knowledge in a multiscaled manner provided by graph framelet decomposition, the student model is capable of adapting both homophilic and heterophilic graphs and has the potential of alleviating the oversquashing issue with a simple yet effective graph surgery. Furthermore, we show how the graph knowledge supplied by the teacher is learned and digested by the student model via both algebra and geometry. Comprehensive experiments show that our proposed model can generate learning accuracy identical to or even surpass the teacher model while maintaining the high speed of inference.

Visual question answering (VQA) models often face two language bias challenges. First, they tend to rely solely on the question to predict the answer, often overlooking relevant information in the accompanying images. Second, even when considering the question, they may focus only on the wh-words, neglecting other crucial keywords that could enhance interpretability and the question sensitivity. Existing debiasing methods attempt to address this by training a bias model using question-only inputs to enhance the robustness of the target VQA model. However, this approach may not fully capture the language bias present. In this article, we propose a multimodality counterfactual dual-bias model to mitigate the linguistic bias issue in target VQA models. Our approach involves designing a shared-parameterized dual-bias model that incorporates both visual and question counterfactual samples as inputs. By doing so, we aim to fully model language biases, with visual and question counterfactual samples, respectively, emphasizing important objects and keywords to relevant the answers. To ensure that our dual-bias model behaves similarly to an ordinary model, we freeze the parameters of the target VQA model, meanwhile using the cross-entropy and Kullback-Leibler (KL) divergence as the loss function to train the dual-bias model. Subsequently, to mitigate language bias in the target VQA model, we freeze the parameters of the dual-bias model to generate pseudo-labels and then incorporate a margin loss to re-train the target VQA model. Experimental results on the VQA-CP datasets demonstrate the superior effectiveness of our proposed counterfactual dual-bias model. Additionally, we conduct an analysis of the unsatisfactory performance on the VQA v2 dataset. The origin code of the proposed model is available at https://github.com/Arrow2022jv/MCD.

The objective of visual question answering (VQA) is to adequately comprehend a question and identify relevant contents in an image that can provide an answer. Existing approaches in VQA often combine visual and question features directly to create a unified cross-modality representation for answer inference. However, this kind of approach fails to bridge the semantic gap between visual and text modalities, resulting in a lack of alignment in cross-modality semantics and the inability to match key visual content accurately. In this article, we propose a model called the caption bridge-based cross-modality alignment and contrastive learning model (CBAC) to address the issue. The CBAC model aims to reduce the semantic gap between different modalities. It consists of a caption-based cross-modality alignment module and a visual-caption (V-C) contrastive learning module. By utilizing an auxiliary caption that shares the same modality as the question and has closer semantic associations with the visual, we are able to effectively reduce the semantic gap by separately matching the caption with both the question and the visual to generate pre-alignment features for each, which are then used in the subsequent fusion process. We also leverage the fact that V-C pairs exhibit stronger semantic connections compared to question-visual (Q-V) pairs to employ a contrastive learning mechanism on visual and caption pairs to further enhance the semantic alignment capabilities of single-modality encoders. Extensive experiments conducted on three benchmark datasets demonstrate that the proposed model outperforms previous state-of-the-art VQA models. Additionally, ablation experiments confirm the effectiveness of each module in our model. Furthermore, we conduct a qualitative analysis by visualizing the attention matrices to assess the reasoning reliability of the proposed model.

The dissemination of fake news, often fueled by exaggeration, distortion, or misleading statements, significantly jeopardizes public safety and shapes social opinion. Although existing multimodal fake news detection methods focus on multimodal consistency, they occasionally neglect modal heterogeneity, missing the opportunity to unearth the most related determinative information concealed within fake news articles. To address this limitation and extract more decisive information, this article proposes the modality perception learning-based determinative factor discovery (MoPeD) model. MoPeD optimizes the steps of feature extraction, fusion, and aggregation to adaptively discover determinants within both unimodality features and multimodality fusion features for the task of fake news detection. Specifically, to capture comprehensive information, the dual encoding module integrates a modal-consistent contrastive language-image pre-training (CLIP) pretrained encoder with a modal-specific encoder, catering to both explicit and implicit information. Motivated by the prompt strategy, the output features of the dual encoding module are complemented by learnable memory information. To handle modality heterogeneity during fusion, the multilevel cross-modality fusion module is introduced to deeply comprehend the complex implicit meaning within text and image. Finally, for aggregating unimodal and multimodal features, the modality perception learning module gauges the similarity between modalities to dynamically emphasize decisive modality features based on the cross-modal content heterogeneity scores. The experimental evaluations conducted on three public fake news datasets show that the proposed model is superior to other state-of-the-art fake news detection methods.

Graph Neural Networks (GNNs) have shown great performance in learning representations for graph-structured data. However, recent studies have found that the interference between topology and attribute can lead to distorted node representations. Most GNNs are designed based on homophily assumptions, thus they cannot be applied to graphs with heterophily. This research critically analyzes the propagation principles of various GNNs and the corresponding challenges from an optimization perspective. A novel GNN called Graph Neural Networks with Soft Association between Topology and Attribute (GNN-SATA) is proposed. Different embeddings are utilized to gain insights into attributes and structures while establishing their interconnections through soft association. Further as integral components of the soft association, a Graph Pruning Module (GPM) and Graph Augmentation Module (GAM) are developed. These modules dynamically remove or add edges to the adjacency relationships to make the model better fit with graphs with homophily or heterophily. Experimental results on homophilic and heterophilic graph datasets convincingly demonstrate that the proposed GNN-SATA effectively captures more accurate adjacency relationships and outperforms state-of-the-art approaches. Especially on the heterophilic graph dataset Squirrel, GNN-SATA achieves a 2.81% improvement in accuracy, utilizing merely 27.19% of the original number of adjacency relationships. Our code is released at https://github.com/wwwfadecom/GNN-SATA.

Maximal Coding Rate Reduction (MCRR or MCR2) has shown its power in image learning. However, few efforts have been made to bring this novel principle to graph learning. In this paper, we investigate the application of the principle of MCR2on graph learning. A Graph Neural Network (GNN) model underpinned by the principle of MCR2is proposed to learn the inherent structures of graphs and produce the graph representations or graph embeddings. The proposed model is applied to the TUDataset, a publicly available dataset encompassing many benchmark datasets for graph classification tasks. The discriminative power of the proposed model is empirically assessed by the downstream graph classification tasks based on the embeddings learned in an unsupervised setting.

Numerous recent research on graph neural networks (GNNs) has focused on formulating GNN architectures as an optimization problem with the smoothness assumption. However, in node classification tasks, the smoothing effect induced by GNNs tends to assimilate representations and over-homogenize labels of connected nodes, leading to adverse effects such as over-smoothing and misclassification. In this paper, we propose a novel bilevel optimization framework for GNNs inspired by the notion of Bregman distance. We demonstrate that the GNN layer proposed accordingly can effectively mitigate the over-smoothing issue by introducing a mechanism reminiscent of the “skip connection”. We validate our theoretical results through comprehensive empirical studies in which Bregman-enhanced GNNs outperform their original counterparts in both homophilic and heterophilic graphs. Furthermore, our experiments also show that Bregman GNNs can produce more robust learning accuracy even when the number of layers is high, suggesting the effectiveness of the proposed method in alleviating the over-smoothing issue.

As a powerful model for deep learning on graph-structured data, the scalability limitation of Graph Neural Networks (GNNs) are receiving increasing attention. To tackle this limitation, two categories of scalable GNNs have been proposed: sampling-based and model simplification methods. However, sampling-based methods suffer from high communication costs and poor performance due to the sampling process. Conversely, existing model simplification methods only rely on parameter-free feature propagation, disregarding its spectral properties. Consequently, these methods can only capture low-frequency information while disregarding valuable middle- and high-frequency information. This paper proposes Automatic Filtering Graph Neural Networks (AutoFGNN), a framework that can extract all frequency information from large-scale graphs. AutoFGNN employs parameter-free low-, middle-, and high-pass filters, which extract the corresponding information for all nodes without introducing parameters. To merge the extracted features, a trainable transformer-based information fusion module is utilized, enabling AutoFGNN to be trained in a mini-batch manner and ensuring scalability for large-scale graphs. Experimental results show that AutoFGNN outperforms existing methods on various scale graphs.

The task of image-text retrieval has gained significant attention in the realm of multimodal artificial intelligence. Nonetheless, existing works encounter challenges in efficiently utilizing inter-modal information and adequately leveraging crucial intra-modal details. In this paper, we propose a novel common-Memory Bridged cross-modal Adaptive Graph Embedding (MBAGE) network for image-text retrieval. Initially, we represent images and text as graphs, wherein nodes symbolize salient regions and words. Subsequently, we incorporate a common-memory bank as an intermediate bridge, facilitating interactions between nodes in the two graphs and enabling efficient utilization of inter-modal information. Additionally, we propose an adaptive graph convolutional network to implement intra-modal interaction, which can adaptively suppress the learning of unimportant nodes. Finally, adaptive pooling is employed to retain essential information, yielding a superior holistic embedding. Experimental results on the Flickr30K and MS-COCO datasets demonstrate that the MBAGE network not only achieves compelling retrieval precision but also exhibits high retrieval efficiency.

In adversarial defense, adversarial purification can be viewed as a special generation task with the purpose to remove adversarial attacks and diffusion models excel in adversarial purification for their strong generative power. With different predetermined generation requirements, various types of guidance have been proposed, but few of them focuses on adversarial purification. In this work, we propose to guide diffusion models for adversarial purification using contrastive guidance. We theoretically derive the proper noise level added in the forward process diffusion models for adversarial purification from a feature learning perspective. For the reverse process, it is implied that the role of contrastive loss guidance is to facilitate the evolution towards the signal direction. From the theoretical findings and implications, we design the forward process with the proper amount of Gaussian noise added and the reverse process with the gradient of contrastive loss as the guidance of diffusion models for adversarial purification. Empirically, extensive experiments on CIFAR-10, CIFAR-100, the German Traffic Sign Recognition Benchmark and ImageNet datasets with ResNet and WideResNet classifiers show that our method outperforms most of current adversarial training and adversarial purification methods by a large improvement.

Attributed graph clustering with auto-encoder (AE) and graph convolutional network (GCN) has achieved promising performance by fusing node attribute feature and structural graph information. However, there are some limitations: (i) structural information from pre-defined graph is inaccurate and insufficient for graph representation learning; (ii) graph embedding of last layer only contains partial information for clustering which inevitably deteriorates clustering performance. To address these issues, we propose the Improved Attributed Graph Clustering method with Representation and Structure Augmentation (IAGC-RSA). The representation augmentor with multi-scale and multi-source representation attention fusion and structure augmentor with adaptive graph learning are designed for information augmentation from structure level and feature level. Thus, IAGC-RSA could learn a more comprehensive and discriminative graph embedding representation for subsequent clustering task. Experimental results conducted on some benchmark datasets demonstrate the effectiveness of IAGC-RSA for node clustering task.

Certain graph neural networks (GNNs) have the capability to glean insights from distant connections while preventing over-smoothing through the use of an implicit approach. Rather than directly computing node representations, these GNNs identify them as solutions to meaningful optimization problems. However, these GNNs depend on the implicit function theorem to derive solutions from the stationary conditions of optimization problems, necessitating repeated attempts with root-finding algorithms. This renders the training of such GNNs computationally expensive on graphs. This paper aims to develop fast and resilient learning algorithms that can speed up the recent implicit graph neural diffusion model based on Dirichlet energy minimization (DIGNN). Our proposed models, named F-DIGNN and L1-DIGNN respectively, demonstrated substantial improvements in training efficiency and robustness through extensive experiments. They consistently achieved comparable or superior performance in node classification tasks across various scenarios.

Network Intrusion Detection Systems (NIDS) are increasingly vulnerable to sophisticated packet-mutation attacks that evade traditional detection methods. This paper presents a runtime adaptive machine-learning strategy to combat such adversarial attacks. We introduce an Adaptive Layered Mutation Algorithm (ALMA) for generating advanced adversarial examples and a runtime adaptive learning framework for real-time detection and response. Our approach integrates these components to create a robust, self-evolving NIDS. Experiments comparing various feature extractors and machine learning classifiers demonstrate that our adaptive approach achieves up to $\mathbf{9 8 \%}$ detection accuracy, significantly improving the identification of mutated packets over static models. The integrated system rapidly adapts to new attack patterns, achieving over $\mathbf{9 0 \%}$ detection accuracy for novel attacks within 2-3 update cycles. This research contributes to network security by presenting an adaptive, high-performance approach to intrusion detection that effectively addresses challenges posed by evolving packet-mutation attacks, offering promising directions for next-generation NIDS development.

Graph neural networks (GNNs) nowadays play a vital role in many high-stakes applications due to their powerful data representation capabilities. In recent years, the issue of fairness discrimination in GNNs has gained significant attention. Existing methods endow different concepts of fairness in GNNs, mainly focusing on group and individual fairness, and give corresponding solution frameworks. However, these existing methods usually only consider optimizing one kind of fairness and claim that group and individual fairness are conflicting and cannot be handled together. Such an approach has limitations and lacks practicality for real-world application scenarios. In this paper, we propose a GNN-based framework for resolving the conflict between group and individual fairness, namely BIG (Balancing Individual and Group fairness). First, we expound on the view that group and individual fairness do not conflict and define a novel notion named balanced fairness and the corresponding evaluation metric. Second, we propose the framework structure and algorithm principle of BIG. Finally, we evaluate the effectiveness of BIG on three real-world datasets. The experimental results show that BIG balances group fairness and individual fairness in GNNs well while ensuring the accuracy of the node classification tasks.

Temporal Knowledge Graphs (TKGs) incorporate a temporal dimension, allowing for a precise capture of the evolution of knowledge and reflecting the dynamic nature of the real world. Typically, TKGs contain complex geometric structures, with various geometric structures interwoven. However, existing Temporal Knowledge Graph Completion (TKGC) methods either model TKGs in a single space or neglect the heterogeneity of different curvature spaces, thus constraining their capacity to capture these intricate geometric structures. In this paper, we propose a novel Integrating Multi-curvature shared and specific Embedding (IME) model for TKGC tasks. Concretely, IME models TKGs into multi-curvature spaces, including hyperspherical, hyperbolic, and Euclidean spaces. Subsequently, IME incorporates two key properties, namely space-shared property and space-specific property. The space-shared property facilitates the learning of commonalities across different curvature spaces and alleviates the spatial gap caused by the heterogeneous nature of multi-curvature spaces, while the space-specific property captures characteristic features. Meanwhile, IME proposes an Adjustable Multi-curvature Pooling (AMP) approach to effectively retain important information. Furthermore, IME innovatively designs similarity, difference, and structure loss functions to attain the stated objective. Experimental results clearly demonstrate the superior performance of IME over existing state-of-the-art TKGC models.

Diffusion models, a family of generative models based on deep learning, have become increasingly prominent in cutting-edge machine learning research. With distinguished performance in generating samples that resemble the observed data, diffusion models are widely used in image, video, and text synthesis nowadays. In recent years, the concept of diffusion has been extended to time-series applications, and many powerful models have been developed. Considering the deficiency of a methodical summary and discourse on these models, we provide this survey as an elementary resource for new researchers in this area and to provide inspiration to motivate future research. For better understanding, we include an introduction about the basics of diffusion models. Except for this, we primarily focus on diffusion-based methods for time-series forecasting, imputation, and generation, and present them, separately, in three individual sections. We also compare different methods for the same application and highlight their connections if applicable. Finally, we conclude with the common limitation of diffusion-based methods and highlight potential future research directions.

人工智能生成内容(AIGC)是近年来人工智能(AI)领域一个研究热点,它有望取代人类以较低成本高效率执行内容生成工作,如音乐、绘画、多模态内容生成、新闻文章、总结报告、股评摘要,以至元宇宙中的内容生成和数字人。AIGC为未来AI发展和实现提供了一条新的技术路径。 在此背景下,《信息与电子工程前沿(英文)》期刊组织了一期关于AIGC最新进展的特刊。本期特刊关注AIGC理论、算法、应用及相关领域。通过吸引高质量论文,我们希望帮助学术界和工业界研究人员更深入了解AIGC背后的基本理论及其潜在应用,激励更多研究人员加入并推进AIGC领域的研究。因此,我们就以下主题(但不限于)征集论文:(1)AI生成音乐;(2)AI生成绘画;(3)AI对话模型;(4)AI新闻摘要;(5)AI与元宇宙;(6)AI与数字人;(7)AI图像编辑;(8)AI生成短视频;(9)AI生成多媒体内容;(10)ChatGPT相关工作。经严格评审,选出12篇论文,包括1篇评论、1篇观点、3篇综述、6篇研究和1篇通讯。我们将其划分为3个主要部分:ChatGPT、扩散模型、提示学习和多模态。 总体而言,本期特刊涵盖了与AIGC开发和应用相关的广泛研究主题,包括人工智能图像/文本生成、三维内容创建、以用户为中心的图形设计、特定风格的音乐生成,以及与因果表征学习、高阶扩散模型相关的工作。此外,还详细调研了概率扩散模型、提示学习和ChatGPT。 最后,感谢所有作者对本期特刊的支持,特别感谢所有评审人对专刊投稿富有见地的意见和有益建议。

Abstract In this paper, we study the differentially private empirical risk minimization problem where the parameter is constrained to a Riemannian manifold. We introduce a framework for performing differentially private Riemannian optimization by adding noise to the Riemannian gradient on the tangent space. The noise follows a Gaussian distribution intrinsically defined with respect to the Riemannian metric on the tangent space. We adapt the Gaussian mechanism from the Euclidean space to the tangent space compatible to such generalized Gaussian distribution. This approach presents a novel analysis as compared to directly adding noise on the manifold. We further prove privacy guarantees of the proposed differentially private Riemannian (stochastic) gradient descent using an extension of the moments accountant technique. Overall, we provide utility guarantees under geodesic (strongly) convex, general nonconvex objectives as well as under the Riemannian Polyak-Łojasiewicz condition. Empirical results illustrate the versatility and efficacy of the proposed framework in several applications.

Abstract In this work, we study the optimal transport (OT) problem between symmetric positive definite (SPD) matrix-valued measures. We formulate the above as a generalized optimal transport problem where the cost, the marginals, and the coupling are represented as block matrices and each component block is a SPD matrix. The summation of row blocks and column blocks in the coupling matrix are constrained by the given block-SPD marginals. We endow the set of such block-coupling matrices with a novel Riemannian manifold structure. This allows to exploit the versatile Riemannian optimization framework to solve generic SPD matrix-valued OT problems. We illustrate the usefulness of the proposed approach in several applications.

Graph data collected from the real world often contains noise, making it imperative to develop robust representation learning tools for graphs. While existing research has primarily focused on feature smoothing, the robustness of the underlying geometric structure is frequently overlooked. In addition, the prevalent use of the$\mathbb {L}_{2}$-norm for achieving global smoothness in graph neural networks shrinks many local characteristics, limiting their expressivity on a node's neighboring information. This paper introduces novel regularizers designed to address noise in both feature and structural aspects of graph data. We employ the alternating direction method of multipliers (ADMM) to optimize the objective function. Our proposed approach effectively prevents oversmoothing graph signal representations when applying multiple layers and ensures convergence to optimal solutions. Empirical results from our study demonstrate the superior performance of our proposedDoTover popular graph convolutions, especially in scenarios where the graph is heavily contaminated.

With the growing demand for text analytics, long document classification (LDC) has received extensive attention, and great progress has been made. To reveal the complex structure and extract the intrinsic feature, the current approaches focus on modeling a long sequence with sparse attention or representing word-sentence or word-section relations partially. However, the thorough hierarchical structure from words, sentences to sections of long documents remains relatively unexplored. For this purpose, we propose a novel Hierarchical Multi-granularity Interaction Graph Convolutional Network (HMIGCN) for long document classification, in which three different granularity graphs, i.e., section graph, sentence graph and word graph, are constructed hierarchically. The section graph encapsulates the macrostructure of a long document, while the sentence and word graphs delve into the document's microstructure. Notably, within the sentence graph, we introduce a Global-Local Graph Convolutional (GLGC) block to adaptively capture both global and local dependency structures among sentence nodes. Additionally, to integrate the three graph networks as a whole, two well-designed techniques, namely section-guided pooling block and transfer fusion block, are proposed to train the model jointly by promoting each other. Extensive experiments on five long document datasets show that our model outperforms the existing state-of-the-art LDC models.

With the continuous emergence of new knowledge, Knowledge Graph (KG) typically suffers from the incompleteness problem, hindering the performance of downstream applications. Thus, Knowledge Graph Completion (KGC) has attracted considerable attention. However, existing KGC methods usually capture the coarse-grained information by directly interacting with the entity and relation, ignoring the important fine-grained information in them. To capture the fine-grained information, in this paper, we divide each entity/relation into several segments and propose a novelMulti-LevelInteraction (MLI) based KGC method, which simultaneously interacts with the entity and relation at the fine-grained level and the coarse-grained level. The fine-grained interaction module applies the Gate Recurrent Unit (GRU) mechanism to guarantee the sequentiality between segments, which facilitates the fine-grained feature interaction and does not obviously sacrifice the model complexity. Moreover, the coarse-grained interaction module designs aHigh-orderFactorizedBilinear (HFB) operation to facilitate the coarse-grained interaction between the entity and relation by applying the tensor factorization based multi-head mechanism, which still effectively reduces its parameter scale. Experimental results show that the proposed method achieves state-of-the-art performances on the link prediction task over five well-established knowledge graph completion benchmarks.

Traffic video question answering (TrafficVQA) constitutes a specialized VideoQA task designed to enhance the basic comprehension and intricate reasoning capacities of videos, specifically focusing on traffic events. Recent VideoQA models employ pretrained visual and textual encoder models to bridge the feature space gap between visual and textual data. However, in addressing the unique challenges inherent to the TrafficVQA task, three pivotal issues must be addressed: (i) Dimension Gap: Between the pretrained image (appearance feature) and video (motion feature) models, there exists a conspicuous dimension difference in static and dynamic visual data; (ii) Scene Gap: The common real-world datasets and the traffic event datasets differ in visual scene content; (iii) Modality Gap: A pronounced feature distribution discrepancy emerges between traffic video and text data. To alleviate these challenges, we introduce the coarse-fine multimodal contrastive alignment network (CFMMC-Align). This model leverages sequence-level and token-level multimodal features, grounded in an unsupervised visual multimodal contrastive loss to mitigate dimension and scene gaps and a supervised visual-textual contrastive loss to alleviate modality discrepancies. Finally, the model is validated on the challenging public TrafficVQA dataset SUTD-TrafficQA and outperforms the state-of-the-art method by a substantial margin (50.2%compared to46.0%). The code is available at https://github.com/guokan987/CFMMC-Align.

In the context of long document classification (LDC), effectively utilizing multi-modal information encompassing texts and images within these documents has not received adequate attention. This task showcases several notable characteristics. Firstly, the text possesses an implicit or explicit hierarchical structure consisting of sections, sentences, and words. Secondly, the distribution of images is dispersed, encompassing various types such as highly relevant topic images and loosely related reference images. Lastly, intricate and diverse relationships exist between images and text at different levels. To address these challenges, we propose a novel approach called Hierarchical Multi-modal Prompting Transformer (HMPT). Our proposed method constructs the uni-modal and multi-modal transformers at both the section and sentence levels, facilitating effective interaction between features. Notably, we design an adaptive multi-scale multi-modal transformer tailored to capture the multi-granularity correlations between sentences and images. Additionally, we introduce three different types of shared prompts, i.e., shared section, sentence, and image prompts, as bridges connecting the isolated transformers, enabling seamless information interaction across different levels and modalities. To validate the model performance, we conducted experiments on two newly created and two publicly available multi-modal long document datasets. The obtained results show that our method outperforms state-of-the-art single-modality and multi-modality classification methods.

In this work, we focus on studying the differentiable relaxations of several linear regression problems, where the original formulations are usually both nonsmooth with one nonconvex term. Unfortunately, in most cases, the standard alternating direction method of multipliers (ADMM) cannot guarantee global convergence when addressing these kinds of problems. To address this issue, by smoothing the convex term and applying a linearization technique before designing the iteration procedures, we employ nonconvex ADMM to optimize challenging nonconvex-convex composite problems. In our theoretical analysis, we prove the boundedness of the generated variable sequence and then guarantee that it converges to a stationary point. Meanwhile, a potential function is derived from the augmented Lagrange function, and we further verify that the objective function is monotonically nonincreasing. Under the Kurdyka-Łojasiewicz (KŁ) property, the global convergence is analyzed step by step. Finally, experiments on face reconstruction, image classification, and subspace clustering tasks are conducted to show the superiority of our algorithms over several state-of-the-art ones.

Temporal knowledge graphs (TKGs) are receiving increased attention due to their time-dependent properties and the evolving nature of knowledge over time. TKGs typically contain complex geometric structures, such as hierarchical, ring, and chain structures, which can often be mixed together. However, embedding TKGs into Euclidean space, as is typically done with TKG completion (TKGC) models, presents a challenge when dealing with high-dimensional nonlinear data and complex geometric structures. To address this issue, we propose a novel TKGC model called multicurvature adaptive embedding (MADE). MADE models TKGs in multicurvature spaces, including flat Euclidean space (zero curvature), hyperbolic space (negative curvature), and hyperspherical space (positive curvature), to handle multiple geometric structures. We assign different weights to different curvature spaces in a data-driven manner to strengthen the ideal curvature spaces for modeling and weaken the inappropriate ones. Additionally, we introduce the quadruplet distributor (QD) to assist the information interaction in each geometric space. Ultimately, we develop an innovative temporal regularization to enhance the smoothness of timestamp embeddings by strengthening the correlation of neighboring timestamps. Experimental results show that MADE outperforms the existing state-of-the-art TKGC models.

Deep learning-based hyperspectral image (HSI) classification methods have recently shown excellent performance. However, the success of these deep learning methods mainly relies on the deep network architecture with a huge amount of parameters trained by a large number of training samples. In this article, a deep high-order tensor sparse representation (SR) network (DHTSRNet) is proposed, which can obtain better classification results in the case of small training samples. Specifically, we propose a high-order tensor SR (HTSR) model that can handle arbitrary-order tensor-type data, and extend it to a deep HTSR model that can be used to train deep high-order tensor filters and features. Then, a deep feature extraction network (DHTSRNet) based on the deep HTSR model is constructed, which is used for feature extraction of HSI. Finally, an HSI classification method is constructed by combining DHTSRNet and the classifier based on graph-based learning (GSL), which can obtain better classification results in the case of small training samples. Experimental results show that the DHTSRNet can obtain better classification performance compared with other state-of-the-art HSI classification methods.

Long Document Classification (LDC) has attracted great attention in Natural Language Processing and achieved considerable progress owing to the large-scale pre-trained language models. In spite of this, as a different problem from the traditional text classification, LDC is far from being settled. Long documents, such as news and articles, generally have more than thousands of words with complex structures. Moreover, compared with flat text, long documents usually contain multi-modal content of images, which provide rich information but not yet being utilized for classification. In this article, we propose a novel cross-modal method for long document classification, in which multiple granularity feature shifting networks are proposed to integrate the multi-scale text and visual features of long documents adaptively. Additionally, a multi-modal collaborative pooling block is proposed to eliminate redundant fine-grained text features and simultaneously reduce the computational complexity. To verify the effectiveness of the proposed model, we conduct experiments on the Food101 dataset and two constructed multi-modal long document datasets. The experimental results show that the proposed cross-modal method outperforms the single-modal text methods and defeats the state-of-the-art related multi-modal baselines.

The goal of mixed-modality clustering, which differs from typical multi-modality/view clustering, is to divide samples derived from various modalities into several clusters. This task has to solve two critical semantic gap problems: i) how to generate the missing modalities without the pairwise-modality data; and ii) how to align the representations of heterogeneous modalities. To tackle the above problems, this paper proposes a novel mixedmodality clustering model, which integrates the missing-modality generation and the heterogeneous modality alignment into a unified framework. During the missing-modality generation process, a bidirectional mapping is established between different modalities, enabling generation of preliminary representations for the missing-modality using information from another modality. Then the intra-modality bipartite graphs are constructed to help generate better missing-modality representations by weighted aggregating existing intra-modality neighbors. In this way, a pairwise-modality representation for each sample can be obtained. In the process of heterogeneous modality alignment, each modality is modelled as a graph to capture the global structure among intra-modality samples and is aligned against the heterogeneous modality representations through the adaptive heterogeneous graph matching module. Experimental results on three public datasets show the effectiveness of the proposed model compared to multiple state-of-the-art multi-modality/view clustering methods.

To date, various neural methods have been proposed for causal effect estimation based on observational data, where a default assumption is the same distribution and availability of variables at both training and inference (i.e., runtime) stages. However, distribution shift (i.e., domain shift) could happen during runtime, and bigger challenges arise from the impaired accessibility of variables. This is commonly caused by increasing privacy and ethical concerns, which can make arbitrary variables unavailable in the entire runtime data and imputation impractical. We term the co-occurrence of domain shift and inaccessible variablesruntime domain corruption, which seriously impairs the generalizability of a trained counterfactual predictor. To counter runtime domain corruption, we subsume counterfactual prediction under the notion of domain adaptation. Specifically, we upper-bound the error w.r.t. the target domain (i.e., runtime covariates) by the sum of source domain error and inter-domain distribution distance. In addition, we build an adversarially unified variational causal effect model, named VEGAN, with a novel two-stage adversarial domain adaptation scheme to reduce the latent distribution disparity between treated and control groups first, and between training and runtime variables afterwards. We demonstrate that VEGAN outperforms other state-of-the-art baselines on individual-level treatment effect estimation in the presence of runtime domain corruption on benchmark datasets.

Due to the homophily assumption in graph convolution networks (GCNs), a common consensus in the graph node classification task is that graph neural networks (GNNs) perform well on homophilic graphs but may fail on heterophilic graphs with many interclass edges. However, the previous interclass edges' perspective and related homo-ratio metrics cannot well explain the GNNs' performance under some heterophilic datasets, which implies that not all the interclass edges are harmful to GNNs. In this work, we propose a new metric based on the von Neumann entropy to reexamine the heterophily problem of GNNs and investigate the feature aggregation of interclass edges from an entire neighbor identifiable perspective. Moreover, we propose a simple yet effective Conv-Agnostic GNN framework (CAGNNs) to enhance the performance of most GNNs on the heterophily datasets by learning the neighbor effect for each node. Specifically, we first decouple the feature of each node into the discriminative feature for downstream tasks and the aggregation feature for graph convolution (GC). Then, we propose a shared mixer module to adaptively evaluate the neighbor effect of each node to incorporate the neighbor information. The proposed framework can be regarded as a plug-in component and is compatible with most GNNs. The experimental results over nine well-known benchmark datasets indicate that our framework can significantly improve performance, especially for the heterophily graphs. The average performance gain is 9.81%, 25.81%, and 20.61% compared with graph isomorphism network (GIN), graph attention network (GAT), and GCN, respectively. Extensive ablation studies and robustness analysis further verify the effectiveness, robustness, and interpretability of our framework. Code is available at https://github.com/JC-202/CAGNN.

Benefiting from exploiting the data topological structure, graph convolutional network (GCN) has made considerable improvements in processing clustering tasks. The performance of GCN significantly relies on the quality of the pretrained graph, while the graph structures are often corrupted by noise or outliers. To overcome this problem, we replace the pre-trained and fixed graph in GCN by the adaptive graph learned from the data. In this article, we propose a novel end-to-end parallelly adaptive graph convolutional clustering (AGCC) model with two pathway networks. In the first pathway, an adaptive graph convolutional (AGC) module alternatively updates the graph structure and the data representation layer by layer. The updated graph can better reflect the data relationship than the fixed graph. In the second pathway, the auto-encoder (AE) module aims to extract the latent data features. To effectively connect the AGC and AE modules, we creatively propose an attention-mechanism-based fusion (AMF) module to weight and fuse the data representations of the two modules, and transfer them to the AGC module. This simultaneously avoids the over-smoothing problem of GCN. Experimental results on six public datasets show that the effectiveness of the proposed AGCC compared with multiple state-of-the-art deep clustering methods. The code is available at https://github.com/HeXiax/AGCC.

Temporal knowledge graph completion (TKGC) is an extension of the traditional static knowledge graph completion (SKGC) by introducing the timestamp. The existing TKGC methods generally translate the original quadruplet to the form of the triplet by integrating the timestamp into the entity/relation, and then use SKGC methods to infer the missing item. However, such an integrating operation largely limits the expressive ability of temporal information and ignores the semantic loss problem due to the fact that entities, relations, and timestamps are located in different spaces. In this article, we propose a novel TKGC method called the quadruplet distributor network (QDN), which independently models the embeddings of entities, relations, and timestamps in their specific spaces to fully capture the semantics and builds the QD to facilitate the information aggregation and distribution among them. Furthermore, the interaction among entities, relations, and timestamps is integrated using a novel quadruplet-specific decoder, which stretches the third-order tensor to the fourth-order to satisfy the TKGC criterion. Equally important, we design a novel temporal regularization that imposes a smoothness constraint on temporal embeddings. Experimental results show that the proposed method outperforms the existing state-of-the-art TKGC methods. The source codes of this article are available at https://github.com/QDN for Temporal Knowledge Graph Completion.git.

Structured clustering networks, which alleviate the oversmoothing issue by delivering hidden features from autoencoder (AE) to graph convolutional networks (GCNs), involve two shortcomings for the clustering task. For one thing, they used vanilla structure to learn clustering representations without considering feature and structure corruption; for another thing, they exhibit network degradation and vanishing gradient issues after stacking multilayer GCNs. In this article, we propose a clustering method called dual-masked deep structural clustering network (DMDSC) with adaptive bidirectional information delivery (ABID). Specifically, DMDSC enables generative self-supervised learning to mine deeper interstructure and interfeature correlations by simultaneously reconstructing corrupted structures and features. Furthermore, DMDSC develops an ABID module to establish an information transfer channel between each pairwise layer of AE and GCNs to alleviate the oversmoothing and vanishing gradient problems. Numerous experiments on six benchmark datasets have shown that the proposed DMDSC outperforms the most advanced deep clustering algorithms.

In this paper, we propose a convergence acceleration scheme for general Riemannian optimization problems by extrapolating iterates on manifolds. We show that when the iterates are generated from the Riemannian gradient descent method, the scheme achieves the optimal convergence rate asymptotically and is computationally more favorable than the recently proposed Riemannian Nesterov accelerated gradient methods. A salient feature of our analysis is the convergence guarantees with respect to the use of general retraction and vector transport. Empirically, we verify the practical benefits of the proposed acceleration strategy, including robustness to the choice of different averaging schemes on manifolds.

The global economic losses caused by credit card fraud are enormous and continuously increasing. Effective and accurate fraud detection has become a crucial task in recent years. Prior approaches can achieve good detection performance under certain conditions. However, these existing methods lack the robustness and scalability to deal with real-world credit card transactional datasets containing a large number of missing values. In this paper, we propose a Feature Importance-based Weighted Graph Neural Network (FIW-GNN) as an effective, stable, and practical solution for credit card fraud detection. First, we propose a method to construct a heterogeneous graph designed for credit card transactional datasets. Next, based on the architecture of the relational graph convolutional network, a feature importance-based method is employed to assign edge weights. Finally, we evaluate the effectiveness of FIW-GNN on two benchmark datasets. The experimental results demonstrate that FIW-GNN outperforms the state-of-the-art baselines in all selected evaluation metrics.

Spectral Graph Convolutional Networks (spectral GCNNs), a powerful tool for analyzing and processing graph data, typically apply frequency filtering via Fourier transform to obtain representations with selective information. Although research shows that spectral GCNNs can be enhanced by framelet-based filtering, the massive majority of such research only considers undirected graphs. In this paper, we introduce Framelet-MagNet, a magnetic framelet-based spectral GCNN for directed graphs (digraphs). The model applies the framelet transform to digraph signals to form a more sophisticated representation for filtering. Digraph framelets are constructed with the complex-valued magnetic Laplacian, simultaneously leading to signal processing in both real and complex domains. We empirically validate the predictive power of Framelet-MagNet over a range of state-of-the-art models in node classification, link prediction, and denoising.

Data similarity computation is a fundamental research topic which underpins many high-level applications based on similarity measures. However, the exact similarity computation has become daunting in large-scale real-world scenarios. Currently, MinHash is a popular technique for efficiently estimating the Jaccard similarity of binary sets and, furthermore, weighted MinHash is utilized to estimate the generalized Jaccard similarity of weighted sets. This review focuses on categorizing and discussing the existing works of weighted MinHash algorithms. Also, we have developed a Python toolbox for the algorithms, and released it in our github.

Audio event recognition (AER), a widely concerned problem, is the problem of classifying environmental sounds, that is, predicting the scenes where they are recorded. Google released the Audioset dataset for AER research in 2017, which greatly expands the size of the previous AER datasets. The dataset has 2 million sound clips, divided into 527 categories. Audioset is a weakly- and multi-labeled dataset. Previous approaches to AER focus on the attention model to solve the weakly labeled problem. However, these methods do not consider the association between labels. Recently, Graph Convolutional Neural Network (GCN) has shown a significant advantage in leveraging relationships between labels. This paper considers the correlation between labels using the GCN method. We improve two basic attention models with GCN structure using label correlation information. Relevant experiments are conducted on the Audioset dataset. The result demonstrates that the use of label relationships can improve the essential model performance.

Dimensionality reduction (DR) and manifold learning (ManL) have been applied extensively in many machine learning tasks, including computer vision, image analysis and pattern recognition just to name a few. However, the geometrical correctness of DR and ManL models learning results is largely neglected. In this work, we investigate this important aspect of some widely used DR and ManL methods through the lens of the chart map function, which is the essential part of the definition of a manifold. It turns out that the mapping functions induced by these methods do not have the injectivity (i.e. one-to-one mapping) between ambient space and latent space. This poses the distinguishability problem for down-stream tasks. Without injectivity, two distinct points on a manifold may be projected to the same point in the latent space learnt by DR or ManL methods, and hence the later process cannot separate them apart, resulting in inevitable errors. To address this problem, we provide a provably correct algorithm called fixed points Laplacian mapping (FPLM), which has the geometric guarantee to find a representation of a manifold with injectivity by using a simplical complex generated on manifold. We further discuss the property of our proposed method via various aspects, including its link to the popular graph neural networks (GNNs) and deep neural networks (DNNs). Its geometric correctness is demonstrated by extensive experimental results and theoretical proofs. Moreover, experiments also show our method is capable of evaluating the quality of simplex decomposition of the manifold and detecting manifold intrinsic dimensions for real-world datasets.

Recently, many works on Graph Neural Networks (GNNs) have been well developed for graph-level representation learning tasks and continuously improve graph classification accuracy. These works, however, mainly focus on the principle of different blocks of GNNs to strengthen their representation learning capability and pay less attention to the choice of a more appropriate loss function as learning objectives. In this paper, we aim to facilitate the representation learning process of any existing GNN frameworks by incorporating more task-oriented objectives. To this end, we propose approaches based on Laplacian eigenmaps to enhance the common-used cross-entropy loss, called LEELoss. Our study shows, with the property of Laplacian eigenmaps for classification problems, that utilizing the Laplacian eigenmaps as a regularizer in the original loss function can further enhance the performance of the graph-level learning tasks. Finally, via extensive experiments on popular benchmark datasets, we demonstrate that the proposed approach actually facilitates the classification performance on various GNN frameworks.

Considerable performance gains have been achieved for knowledge-based visual question answering due to the visual-language pre-training models with pre-training-then-fine-tuning paradigm. However, because the targets of the pre-training and fine-tuning stages are different, there is an evident barrier that prevents the cross-modal comprehension ability developed in the pre-training stage from fully endowing the fine-tuning task. To break this barrier, in this paper, we propose a novel hybrid prompting model for knowledge-based VQA, which inherits and incorporates the pre-training and fine-tuning tasks with a shared objective. Specifically, based on static declaration prompt, we construct a consistent goal with the fine-tuning via masked language modeling to inherit capabilities of pre-training task, while selecting the top-t relevant knowledge in a dense retrieval manner. Additionally, a dynamic knowledge prompt is learned from retrieved knowledge, which not only alleviates the length constraint on inputs for visual-language pre-trained models but also assists in providing answer features via fine-tuning. Combining and unifying the aims of the two stages could fully exploit the abilities of pre-training and fine-tuning to predict answer. We evaluate the proposed model on the OKVQA dataset, and the result shows that our model outperforms the state-of-the-art methods based on visual-language pre-training models with a noticeable performance gap and even exceeds the large-scale language model of GPT-3, which proves the benefits of the hybrid prompts and the advantages of unifying pre-training to fine-tuning.

The exponential growth of multimedia data has sparked a surge of interest in multimodal summarisation with multimodal output (MSMO). A relatively unexplored but essential task within this field is extreme multimodal summarisation, a process that involves creating extremely concise multimodal summaries to further address the issue of multimedia information overload. In this study, we propose a novel Unsupervised Topic-guided Co-Attention Transformer (TopicCAT) neural network to produce extreme multimodal summaries for video-document pairs. The approach consists of two learning stages for a comprehensive multimodal understanding, guided by topic-based insights: a unimodal learning stage and a cross-modal learning stage, in which a cross-modal topic model is devised to capture the overarching themes present in both documents and videos. To achieve unsupervised learning, eliminating the need for resource-expensive collection of ground-truth multimodal summaries, we propose an optimal transport-based optimisation scheme to evaluate summary coverage from a semantic distribution perspective at the topic-level. Comprehensive experiments demonstrate the effectiveness of our proposed TopicCAT method on a multimodal news dataset, achieving a BERTScore of 84.46 and an accuracy of 0.60.

The performance of graph representation learning is affected by the quality of graph input. While existing research usually pursues a globally smoothed graph embedding, we believe the rarely observed anomalies are as well harmful to an accurate prediction. This work establishes a graph learning scheme that automatically detects (locally) corrupted feature attributes and recovers robust embedding for prediction tasks. The detection operation leverages a graph autoencoder, which does not make any assumptions about the distribution of the local corruptions. It pinpoints the positions of the anomalous node attributes in an unbiased mask matrix, where robust estimations are recovered with sparsity promoting regularizer. The optimizer approaches a new embedding that is sparse in the framelet domain and conditionally close to input observations. Extensive experiments are provided to validate our proposed model can recover a robust graph representation from black-box poisoning and achieve excellent performance.

The reverse k nearest neighbors (RkNN) query is a prominent yet time-consuming spatial query used in facility siting, influential domain analysis, potential customer analysis, etc. Its aim is to identify all points that consider the query point as one of their k closest points. However, when k is relatively large (e.g. k = 1000), existing RkNN techniques often struggle to provide acceptable response times (within a few seconds). To address this issue, we propose a verification approach called conic section discriminance (CSD). This method serves to determine whether points belong to the RkNN set. With CSD, only a small fraction of candidates require costly k nearest neighbors (kNN) queries for verification, while the rest can be rapidly verified with O(1) complexity. Furthermore, we propose a Voronoi-based candidate generation approach to curtail the candidate set size. By leveraging the VoR-tree structure, we integrate these two approaches to form a novel RkNN algorithm named CSD-RkNN. A comprehensive set of experiments is conducted to compare CSD-RkNN with Slice as the state-of-the-art RkNN algorithm, and VR-RkNN as the original RkNN algorithm on VoR-tree. The results indicate that CSD-RkNN consistently outperforms the other two algorithms, especially when k is relatively large.

The low-rank tensor could characterize inner structure and explore high-order correlation among multi-view representations, which has been widely used in multi-view clustering. Existing approaches adopt the tensor nuclear norm (TNN) as a convex approximation of non-convex tensor rank function. However, TNN treats the different singular values equally and over-penalizes the main rank components, leading to sub-optimal tensor representation. In this paper, we devise a better surrogate of tensor rank, namely the tensor logarithmic Schatten- p norm ([Formula: see text]N), which fully considers the physical difference between singular values by the non-convex and non-linear penalty function. Further, a tensor logarithmic Schatten- p norm minimization ([Formula: see text]NM)-based multi-view subspace clustering ([Formula: see text]NM-MSC) model is proposed. Specially, the proposed [Formula: see text]NM can not only protect the larger singular values encoded with useful structural information, but also remove the smaller ones encoded with redundant information. Thus, the learned tensor representation with compact low-rank structure will well explore the complementary information and accurately characterize the high-order correlation among multi-views. The alternating direction method of multipliers (ADMM) is used to solve the non-convex multi-block [Formula: see text]NM-MSC model where the challenging [Formula: see text]NM problem is carefully handled. Importantly, the algorithm convergence analysis is mathematically established by showing that the sequence generated by the algorithm is of Cauchy and converges to a Karush-Kuhn-Tucker (KKT) point. Experimental results on nine benchmark databases reveal the superiority of the [Formula: see text]NM-MSC model.

Graph convolutional networks (GCN) have been applied in the traffic flow forecasting tasks with the graph capability in describing the irregular topology structures of road networks. However, GCN based traffic flow forecasting methods often fail to simultaneously capture the short-term and long-term temporal relations carried by the traffic flow data, and also suffer the over-smoothing problem. To overcome the problems, we propose a hierarchical traffic flow forecasting network by merging newly designed the long-term temporal Transformer network (LTT) and the spatio-temporal graph convolutional networks (STGC). Specifically, LTT aims to learn the long-term temporal relations among the traffic flow data, while the STGC module aims to capture the short-term temporal relations and spatial relations among the traffic flow data, respectively, via cascading between the one-dimensional convolution and the graph convolution. In addition, an attention fusion mechanism is proposed to combine the long-term with the short-term temporal relations as the input of the graph convolution layer in STGC, in order to mitigate the over-smoothing problem of GCN. Experimental results on three public traffic flow datasets prove the effectiveness and robustness of the proposed method.

Knowledge Graph Completion (KGC) aims at inferring missing entities or relations by embedding them in a low-dimensional space. However, most existing KGC methods generally fail to handle the complex concepts hidden in triplets, so the learned embeddings of entities or relations may deviate from the true situation. In this article, we propose a novel M ulti- c oncept R epresentation L earning (McRL) method for the KGC task, which mainly consists of a multi-concept representation module, a deep residual attention module, and an interaction embedding module. Specifically, instead of the single-feature representation, the multi-concept representation module projects each entity or relation to multiple vectors to capture the complex conceptual information hidden in them. The deep residual attention module simultaneously explores the inter- and intra-connection between entities and relations to enhance the entity and relation embeddings corresponding to the current contextual situation. Moreover, the interaction embedding module further weakens the noise and ambiguity to obtain the optimal and robust embeddings. We conduct the link prediction experiment to evaluate the proposed method on several standard datasets, and experimental results show that the proposed method outperforms existing state-of-the-art KGC methods.

With the powerful learning ability of deep convolutional networks, deep clustering methods can extract the most discriminative information from individual data and produce more satisfactory clustering results. However, existing deep clustering methods usually ignore the relationship between the data. Fortunately, the graph convolutional network can handle such relationships, opening a new research direction for deep clustering. In this paper, we propose a cross-attention based deep clustering framework, named Cross-Attention Fusion based Enhanced Graph Convolutional Network (CaEGCN), which contains four main modules: the cross-attention fusion module which innovatively concatenates the Content Auto-encoder module (CAE) relating to the individual data and Graph Convolutional Auto-encoder module (GAE) relating to the relationship between the data in a layer-by-layer manner, and the self-supervised model that highlights the discriminative information for clustering tasks. While the cross-attention fusion module fuses two kinds of heterogeneous representation, the CAE module supplements the content information for the GAE module, which avoids the over-smoothing problem of GCN. In the GAE module, two novel loss functions are proposed that reconstruct the content and relationship between the data, respectively. Finally, the self-supervised module constrains the distributions of the middle layer representations of CAE and GAE to be consistent. Experimental results on different types of datasets prove the superiority and robustness of the proposed CaEGCN.

Conventional Knowledge Graph Completion (KGC) methods typically map entities and relations to a unified space through the shared mapping matrix, and then interact with entities and relations to infer the missing items in the knowledge graph. Although this shared mapping matrix considers the suitability of all triplets, it neglects the specificity of each triplet. To solve this problem, we dynamically learn one information distributor for each triplet to exchange its specific information. In this paper, we propose a novel Triplet Distributor Network (TDN) for the knowledge graph completion task. Specifically, we adaptively learn one Triplet Distributor (TD) for each triplet to assist the interaction between the entity and relation. Furthermore, on the basis of TD, we creatively design the information exchange layer to dynamically propagate the information of the entity and relation, thus mutually enhancing entity and relation representations. Except for several commonly-used knowledge graph datasets, we still implement the link prediction task on the social-relational and medical datasets to test the proposed method. Experimental results demonstrate that the proposed method performs better than existing state-of-the-art KGC methods. The source codes of this paper are available athttps://github.com/TDNfor Knowledge Graph Completion.git.

Graph neural networks (GNNs) have been ubiquitous in graph node classification tasks. Most GNN methods update the node embedding iteratively by aggregating its neighbors' information. However, they often suffer from negative disturbances, due to edges connecting nodes with different labels. One approach to alleviate this negative disturbance is to use attention to learn the weights of aggregation, but current attention-based GNNs only consider feature similarity and suffer from the lack of supervision. In this article, we consider label dependency of graph nodes and propose a decoupling attention mechanism to learn both hard and soft attention. The hard attention is learned on labels for a refined graph structure with fewer interclass edges so that the aggregation's negative disturbance can be reduced. The soft attention aims to learn the aggregation weights based on features over the refined graph structure to enhance information gains during message passing. Particularly, we formulate our model under the expectation-maximization (EM) framework, and the learned attention is used to guide label propagation in the M-step and feature propagation in the E-step, respectively. Extensive experiments are performed on six well-known benchmark graph datasets to verify the effectiveness of the proposed method.

Long document classification (LDC) has been a focused interest in natural language processing (NLP) recently with the exponential increase of publications. Based on the pretrained language models, many LDC methods have been proposed and achieved considerable progression. However, most of the existing methods model long documents as sequences of text while omitting the document structure, thus limiting the capability of effectively representing long texts carrying structure information. To mitigate such limitation, we propose a novel hierarchical graph convolutional network (HGCN) for structured LDC in this article, in which a section graph network is proposed to model the macrostructure of a document and a word graph network with a decoupled graph convolutional block is designed to extract the fine-grained features of a document. In addition, an interaction strategy is proposed to integrate these two networks as a whole by propagating features between them. To verify the effectiveness of the proposed model, four structured long document datasets are constructed, and the extensive experiments conducted on these datasets and another unstructured dataset show that the proposed method outperforms the state-of-the-art related classification methods.

Domain adaptation in the Euclidean space is a challenging task on which researchers recently have made great progress. However, in practice, there are rich data representations that are not Euclidean. For example, many high-dimensional data in computer vision are in general modeled by a low-dimensional manifold. This prompts the demand of exploring domain adaptation between non-Euclidean manifold spaces. This article is concerned with domain adaption over the classic Grassmann manifolds. An optimal transport-based domain adaptation model on Grassmann manifolds has been proposed. The model implements the adaption between datasets by minimizing the Wasserstein distances between the projected source data and the target data on Grassmann manifolds. Four regularization terms are introduced to keep task-related consistency in the adaptation process. Furthermore, to reduce the computational cost, a simplified model preserving the necessary adaption property and its efficient algorithm is proposed and tested. The experiments on several publicly available datasets prove the proposed model outperforms several relevant baseline domain adaptation methods.

We propose a meta-learning approach with graph neural networks for link prediction: Neural Processes for Graph Neural Networks (NPGNN), which can not only perform both transductive and inductive learning tasks, but also generalize well when only training on a small subgraph. The key idea is to assume the node embeddings follow a Gaussian Process parameterised by graph neural networks and then use a meta-learning framework to pass information from the subgraph to the complete graph. Experiments on real-world citation networks are conducted to validate our model, where the results suggest that the proposed method achieves stronger performance compared to other state-of-the-art models.

In the recent decade, multidimensional or tenso-rial time series have drawn increasing attention for their rich spatial and temporal information. In data collection, missing tensorial time instances can always occur at any random time steps, which cause the time steps to be not equally spaced, i.e., irregular. Furthermore, it is very likely that the tensorial time series are influenced by other tensor factors, yet from external resources. These driving tensor factors are referred to as tensor controls, whereas the influenced tensorial time series are named as tensor responses. Existing methods either cannot predict tensor responses using all available past tensor controls, or are not able to directly model continuous-time tensor-valued processes, incurring the curse-of-dimensionality issue. Therefore, we propose tensor neural controlled differential equations (TENCDEs) which incorporate tensor controls and explicitly define the continuous-time tensor-valued processes for tensor controls and tensor responses aiming at resolving irregular tensorial time series issues. TENCDE also effectively captures both spatial and temporal information, and avoids the curse-of-dimensionality issue. The unique solution and the consistency and convergence of solutions are also guaranteed. The experiments demonstrate the proposed TENCDE outperforms the existing models including the state-of-the-art methods.

Long document classification has aroused tremendous attention in the field of Nature Language Processing, due to the exponential increasing of publications. Although the common text classification methods can be extended for long document classification, they are confined to the length of text and do not have enough expressiveness to model the structure of the long document. To solve these problems, we proposed a hierarchical multiple granularity attention network for long document classification, in which the word and section level features are extracted and fused to represent the complex structure of the long document. Furthermore, a feature-based section pooling module is adopted to eliminate redundant text information and accelerate the computing. A series of experiments are conducted to evaluate the proposed method. The experimental results verify that our method is effective, efficient and competitive compared with the related state-of-the-art methods.

Graph-based clustering learns underlying data representation by employing topological graph structure. Recently, Graph Convolutional Network (GCN)-based clustering methods have accumulated great attentions and achieved great performance. Its performance is seriously determined by the quality of pre-provided graph, which is usually constructed by predefined model (such as k-Nearest-Neighbor). However, the graph may be inaccurate due to the noises and fixed graph limits flexibility of model learning. In this paper, we propose a Robust Graph Convolutional Clustering (RGCC) method, which adaptively learns a clean and accurate graph from original graph. Specifically, adaptive graph with low-rank and sparse structures be learned during the optimization process, which can better encode structural information of data than fixed graph. Then, to explore the local connectivity of data, graph Laplacian constraint is introduced. Thus, optimal graph relationships and discriminative representation of data could be simultaneously learned, which improves the flexibility of the RGCC model. By designing a self-supervised clustering module, it can self-supervise the node representations learning and thus explore the better clustering structure. Experimental results on several benchmark databases reveal the superiority of the proposed RGCC approach.

Collaborative filtering is a central task in a broad range of recommender systems. As traditional methods train latent variables for user/item individuals under a transductive setting, it requires re-training for out-of-sample inferences. Inductive matrix completion (IMC) solves this problem by learning transformation functions upon engineered features, but it sacrifices model expressiveness and highly depends on feature qualities. In this paper, we propose Geometric Inductive Matrix Completion (GIMC) by introducing hyperbolic geometry and a unified message passing scheme into this generic task. The proposed method is the earliest attempt utilizing capacious hyperbolic space to enhance the capacity of IMC. It is the first work defining continuous explicit feedback prediction within non-Euclidean space by introducing hyperbolic regression for vertex interactions. This is also the first to provide comprehensive evidence that edge semantics can significantly improve recommendations, which is ignored by previous works. The proposed method outperforms the state-of-the-art algorithms with less than 1% parameters compared to its transductive counterparts. Extensive analysis and ablation studies are conducted to reveal the design considerations and practicability for a positive impact to the research community.

Variance reduction is popular in accelerating gradient descent and stochastic gradient descent for optimization problems defined on both euclidean space and Riemannian manifold. This paper further improves on existing variance reduction methods for non-convex Riemannian optimization, including R-SVRG and R-SRG/R-SPIDER by providing a unified framework for batch size adaptation. Such framework is more general than the existing works by considering retraction and vector transport and mini-batch stochastic gradients. We show that the adaptive-batch variance reduction methods require lower gradient complexities for both general non-convex and gradient dominated functions, under both finite-sum and online optimization settings. Moreover, under the new framework, we complete the analysis of R-SVRG and R-SRG, which is currently missing in the literature. We prove convergence of R-SVRG with much simpler analysis, which leads to curvature-free complexity bounds. We also show improved results for R-SRG under double-loop convergence, which match the optimal complexities as the R-SPIDER. In addition, we prove the first online complexity results for R-SVRG and R-SRG. Lastly, we discuss the potential of adapting batch size for non-smooth, constrained and second-order Riemannian optimizers. Extensive experiments on a variety of applications support the analysis and claims in the paper.

Public speaking is a critical skill in daily communication. While more practicing such as rehearsal is helpful to improve such a skill, lack of personalized feedback limits the effectiveness of practicing. Therefore, we formulate the task of personalized feedback as an affective audio annotation problem by learning knowledge from online public speech videos. Considering the great success of deep learning techniques such as convolutional neural networks in a wide range of applications including speech recognition and object recognition, we propose a novel convolutional clustering neural network (CCNN) to solve this multi-label classification problem. Instead of aggregating the features of different channels through pooling, we introduce a novel clustering layer to derive intermediate representation for improved annotation performance. In order to evaluate the performance of our proposed method, we purposely built an affective audio annotation dataset by collecting more than 2,000 video clips from the TED website. Experimental results on this dataset demonstrate that our proposed method outperforms traditional CNN-based approaches with a lower hamming loss for affective annotation.

Probabilistic linear discriminant analysis (PLDA) is a very effective feature extraction approach and has obtained extensive and successful applications in supervised learning tasks. It employs the squared$L_{2}$-norm to measure the model errors, which assumes a Gaussian noise distribution implicitly. However, the noise in real-life applications may not follow a Gaussian distribution. Particularly, the squared$L_{2}$-norm could extremely exaggerate data outliers. To address this issue, this article proposes a robust PLDA model under the assumption of a Laplacian noise distribution, called L1-PLDA. The learning process employs the approach by expressing the Laplacian density function as a superposition of an infinite number of Gaussian distributions via introducing a new latent variable and then adopts the variational expectation–maximization (EM) algorithm to learn parameters. The most significant advantage of the new model is that the introduced latent variable can be used to detect data outliers. The experiments on several public databases show the superiority of the proposed L1-PLDA model in terms of classification and outlier detection.

Hyperspectral unmixing (HU) has been one of the most significant tasks in hyperspectral image (HSI) processing. In recent years, nonnegative matrix factorization (NMF) has received great attention in the HU due to its simultaneous estimation, flexible modeling, and little requirement on prior information. However, several common NMF algorithms still suffer from high computational complexity, instability, and low convergence rate. Motivated by the matrix manifold theory, this article proposes a new sparse oblique-manifold ($\mathcal {OB}$) NMF method from the perspective of matrix manifold. The critical idea of the proposed method is to regard the abundance matrix as locating on the oblique manifold, which eliminates its constraint of nonnegativity and sum-to-one and incorporates its intrinsic Riemannian geometry. Meanwhile, the$L_{1/2}$-norm on the Euclidean space can be transformed equivalently into the$L_{1}$-norm on oblique manifold. Then, via solving this sparse$\mathcal {OB}$NMF by the Riemannian conjugated gradient (RCG) algorithm and the multiplicative iterative rule, the proposed method not only ensures improvement in the solution accuracy but also leads to a much faster convergence rate. Experimental results from the synthetic and real-world datasets illustrate the effectiveness and efficiency of the proposed method compared with the state-of-the-art NMF methods in HU.

Self-expressiveness based subspace clustering methods have received wide attention for unsupervised learning tasks. However, most existing subspace clustering methods consider data features as a whole and then focus only on one single self-representation. These approaches ignore the intrinsic multi-attribute information embedded in the original data feature and result in one-attribute self-representation. This paper proposes a novel multi-attribute subspace clustering (MASC) model that understands data from multiple attributes. MASC simultaneously learns multiple subspace representations corresponding to each specific attribute by exploiting the intrinsic multi-attribute features drawn from original data. In order to better capture the high-order correlation among multi-attribute representations, we represent them as a tensor in low-rank structure and propose the auto-weighted tensor nuclear norm (AWTNN) as a superior low-rank tensor approximation. Especially, the non-convex AWTNN fully considers the difference between singular values through the implicit and adaptive weights splitting during the AWTNN optimization procedure. We further develop an efficient algorithm to optimize the non-convex and multi-block MASC model and establish the convergence guarantees. A more comprehensive subspace representation can be obtained via aggregating these multi-attribute representations, which can be used to construct a clustering-friendly affinity matrix. Extensive experiments on eight real-world databases reveal that the proposed MASC exhibits superior performance over other subspace clustering methods.

Traffic forecasting is a challenging problem in the transportation research field as the complexity and non-stationary changing of the traffic data, thus the key to the issue is how to explore proper spatial and temporal characteristics. Based on this thought, many creative methods have been proposed, in which Graph Convolution Network (GCN) based methods have shown promising performance. However, these methods depend on the graph construction, which mainly uses the prior knowledge of the road network. Recently, some works realized the fact of the road network graph changing and tried to construct dynamic graphs for GCN, but they do not fully exploit the spatial and temporal properties of the traffic data in the graph construction. In this paper, we propose a novel dynamic graph convolution network for traffic forecasting, in which a latent network is introduced to extract spatial-temporal features for constructing the dynamic road network graph matrices adaptively. The proposed method is evaluated on several traffic datasets and the experimental results show that it outperforms the state of the art traffic forecasting methods. The website of the code ishttps://github.com/guokan987/DGCN.git.

Recently, Graph Convolution Network (GCN) and Temporal Convolution Network (TCN) are introduced into traffic prediction and achieve state-of-the-art performance due to their good ability for modeling the spatial and temporal property of traffic data. In spite of having good performance, the current methods generally focus on the traffic measurement of road segments, i.e. the nodes of traffic flow graph, while the edges of the graph, which represent the correlation of traffic data of different road segments and form the affinity matrix for GCN, are usually constructed according to the structure of road network, but the spatial and temporal properties are not well exploited in their theories. In this paper, we propose a Dual Dynamic Spatial-Temporal Graph Convolution Network (DDSTGCN), which not only models the dynamic property of the nodes of the traffic flow graph but also captures the dynamic spatial-temporal feature of the edges by transforming the traffic flow graph into its dual hypergraph. The traffic prediction is enhanced by the collaborative convolutions on the traffic flow graph and its dual hypergraph. The proposed method is evaluated by extensive traffic prediction experiments on six real road datasets and the results show that it outperforms state-of-the-art related methods. Source codes are available athttps://github.com/j1o2h3n/DDSTGCN.

Data similarity (or distance) computation is a fundamental research topic which underpins many high-level applications based on similarity measures in machine learning and data mining. However, in large-scale real-world scenarios, the exact similarity computation has become daunting due to “3V” nature (volume, velocity and variety) of big data. In this case, the hashing techniques have been verified to efficiently conduct similarity estimation in terms of both theory and practice. Currently, MinHash is a popular technique for efficiently estimating the Jaccard similarity of binary sets and furthermore, weighted MinHash is generalized to estimate the generalized Jaccard similarity of weighted sets. This review focuses on categorizing and discussing the existing works of weighted MinHash algorithms. In this review, we mainly categorize the weighted MinHash algorithms into quantization-based approaches, “active index”-based ones and others, and show the evolution and inherent connection of the weighted MinHash algorithms, from the integer weighted MinHash ones to the real-valued weighted MinHash ones. Also, we have developed a Python toolbox for the algorithms, and released it in our github. We experimentally conduct a comprehensive study of the standard MinHash algorithm and the weighted MinHash ones in the similarity estimation error and the information retrieval task.

Multiview subspace clustering has been demonstrated to achieve excellent performance in practice by exploiting multiview complementary information. One of the strategies used in most existing methods is to learn a shared self-expressiveness coefficient matrix for all the view data. Different from such a strategy, this article proposes a rank consistency induced multiview subspace clustering model to pursue a consistent low-rank structure among view-specific self-expressiveness coefficient matrices. To facilitate a practical model, we parameterize the low-rank structure on all self-expressiveness coefficient matrices through the tri-factorization along with orthogonal constraints. This specification ensures that self-expressiveness coefficient matrices of different views have the same rank to effectively promote structural consistency across multiviews. Such a model can learn a consistent subspace structure and fully exploit the complementary information from the view-specific self-expressiveness coefficient matrices, simultaneously. The proposed model is formulated as a nonconvex optimization problem. An efficient optimization algorithm with guaranteed convergence under mild conditions is proposed. Extensive experiments on several benchmark databases demonstrate the advantage of the proposed model over the state-of-the-art multiview clustering approaches.

In deep clustering frameworks, autoencoder (AE)- or variational AE-based clustering approaches are the most popular and competitive ones that encourage the model to obtain suitable representations and avoid the tendency for degenerate solutions simultaneously. However, for the clustering task, the decoder for reconstructing the original input is usually useless when the model is finished training. The encoder-decoder architecture limits the depth of the encoder so that the learning capacity is reduced severely. In this article, we propose a decoder-free variational deep embedding for unsupervised clustering (DFVC). It is well known that minimizing reconstruction error amounts to maximizing a lower bound on the mutual information (MI) between the input and its representation. That provides a theoretical guarantee for us to discard the bloated decoder. Inspired by contrastive self-supervised learning, we can directly calculate or estimate the MI of the continuous variables. Specifically, we investigate unsupervised representation learning by simultaneously considering the MI estimation of continuous representations and the MI computation of categorical representations. By introducing the data augmentation technique, we incorporate the original input, the augmented input, and their high-level representations into the MI estimation framework to learn more discriminative representations. Instead of matching to a simple standard normal distribution adversarially, we use end-to-end learning to constrain the latent space to be cluster-friendly by applying the Gaussian mixture distribution as the prior. Extensive experiments on challenging data sets show that our model achieves higher performance over a wide range of state-of-the-art clustering approaches.

Traffic forecasting is attracting considerable interest due to its widespread application in intelligent transportation systems. Given the complex and dynamic traffic data, many methods focus on how to establish a spatial-temporal model to express the non-stationary traffic patterns. Recently, the latest Graph Convolution Network (GCN) has been introduced to learn spatial features while the time neural networks are used to learn temporal features. These GCN based methods obtain state-of-the-art performance. However, the current GCN based methods ignore the natural hierarchical structure of traffic systems which is composed of the micro layers of road networks and the macro layers of region networks, in which the nodes are obtained through pooling method and could include some hot traffic regions such as downtown and CBD etc., while the current GCN is only applied on the micro graph of road networks. In this paper, we propose a novel Hierarchical Graph Convolution Networks (HGCN) for traffic forecasting by operating on both the micro and macro traffic graphs. The proposed method is evaluated on two complex city traffic speed datasets. Compared to the latest GCN based methods like Graph WaveNet, the proposed HGCN gets higher traffic forecasting precision with lower computational cost.The website of the code is https://github.com/guokan987/HGCN.git.

With the prevalence of smart devices, billions of people are accessing digital resource in their daily life. Online user-behavior modeling, as such, has been actively researched in recent years. However, due to the data uncertainty (sparse-ness and skewness), traditional techniques suffer from certain drawbacks, such as relying on labor-intensive expertise or prior knowledge, lacking of interpretability and transparency, and expensive computational cost. As a step toward bridging the gap, this paper proposes a fuzzy-set based contrastive learning algorithm. The general idea is to design an end-to-end learning framework of optimizing representation from contrastive samples. The proposed algorithm is characterized by three main modules, including data augmentation, fuzzy encoder, and semi-supervised optimization. More precisely, data augmentation is used to produce contrastive (positive and negative) samples based on anchor ones. The fuzzy encoder is introduced to fuzzify (or encode) latent representation of those contrastive samples, while the semi-supervised learning is then implemented to optimize the fuzzy encoder and minimize the training loss simultaneously. The advantage of the proposed algorithm includes no requirement of domain knowledge, preserving the transparency and interpretability of result, and computational effectiveness. Experimental results, based on a real-world app usage dataset, demonstrate that the applicability and flexibility of the proposed algorithm, compared with other state-of-the-art methods.

This paper presents a new approach for assembling graph neural networks based on framelet transforms. The latter provides a multi-scale representation for graph-structured data. We decompose an input graph into low-pass and high-pass frequencies coefficients for network training, which then defines a framelet-based graph convolution. The framelet decomposition naturally induces a graph pooling strategy by aggregating the graph feature into low-pass and high-pass spectra, which considers both the feature values and geometry of the graph data and conserves the total information. The graph neural networks with the proposed framelet convolution and pooling achieve state-of-the-art performance in many node and graph prediction tasks. Moreover, we propose shrinkage as a new activation for the framelet convolution, which thresholds high-frequency information at different scales. Compared to ReLU, shrinkage activation improves model performance on denoising and signal compression: noises in both node and structure can be significantly reduced by accurately cutting off the high-pass coefficients from framelet decomposition, and the signal can be compressed to less than half its original size with well-preserved prediction performance.

We propose a stochastic recursive momentum method for Riemannian non-convex optimization that achieves a nearly-optimal complexity to find epsilon-approximate solution with one sample. The new algorithm requires one-sample gradient evaluations per iteration and does not require restarting with a large batch gradient, which is commonly used to obtain a faster rate. Extensive experiment results demonstrate the superiority of the proposed algorithm. Extensions to nonsmooth and constrained optimization settings are also discussed.

In high dimensional tensorial time series prediction, it is highly desired to preserve spatial structural and underlying continuous sequential information in modelling. Existing methods either destroy the spatial structure and require a large number of parameters, such as neural ordinary differential equations (neural ODEs), or cannot capture the temporal information, especially for unevenly spaced time series, such as tensorial neural network families. Hence we propose Tensor Neural Ordinary Differential Equations (TENODEs) to address these issues. The dynamics of data is modelled by a tensorial neural network (TNN) consisting of the Tucker structure. Compared with neural ODEs, it reduces the number of parameters from$O(I^{2N})$to$O(NI^{2})$. To further ease the computational cost, we also propose a TENODE with further dimensionality reduction which produces a low-dimensional representation of the aforementioned two pieces of information and is projected to the target space by a 1-layer TNN. In each layer of the RHS for TENODE, the weight matrices should maintain Lipschitzness according to the Picard's theorem. Hence TENODEs are consistent and converge. Multiplication of weight matrices together can affect the optimisation stability: when values of some weights are enlarged, values of the other weights are correspondingly decreased, with the loss unchanged. This may create infinite solution spaces. We thus constrain weights to be orthogonal and solve this scaling issue to enhance the optimisation process. In consequence, the proposed TEN-ODEs successfully preserve data spatial and latent continuous sequential information. Experimental results demonstrate the efficacy of our proposed methods over competitive baselines.

Profiting from the pre-trained language representation models like BERT, the recently proposed document classification methods have obtained considerable improvement. However, most of these methods usually model the document as a sequence of text and omit the structure information, which appears obviously in long document composed of several sections with assigned relations. For this purpose, we propose a novel Hierarchical Attention Transformer Network (HATN) for long document classification, which extracts the structure of the long document by intra- and inter-section attention transformers, and further strengths the feature interaction by two fusion gates: the Residual Fusion Gate (RFG) and the Feature Fusion Gate (FFG). The proposed method is evaluated on three long document datasets and the experimental results show that our approach outperforms the related state-of-the-art methods. The code will be available at https://github.com/TengfeiLiu966/HATN

Prediction performance of stock price movement is impacted significantly by choice of input features and prediction models. Expert opinion-based input features such as previous prices or technical indicators have been used aplenty for stock price movement prediction. This study focuses on a specific event of ex-dividend day and uses event specific input features of cum-dividend period for predicting price movement on the exdividend day. These input features are relevant only around the ex-dividend period but mostly not in other periods. Different feature selection and extraction techniques are used further to select optimum input features. Performance improves significantly when these event specific optimum input features are used in conjunction with machine learning models compared to the combination of expert opinion-based features and non-machine learning models.

In this paper, we comparatively analyze the Bures-Wasserstein (BW) geometry with the popular Affine-Invariant (AI) geometry for Riemannian optimization on the symmetric positive definite (SPD) matrix manifold. Our study begins with an observation that the BW metric has a linear dependence on SPD matrices in contrast to the quadratic dependence of the AI metric. We build on this to show that the BW metric is a more suitable and robust choice for several Riemannian optimization problems over ill-conditioned SPD matrices. We show that the BW geometry has a non-negative curvature, which further improves convergence rates of algorithms over the non-positively curved AI geometry. Finally, we verify that several popular cost functions, which are known to be geodesic convex under the AI geometry, are also geodesic convex under the BW geometry. Extensive experiments on various applications support our findings.

Abstract We propose that the square root of a probability density function can be represented as a linear combination of Gaussian kernels. It is shown that if the Gaussian kernel centres and kernel width are known, then the maximum likelihood parameter estimator can be formulated as a Riemannian optimisation problem on sphere manifold. The first order Riemannian geometry of the sphere manifold and vector transport are initially explored, then the well‐known Riemannian conjugate gradient algorithm is used to estimate the model parameters. For completeness, the k‐means clustering algorithm and a grid search are employed to determine the centres and kernel width, respectively. Simulated examples are employed to demonstrate that the proposed approach is effective in constructing the estimate of the square root of probability density function.

Abstract This study proposes a novel multi‐view soft block diagonal representation framework for clustering complete and incomplete multi‐view data. First, given that the multi‐view self‐representation model offers better performance in exploring the intrinsic structure of multi‐view data, it can be nicely adopted to individually construct a graph for each view. Second, since an ideal block diagonal graph is beneficial for clustering, a ‘soft’ block diagonal affinity matrix is constructed by fusing multiple previous graphs. The soft diagonal block regulariser encourages a matrix to approximately have (not exactly) diagonal blocks, where is the number of clusters. This strategy adds robustness to noise and outliers. Third, to handle incomplete multi‐view data, multiple indicator matrices are utilised, which can mark the position of missing elements of each view. Finally, the alternative direction of multipliers algorithm is employed to optimise the proposed model, and the corresponding algorithm complexity and convergence are also analysed. Extensive experimental results on several real‐world datasets achieve the best performance among the state‐of‐the‐art complete and incomplete clustering methods, which proves the effectiveness of the proposed methods.

With the popularity of cameras and sensors, massive data are captured from various view angles or modalities, which provide abundant complementary information and also bring great challenges for traditional clustering methods. In this article, we propose a novel Adaptive K-Multiple-Means for multi-view clustering method (AKM3C). Unlike traditional multi-view K-means methods by grouping samples into$C$clusters each with a cluster center in every view, the proposed AKM3C employs$M (M>C)$sub-cluster centers in each view to reveal the sub-cluster structure in the multi-view data thus enhances the clustering performance. Additionally, to distinguish the importance of different views, instead of using empirical weights, AKM3C exploits the multi-view combination weights strategy to assign a weight to each view automatically and thus fuses the complementary information of different views properly to get an optimally shared bipartite graph, on which the Laplacian rank constraint is executed and the final clusters are obtained by directly partitioning. An efficient optimization algorithm proposed with complexity and convergence analysis is used to solve the proposed AKM3C method. The extensive experimental results on eight public datasets show that the proposed AKM3C performs better than state-of-the-art multi-view clustering methods. The code can be downloaded athttps://drive.google.com/file/d/1CQ0royrYxKFJdNLnbBQSbDrohtfH71di/view?usp=sharing.

Traffic prediction is a core problem in the intelligent transportation system and has broad applications in the transportation management and planning, and the main challenge of this field is how to efficiently explore the spatial and temporal information of traffic data. Recently, various deep learning methods, such as convolution neural network (CNN), have shown promising performance in traffic prediction. However, it samples traffic data in regular grids as the input of CNN, thus it destroys the spatial structure of the road network. In this paper, we introduce a graph network and propose an optimized graph convolution recurrent neural network for traffic prediction, in which the spatial information of the road network is represented as a graph. Additionally, distinguishing with most current methods using a simple and empirical spatial graph, the proposed method learns an optimized graph through a data-driven way in the training phase, which reveals the latent relationship among the road segments from the traffic data. Lastly, the proposed method is evaluated on three real-world case studies, and the experimental results show that the proposed method outperforms state-of-the-art traffic prediction methods.

Locality preserving projection (LPP) is a dimensionality reduction algorithm preserving the neighhorhood graph structure of data. However, the conventional LPP is sensitive to outliers existing in data. This article proposes a novel low-rank LPP model called LR-LPP. In this new model, original data are decomposed into the clean intrinsic component and noise component. Then the projective matrix is learned based on the clean intrinsic component which is encoded in low-rank features. The noise component is constrained by theℓ1-norm which is more robust to outliers. Finally, LR-LPP model is extended to LR-FLPP in which low-dimensional feature is measured by F-norm. LR-FLPP will reduce aggregated error and weaken the effect of outliers, which will make the proposed LR-FLPP even more robust for outliers. The experimental results on public image databases demonstrate the effectiveness of the proposed LR-LPP and LR-FLPP.

Short-term monitoring of lesion changes has been a widely accepted clinical guideline for melanoma screening. When there is a significant change of a melanocytic lesion at three months, the lesion will be excised to exclude melanoma. However, the decision on change or no-change heavily depends on the experience and bias of individual clinicians, which is subjective. For the first time, a novel deep learning based method is developed in this paper for automatically detecting short-term lesion changes in melanoma screening. The lesion change detection is formulated as a task measuring the similarity between two dermoscopy images taken for a lesion in a short time-frame, and a novel Siamese structure based deep network is proposed to produce the decision: changed (i.e. not similar) or unchanged (i.e. similar enough). Under the Siamese framework, a novel structure, namely Tensorial Regression Process, is proposed to extract the global features of lesion images, in addition to deep convolutional features. In order to mimic the decision-making process of clinicians who often focus more on regions with specific patterns when comparing a pair of lesion images, a segmentation loss (SegLoss) is further devised and incorporated into the proposed network as a regularization term. To evaluate the proposed method, an in-house dataset with 1,000 pairs of lesion images taken in a short time-frame at a clinical melanoma centre was established. Experimental results on this first-of-a-kind large dataset indicate that the proposed model is promising in detecting the short-term lesion change for objective melanoma screening.

The objective of self-expression based spectral clustering is to learn an affinity matrix which accurately reflects the similarity among data, and the Laplacian constraint is usually exploited to make the affinity matrix preserve the global structure of raw data. However, there exist two drawbacks: firstly, these methods are mostly designed for vectorial data in Euclidean spaces, which are not suitable for multidimensional data with nonlinear manifold structure, e.g., videos and image-sets. Secondly, the clustering performance heavily relies on the quality of a pre-learned Laplacian matrix in which the global structure may be mis-interpreted without considering manifold structures. In this paper, we firstly provide a unified framework about self-expression learning on Grassmann manifolds, which implements the clustering tasks for multidimensional data under subspace views. Then, to assign optimal neighbors to each data depending on the local distance, we adaptively learn the neighborhood relationship from the obtained self-expression coefficient matrix, referred to Learning Adaptive Neighborhood Graph on Grassmann manifolds (GMAN). In the optimization process, the neighborhood relationship can be adaptively learned and updated with the coefficient matrix. The experimental results on five public datasets show that the proposed method is obviously better than many related clustering methods based on Grassmann manifolds, proving the effectiveness of GMAN in multidimensional data clustering.

In machine learning, it is common to interpret each data sample as a multivariate vector disregarding the correlations among covariates. However, the data may actually be functional, i.e., each data point is a function of some variable, such as time, and the function is discretely sampled. The naive treatment of functional data as traditional multivariate data can lead to poor performance due to the correlations. In this article, we focus on subspace clustering for functional data or curves and propose a new method robust to shift and rotation. The idea is to define a function or curve and all its versions generated by shift and rotation as an equivalent class and then to find the subspace structure among all equivalent classes as the surrogate for all curves. Experimental evaluation on synthetic and real data reveals that this method massively outperforms prior clustering methods in both speed and accuracy when clustering functional data.

Multiview clustering (MVC) has recently received great interest due to its pleasing efficacy in combining the abundant and complementary information to improve clustering performance, which overcomes the drawbacks of view limitation existed in the standard single-view clustering. However, the existing MVC methods are mostly designed for vectorial data from linear spaces and, thus, are not suitable for multiple dimensional data with intrinsic nonlinear manifold structures, e.g., videos or image sets. Some works have introduced manifolds' representation methods of data into MVC and obtained considerable improvements, but how to fuse multiple manifolds efficiently for clustering is still a challenging problem. Particularly, for heterogeneous manifolds, it is an entirely new problem. In this article, we propose to represent the complicated multiviews' data as heterogeneous manifolds and a fusion framework of heterogeneous manifolds for clustering. Different from the empirical weighting methods, an adaptive fusion strategy is designed to weight the importance of different manifolds in a data-driven manner. In addition, the low-rank representation is generalized onto the fused heterogeneous manifolds to explore the low-dimensional subspace structures embedded in data for clustering. We assessed the proposed method on several public data sets, including human action video, facial image, and traffic scenario video. The experimental results show that our method obviously outperforms a number of state-of-the-art clustering methods.

Person re-identification (re-ID) favors discriminative representations over unseen shots to recognize identities in disjoint camera views. Effective methods are developed via pair-wise similarity learning to detect a fixed set of region features, which can be mapped to compute the similarity value. However, relevant parts of each image are detected independently without referring to the correlation on the other image. Also, region-based methods spatially position local features for their aligned similarities. In this article, we introduce the deep coattention-based comparator (DCC) to fuse codependent representations of paired images so as to correlate the best relevant parts and produce their relative representations accordingly. The proposed approach mimics the human foveation to detect the distinct regions concurrently across images and alternatively attends to fuse them into the similarity learning. Our comparator is capable of learning representations relative to a test shot and well-suited to reidentifying pedestrians in surveillance. We perform extensive experiments to provide the insights and demonstrate the state of the arts achieved by our method in benchmark data sets: 1.2 and 2.5 points gain in mean average precision (mAP) on DukeMTMC-reID and Market-1501, respectively.

Clustering multi-view data has been a fundamental research topic in the computer vision community. It has been shown that a better accuracy can be achieved by integrating information of all the views than just using one view individually. However, the existing methods often struggle with the issues of dealing with the large-scale datasets and the poor performance in reconstructing samples. This paper proposes a novel multi-view clustering method by learning a shared generative latent representation that obeys a mixture of Gaussian distributions. The motivation is based on the fact that the multi-view data share a common latent embedding despite the diversity among the various views. Specifically, benefitting from the success of the deep generative learning, the proposed model can not only extract the nonlinear features from the views, but render a powerful ability in capturing the correlations among all the views. The extensive experimental results on several datasets with different scales demonstrate that the proposed method outperforms the state-of-the-art methods under a range of performance criteria.

As an effective descriptor, Symmetric Positive Definite (SPD) matrix is widely used in several areas such as image clustering. Recently, researchers proposed some effective methods based on low rank theory for SPD data clustering with nonlinear metric. However, single nuclear norm is always adopted to formulate the low rank model in these methods, which would lead to suboptimal solution. In this paper, we proposed a novel double low rank representation method for SPD clustering problem, in which matrix factorization and nonconvex rank constraint are combined to reveal the intrinsic property of the data instead of employing the nuclear norm. Meanwhile, kernel method and Log-Euclidean metric are combined to better explore the intrinsic geometry within SPD data. The proposed method has been evaluated on several public datasets and the experimental results demonstrate that the proposed method outperforms the state-of-the-art ones.

Clustering high dimension multi-view data with complex intrinsic properties and nonlinear manifold structure is a challenging task since these data are always embedded in low dimension manifolds. Inspired by Low Rank Representation (LRR), some researchers extended classic LRR on Grassmann manifold or Product Grassmann manifold to represent data with non-linear metrics. However, most of these methods utilized convex nuclear norm to leverage a low-rank structure, which was over-relaxation of true rank and would lead to the results deviated from the true underlying ones. And, the computational complexity of singular value decomposition of matrix is high for nuclear norm minimization. In this paper, we propose a new low rank model for high-dimension multi-view data clustering on Product Grassmann Manifold with the matrix tri-factorization which is used to control the upper bound of true rank of representation matrix. And, the original problem can be transformed into the nuclear norm minimization with smaller scale matrices. An effective solution and theoretical analysis are also provided. The experimental results show that the proposed method obviously outperforms other state-of-the-art methods on several multi-source human/crowd action video datasets.

Activation in deep neural networks is fundamental to achieving non-linear mappings. Traditional studies mainly focus on finding fixed activations for a particular set of learning tasks or model architectures. The research on flexible activation is quite limited in both designing philosophy and application scenarios. In this study, three principles of choosing flexible activation components are proposed and a general combined form of flexible activation functions is implemented. Based on this, a novel family of flexible activation functions that can replace sigmoid or tanh in LSTM cells are implemented, as well as a new family by combining ReLU and ELUs. Also, two new regularisation terms based on assumptions as prior knowledge are introduced. It has been shown that LSTM models with proposed flexible activations P-Sig- Ramp provide significant improvements in time series forecasting, while the proposed P-E2-ReLU achieves better and more stable performance on lossy image compression tasks with convolutional auto-encoders. In addition, the proposed regularization terms improve the convergence, performance and stability of the models with flexible activation functions. The code for this paper is available at https://github.com/9NXJRDDRQK/Flexible_Activation.

As a challenging task of Natural Language Processing(NLP), zero-shot text classification has attracted more and more attention recently. It aims to detect classes that the model has never seen in the training set. For this purpose, a feasible way is to construct connection between the seen and unseen classes by semantic extension and classify the unseen classes by information propagation over the connection. Although many related zero-shot text classification methods have been exploited, how to realize semantic extension properly and propagate information effectively are far from solved. In this paper, we propose a novel zero-shot text classification method called Semantically Extended Graph Convolutional Network (SEGCN). In the proposed method, the semantic category knowledge from ConceptNet is utilized to semantic extension for linking seen classes to unseen classes and constructing a graph of all categories. Then, we build upon Graph Convolutional Network (GCN) for predicting the textual classifier for each category, which transfers the category knowledge by the convolution operators on the constructed graph and is trained in a semi-supervised manner using the samples of the seen classes. The experimental results on Dbpedia and 20newsgroup datasets show that our method outperforms the state of the art zero-shot text classification methods.

Data clustering is a fundamental topic in machine learning and data mining areas. In recent years, researchers have proposed a series of effective methods based on Low Rank Representation (LRR) which could explore low-dimension subspace structure embedded in original data effectively. The traditional LRR methods usually are designed for vectorial data from linear spaces with Euclidean distance. However, high-dimension data (such as video clip or imageset) are always considered as non-linear manifold data such as Grassmann manifold with non-linear metric. In addition, traditional LRR clustering method always adopt single nuclear norm as low rank constraint which would lead to suboptimal solution and decrease the clustering accuracy. In this paper, we proposed a new low rank method on Grassmann manifold for video or imageset data clustering task. In the proposed method, video or imageset data are formulated as sample data on Grassmann manifold first. And then a double low rank constraint is proposed by combining the nuclear norm and bilinear representation for better construct the representation matrix. The experimental results on several public datasets show that the proposed method outperforms the state-of-the-art clustering methods.

Hyperbolic space is a well-defined space with constant negative curvature. Recent research demonstrates its odds of capturing complex hierarchical structures with its exceptional high capacity and continuous tree-like properties. This paper bridges hyperbolic space's superiority to the power-law structure of documents by introducing a hyperbolic neural network architecture named Hyperbolic Hierarchical Attention Network (Hype-HAN). Hype-HAN defines three levels of embeddings (word/sentence/document) and two layers of hyperbolic attention mechanism (word-to-sentence/sentence-to-document) on Riemannian geometries of the Lorentz model, Klein model and Poincaré model. Situated on the evolving embedding spaces, we utilize both conventional GRUs (Gated Recurrent Units) and hyperbolic GRUs with Möbius operations. Hype-HAN is applied to large scale datasets. The empirical experiments show the effectiveness of our method.

In this paper a novel nonlinear logistic regression model based on a simplex basis function neural network is introduced that outputs probability of categorical variables in response to multiple predictors. It is shown that since a linear combination of the simplex basis functions can be represented as a piecewise linear model, the proposed nonlinear logistic regression model retains the main advantage of linear logistic regression model, that is, allowing probabilistic interpretation of the data sets from an identified model. The associated estimation problem is treated based on the principle of maximum likelihood by alternating over two algorithms; the iteratively reweighted least squares algorithm for linear parameters, while the simplex basis functions are fixed; then nonlinear parameters in each simplex basis function are adapted in turn based on gradient descent of the negative likelihood. The proposed algorithm is then extended to estimation of nonlinear multinomial logistic model. Numerical experiments are initially carried out to illustrate the advantage of nonlinear logistic regression model versus its linear counterpart in terms of approximation capability. Then we apply the proposed method for a difficult computer vision example of land-cover real data set.

1-bit matrix completion plays a significant role in recommender systems. However, conventional 1-bit methods do not consider the interactive clustering nature of customers and products. To tackle the gap, we introduce Group-Specific 1-bit Matrix Completion (GS1MC) by first-time consolidating cluster effects in 1-bit matrix completion tasks. To empower GS1MC even when grouping information is unobtainable, GS1MC is extended as Cluster Developing Matrix Completion (CDMC) by integrating subspace clustering techniques. CDMC can group entities and utilize their cluster effects simultaneously. Experiments show that GS1MC outperforms current 1-bit matrix completion methods and CDMC successfully captures items' genre features only with sparse binary rating data. Notably, GS1MC provides a new insight to incorporate and evaluate the efficacy of different clustering methods while CDMC can be served as a new tool to explore unrevealed social behavior or market phenomena.

Adam-type optimizers, as a class of adaptive moment estimation methods with the exponential moving average scheme, have been successfully used in many applications of deep learning. Such methods are appealing due to the capability on large-scale sparse datasets with high computational efficiency. In this paper, we present a new framework for Adam-type methods with the trend information when updating the parameters with the adaptive step size and gradients. The additional terms in the algorithm promise an efficient movement on the complex cost surface, and thus the loss would converge more rapidly. We show empirically the importance of adding the trend component, where our framework outperforms the conventional Adam and AMSGrad methods constantly on the classical models with several real-world datasets.

Linear discriminant analysis (LDA) is a well-known supervised method for dimensionality reduction in which the global structure of data can be preserved. The classical LDA is sensitive to the noises, and the projection direction of LDA cannot preserve the main energy. This article proposes a novel feature extraction model with l 2,1 norm constraint based on LDA, termed as RALDA. This model preserves within-class local structure in the latent subspace according to the label information. To reduce information loss, it learns a projection matrix and an inverse projection matrix simultaneously. By introducing an implicit variable and matrix norm transformation, the alternating direction multiple method with updating variables is designed to solve the RALDA model. Moreover, both computational complexity and weak convergence property of the proposed algorithm are investigated. The experimental results on several public databases have demonstrated the effectiveness of our proposed method.

Unsupervised clustering for high-dimension data (such as imageset or video) is a hard issue in data processing and data mining area since these data always lie on a manifold (such as Grassmann manifold). Inspired of Low Rank representation theory, researchers proposed a series of effective clustering methods for high-dimension data with non-linear metric. However, most of these methods adopt the traditional single nuclear norm as the relaxation of the rank function, which would lead to suboptimal solution deviated from the original one. In this paper, we propose a new low rank model for high-dimension data clustering task on Grassmann manifold based on the Double Nuclear norm which is used to better approximate the rank minimization of matrix. Further, to consider the inner geometry or structure of data space, we integrated the adaptive Laplacian regularization to construct the local relationship of data samples. The proposed models have been assessed on several public datasets for imageset clustering. The experimental results show that the proposed models outperform the state-of-the-art clustering ones.

Dimensionality reduction is an essential problem in data mining and machine learning fields. Locality Preserving Projection (LPP) is a well-known dimensionality reduction method which can preserve the neighborhood graph structure of data, and has achieved promising performance. However linear projection makes it difficult to analyze complex data with nonlinear structure. In order to deal with this issue, this paper proposes a novel nonlinear locality preserving projection method via deep neural network, termed as DNLPP, which replaces the linear projection with an appropriate deep neural network. Benefiting from the nonlinearity of neural networks and its powerful representation capability, the proposed method is more discriminative than the conventional LPP. In order to solve the new model, we propose an iterative optimization algorithm. Extensive experiments on several public datasets illustrate that the proposed method is overall superior to the other state-of-art dimensionality reduction methods.

Fine-grained visual recognition is an important problem in pattern recognition applications. However, it is a challenging task due to the subtle interclass difference and large intraclass variation. Recent visual attention models are able to automatically locate critical object parts and represent them against appearance variations. However, without consideration of spatial dependencies in discriminative feature learning, these methods are underperformed in classifying fine-grained objects. In this paper, we present a deep attention-based spatially recursive model that can learn to attend to critical object parts and encode them into spatially expressive representations. Our network is technically premised on bilinear pooling, enabling local pairwise feature interactions between outputs from two different convolutional neural networks (CNNs) that correspond to distinct region detection and relevant feature extraction. Then, spatial long-short term memory (LSTMs) units are introduced to generate spatially meaningful hidden representations via the long-range dependency on all features in two dimensions. The attention model is leveraged between bilinear outcomes and spatial LSTMs for dynamic selection on varied inputs. Our model, which is composed of two-stream CNN layers, bilinear pooling, and spatial recursive encoding with attention, is end-to-end trainable to serve as the part detector and feature extractor whereby relevant features are localized, extracted, and encoded spatially for recognition purpose. We demonstrate the superiority of our method over two typical fine-grained recognition tasks: fine-grained image classification and person re-identification.

In this paper, we present a novel Bayesian classification framework of the matrix variate Bingham distributions with the inclusion of its normalizing constant and develop a consistent general parametric modeling framework based on the Grassmann manifolds. To calculate the normalizing constants of the Bingham model, this paper extends the method of saddle-point approximation (SPA) to a new setting. Furthermore, it employs the standard theory of maximum likelihood estimation (MLE) to evaluate the involved parameters in the used probability density functions. The validity and performance of the proposed approach are tested on 14 real-world visual classification databases. We have compared the classification performance of our proposed approach with the baselines from the previous related approaches. The comparison shows that on most of the databases, the performance of our approach is superior.

Dimensionality reduction is widely used to deal with high-dimensional data. As a famous dimensionality reduction method, principal component analysis (PCA) aiming at finding the low dimension feature of original data has made great successes, and many improved PCA algorithms have been proposed. However, most algorithms based on PCA only consider the linear correlation of data features. In this article, we propose a novel dimensionality reduction model called maximally correlated PCA based on deep parameterization learning (MCPCADP), which takes nonlinear correlation into account in the deep parameterization framework for the purpose of dimensionality reduction. The new model explores nonlinear correlation by maximizing Ky-Fan norm of the covariance matrix of nonlinearly mapped data features. A new BP algorithm for model optimization is derived. In order to assess the proposed method, we conduct experiments on both a synthetic database and several real-world databases. The experimental results demonstrate that the proposed algorithm is comparable to several widely used algorithms.

Restricted Boltzmann machine (RBM) is a famous model for feature extraction and can be used as an initializer for neural networks. When applying the classic RBM to multidimensional data such as 2D/3D tensors, one needs to vectorize such as high-order data. Vectorizing will result in dimensional disaster and valuable spatial information loss. As RBM is a model with fully connected layers, it requires a large amount of memory. Therefore, it is difficult to use RBM with high-order data on low-end devices. In this article, to utilize classic RBM on tensorial data directly, we propose a new tensorial RBM model parameterized by the tensor train format (TTRBM). In this model, both visible and hidden variables are in tensorial form, which are connected by a parameter matrix in tensor train format. The biggest advantage of the proposed model is that TTRBM can obtain comparable performance compared with the classic RBM with much fewer model parameters and faster training process. To demonstrate the advantages of TTRBM, we conduct three real-world applications, face reconstruction, handwritten digit recognition, and image super-resolution in the experiments.

Video-based person re-identification (re-id) is a central application in surveillance systems with a significant concern in security. Matching persons across disjoint camera views in their video fragments are inherently challenging due to the large visual variations and uncontrolled frame rates. There are two steps crucial to person re-id, namely, discriminative feature learning and metric learning. However, existing approaches consider the two steps independently, and they do not make full use of the temporal and spatial information in the videos. In this paper, we propose a Siamese attention architecture that jointly learns spatiotemporal video representations and their similarity metrics. The network extracts local convolutional features from regions of each frame and enhances their discriminative capability by focusing on distinct regions when measuring the similarity with another pedestrian video. The attention mechanism is embedded into spatial gated recurrent units to selectively propagate relevant features and memorize their spatial dependencies through the network. The model essentially learns which parts (where) from which frames (when) are relevant and distinctive for matching persons and attaches higher importance therein. The proposed Siamese model is end-to-end trainable to jointly learn comparable hidden representations for paired pedestrian videos and their similarity value. Extensive experiments on three benchmark datasets show the effectiveness of each component of the proposed deep network while outperforming state-of-the-art methods.

Partial least squares regression (PLSR) has been a popular technique to explore the linear relationship between two data sets. However, all existing approaches often optimize a PLSR model in Euclidean space and take a successive strategy to calculate all the factors one by one for keeping the mutually orthogonal PLSR factors. Thus, a suboptimal solution is often generated. To overcome the shortcoming, this paper takes statistically inspired modification of PLSR (SIMPLSR) as a representative of PLSR, proposes a novel approach to transform SIMPLSR into optimization problems on Riemannian manifolds, and develops corresponding optimization algorithms. These algorithms can calculate all the PLSR factors simultaneously to avoid any suboptimal solutions. Moreover, we propose sparse SIMPLSR on Riemannian manifolds, which is simple and intuitive. A number of experiments on classification problems have demonstrated that the proposed models and algorithms can get lower classification error rates compared with other linear regression methods in Euclidean space. We have made the experimental code public at https://github.com/Haoran2014.

Linear discriminant analysis (LDA) has been a widely used supervised feature extraction and dimension reduction method in pattern recognition and data analysis. However, facing high-order tensor data, the traditional LDA-based methods take two strategies. One is vectorizing original data as the first step. The process of vectorization will destroy the structure of high-order data and result in high dimensionality issue. Another is tensor LDA-based algorithms that extract features from each mode of high-order data and the obtained representations are also high-order tensor. This paper proposes a new probabilistic LDA (PLDA) model for tensorial data, namely, tensor PLDA. In this model, each tensorial data are decomposed into three parts: the shared subspace component, the individual subspace component, and the noise part. Furthermore, the first two parts are modeled by a linear combination of latent tensor bases, and the noise component is assumed to follow a multivariate Gaussian distribution. Model learning is conducted through a Bayesian inference process. To further reduce the total number of model parameters, the tensor bases are assumed to have tensor CandeComp/PARAFAC (CP) decomposition. Two types of experiments, data reconstruction and classification, are conducted to evaluate the performance of the proposed model with the convincing result, which is superior or comparable against the existing LDA-based methods.

The ubiquitous information from multiple-view data, as well as the complementary information among different views, is usually beneficial for various tasks, for example, clustering, classification, denoising, and so on. Multiview subspace clustering is based on the fact that multiview data are generated from a latent subspace. To recover the underlying subspace structure, a successful approach adopted recently has been sparse and/or low-rank subspace clustering. Despite the fact that existing subspace clustering approaches may numerically handle multiview data, by exploring all possible pairwise correlation within views, high-order statistics that can only be captured by simultaneously utilizing all views are often overlooked. As a consequence, the clustering performance of the multiview data is compromised. To address this issue, in this paper, a novel multiview clustering method is proposed by using t-product in the third-order tensor space. First, we propose a novel tensor construction method to organize multiview tensorial data, to which the tensor-tensor product can be applied. Second, based on the circular convolution operation, multiview data can be effectively represented by a t-linear combination with sparse and low-rank penalty using "self-expressiveness." Our extensive experimental results on face, object, digital image, and text data demonstrate that the proposed method outperforms the state-of-the-art methods for a range of criteria.

-norm and rank function minimization problems. However, due to the absence of convexity in these nonconvex problems, developing efficient algorithms with convergence guarantee becomes very challenging. Inspired by the basic ideas of both the Jacobian alternating direction method of multipliers (JADMMs) for solving linearly constrained problems with separable objectives and the proximal gradient methods (PGMs) for optimizing the unconstrained problems with one variable, this paper focuses on extending the PGMs to the proximal Jacobian iteration methods (PJIMs) for handling with a family of nonconvex composite optimization problems with two splitting variables. To reduce the total computational complexity by decreasing the number of iterations, we devise the accelerated version of PJIMs through the well-known Nesterov's acceleration strategy and further extend both to solve the multivariable cases. Most importantly, we provide a rigorous convergence analysis, in theory, to show that the generated variable sequence globally converges to a critical point by exploiting the Kurdyka-Łojasiewica (KŁ) property for a broad class of functions. Furthermore, we also establish the linear and sublinear convergence rates of the obtained variable sequence in the objective function. As the specific application to the nonconvex sparse and low-rank recovery problems, several numerical experiments can verify that the newly proposed algorithms not only keep fast convergence speed but also have high precision.

Locality preserving projection (LPP) is a well-known method for dimensionality reduction in which the neighborhood graph structure of data is preserved. Traditional LPP employ squared F-norm for distance measurement. This may exaggerate more distance errors, and result in a model being sensitive to outliers. In order to deal with this issue, we propose two novel F-norm-based models, termed as F-LPP and F-2DLPP, which are developed for vector-based and matrix-based data, respectively. In F-LPP and F-2DLPP, the distance of data projected to a low dimensional space is measured by F-norm. Thus it is anticipated that both methods can reduce the influence of outliers. To solve the F-norm-based models, we propose an iterative optimization algorithm, and give the convergence analysis of algorithm. The experimental results on three public databases have demonstrated the effectiveness of our proposed methods.

Inspired by low rank representation and sparse subspace clustering acquiring success, ones attempt to simultaneously perform low rank and sparse constraints on the affinity matrix to improve the performance. However, it is just a trade-off between these two constraints. In this paper, we propose a novel Cascaded Low Rank and Sparse Representation (CLRSR) method for subspace clustering, which seeks the sparse expression on the former learned low rank latent representation. To make our proposed method suitable to multi-dimension or imageset data, we extend CLRSR onto Grassmann manifolds. An effective solution and its convergence analysis are also provided. The excellent experimental results demonstrate the proposed method is more robust than other state-of-the-art clustering methods on imageset data.

Dimensionality Reduction (DR) which tries to discover low-dimensional feature representation embedded into the high-dimensional observations are significant for data visualization and data preprocessing. However, most DR models are designed for vector-valued data while only few of them are for functional data where samples are considered as continuous data such as curves or surfaces compared to discrete vector-valued data. Motivated by Functional Principal Component Analysis (FPCA), which generalizes the idea of Principal Component Analysis (PCA) to the Hilbert space of square-integrable functions, in this paper we propose Functional Locality Preserving Projection (FLPP), where classic Locality Preserving Projection (LPP) is extended for functional data analysis. Different from FPCA which only focuses on the global structure, FLPP could preserve local manifold structure embedded into the functional data, thus FLPP is capable of dealing with noise data. Experimental results on both synthetic data and real-world data verify that FLPP outperforms FPCA and typical LPP.

Although machine learning and statistical methods have been extensively used to study cyclones, the prediction of cyclone trajectories remains a challenging problem. Matrix neural networks have the ability of handling spatial correlations in the data which made them suitable for image recognition tasks. Cyclone trajectories are defined by the latitude and the longitudes coordinates as a temporal sequence. Matrix neural networks are suitable for track prediction as the dataset can be conveniently given as input without vectorization that could result in loss of correlation of the spatial information. In this paper, matrix neural networks are used for cyclone track prediction for the South Indian Ocean. The results show that matrix neural networks have the ability to preserve spatial correlation that empower them to make a better prediction when compared to prominent recurrent neural network architectures.

Traditional synthesis/analysis sparse representation models signals in a one dimensional (1D) way, in which a multidimensional (MD) signal is converted into a 1D vector. 1D modeling cannot sufficiently handle MD signals of high dimensionality in limited computational resources and memory usage, as breaking the data structure and inherently ignores the diversity of MD signals (tensors). We utilize the multilinearity of tensors to establish the redundant basis of the space of multi linear maps with the sparsity constraint, and further propose MD synthesis/analysis sparse models to effectively and efficiently represent MD signals in their original form. The dimensional features of MD signals are captured by a series of dictionaries simultaneously and collaboratively. The corresponding dictionary learning algorithms and unified MD signal restoration formulations are proposed. The effectiveness of the proposed models and dictionary learning algorithms is demonstrated through experiments on MD signals denoising, image super-resolution and texture classification. Experiments show that the proposed MD models outperform state-of-the-art 1D models in terms of signal representation quality, computational overhead, and memory storage. Moreover, our proposed MD sparse models generalize the 1D sparse models and are flexible and adaptive to both homogeneous and inhomogeneous properties of MD signals.

Subspace data representation has recently become a common practice in many computer vision tasks. Low-rank representation (LRR) is one of the most successful models for clustering vectorial data according to their subspace structures. This paper explores the possibility of extending LRR for subspace data on Grassmann manifold. Rather than directly embedding the Grassmann manifold into the symmetric matrix space, an extrinsic view is taken to build the self-representation in the local area of the tangent space at each Grassmannian point, resulting in a localized LRR method on Grassmann manifold. A novel algorithm for solving the proposed model is investigated and implemented. The performance of the new clustering algorithm is assessed through experiments on several real-world data sets including MNIST handwritten digits, ballet video clips, SKIG action clips, and DynTex++ data set and highway traffic video clips. The experimental results show that the new method outperforms a number of state-of-the-art clustering methods.

By using a sparse representation or low-rank representation of data, the graph-based subspace clustering has recently attracted considerable attention in computer vision, given its capability and efficiency in clustering data. However, the graph weights built using the representation coefficients are not the exact ones as the traditional definition is in a deterministic way. The two steps of representation and clustering are conducted in an independent manner, thus an overall optimal result cannot be guaranteed. Furthermore, it is unclear how the clustering performance will be affected by using this graph. For example, the graph parameters, i.e., the weights on edges, have to be artificially pre-specified while it is very difficult to choose the optimum. To this end, in this paper, a novel subspace clustering via learning an adaptive low-rank graph affinity matrix is proposed, where the affinity matrix and the representation coefficients are learned in a unified framework. As such, the pre-computed graph regularizer is effectively obviated and better performance can be achieved. Experimental results on several famous databases demonstrate that the proposed method performs better against the state-of-the-art approaches, in clustering.

Depth estimation from the monocular RGB image is a challenging task for computer vision due to no reliable cues as the prior knowledge. Most existing monocular depth estimation works including various geometric or network learning methods lack of an effective mechanism to preserve the cross-border details of depth maps, which yet is very important for the performance promotion. In this paper, we propose a novel end-to-end progressive hard-mining network (PHN) framework to address this problem. Specifically, we construct the hard-mining objective function, the intra-scale and inter-scale refinement subnetworks to accurately localize and refine those hard-mining regions. The intra-scale refining block recursively recovers details of depth maps from different semantic features in the same receptive field while the inter-scale block favors a complementary interaction among multi-scale depth cues of different receptive fields. For further reducing the uncertainty of the network, we design a difficulty-ware refinement loss function to guide the depth learning process, which can adaptively focus on mining these hard-regions where accumulated errors easily occur. All three modules collaborate together to progressively reduce the error propagation in the depth learning process, and then, boost the performance of monocular depth estimation to some extent. We conduct comprehensive evaluations on several public benchmark data sets (including NYU Depth V2, KITTI, and Make3D). The experiment results well demonstrate the superiority of our proposed PHN framework over other state of the arts for monocular depth estimation task.

Clustering is one of the fundamental topics in data mining and pattern recognition. As a prospective clustering method, the subspace clustering has made considerable progress in recent researches, e.g., sparse subspace clustering (SSC) and low rank representation (LRR). However, most existing subspace clustering algorithms are designed for vectorial data from linear spaces, thus not suitable for high-dimensional data with intrinsic non-linear manifold structure. For high-dimensional or manifold data, few research pays attention to clustering problems. The purpose of clustering on manifolds tends to cluster manifold-valued data into several groups according to the mainfold-based similarity metric. This article proposes an extended LRR model for manifold-valued Grassmann data that incorporates prior knowledge by minimizing partial sum of singular values instead of the nuclear norm, namely Partial Sum minimization of Singular Values Representation (GPSSVR). The new model not only enforces the global structure of data in low rank, but also retains important information by minimizing only smaller singular values. To further maintain the local structures among Grassmann points, we also integrate the Laplacian penalty with GPSSVR. The proposed model and algorithms are assessed on a public human face dataset, some widely used human action video datasets and a real scenery dataset. The experimental results show that the proposed methods obviously outperform other state-of-the-art methods.

Multiview data clustering attracts more attention than their single-view counterparts due to the fact that leveraging multiple independent and complementary information from multiview feature spaces outperforms the single one. Multiview spectral clustering aims at yielding the data partition agreement over their local manifold structures by seeking eigenvalue-eigenvector decompositions. Among all the methods, low-rank representation (LRR) is effective, by exploring the multiview consensus structures beyond the low rankness to boost the clustering performance. However, as we observed, such classical paradigm still suffers from the following stand-out limitations for multiview spectral clustering of overlooking the flexible local manifold structure, caused by aggressively enforcing the low-rank data correlation agreement among all views, and such a strategy, therefore, cannot achieve the satisfied between-views agreement; worse still, LRR is not intuitively flexible to capture the latent data clustering structures. In this paper, first, we present the structured LRR by factorizing into the latent low-dimensional data-cluster representations, which characterize the data clustering structure for each view. Upon such representation, second, the Laplacian regularizer is imposed to be capable of preserving the flexible local manifold structure for each view. Third, we present an iterative multiview agreement strategy by minimizing the divergence objective among all factorized latent data-cluster representations during each iteration of optimization process, where such latent representation from each view serves to regulate those from other views, and such an intuitive process iteratively coordinates all views to be agreeable. Fourth, we remark that such data-cluster representation can flexibly encode the data clustering structure from any view with an adaptive input cluster number. To this end, finally, a novel nonconvex objective function is proposed via the efficient alternating minimization strategy. The complexity analysis is also presented. The extensive experiments conducted against the real-world multiview data sets demonstrate the superiority over the state of the arts.

It is time consuming to train support vector regression (SVR) for large-scale problems even with efficient quadratic programming solvers. This issue is particularly serious when tuning the model's parameters. One way to address the issue is to reduce the problem's scale by selecting a subset of the training set. This paper presents a fast pattern selection method by scanning the training data set to reduce a problem's scale. In particular, we find the k-nearest neighbors (kNNs) in a local region around each pattern's target value, and then determine to retain the pattern according to the distribution of its nearest neighbors. There is a high probability that the pattern locates outside the -tube. Since the kNNs of a pattern are found in a very small region, it is fast to scan the whole training data set. The proposed method deals with the year prediction Million Song Data set, which contains 463 715 patterns, within 10 s on a personal computer with an Intel Core i5-4690 CPU at 3.50 GHz and 8GB DRAM. An additional advantage of the proposed method is that it can predefine the size of the selected subset according to the training set. Comprehensive empirical evaluations demonstrate that the proposed method can significantly eliminate redundant patterns for SVR training with only a slight decrease in performance.

Spectral Clustering is one of pioneered clustering methods in machine learning and pattern recognition field. It relies on the spectral decomposition criterion to learn a low-dimensional embedding of data for a basic clustering algorithm such as the k-means. The recent sparse Spectral clustering (SSC) introduces the sparsity for the similarity in low-dimensional space by enforcing a sparsity-induced penalty, resulting a non-convex optimization, and the solution is calculated through a relaxed convex problem via the standard ADMM (Alternative Direction Method of Multipliers), rather than inferring latent representation from eigen-structure. This paper provides a direct solution as solving a new Grassmann optimization problem. By this way calculating latent embedding becomes part of optimization on manifolds and the recently developed manifold optimization methods can be applied. It turns out the learned new features are not only very informative for clustering, but also more intuitive and effective in visualization after dimensionality reduction. We conduct empirical studies on simulated datasets and several real-world benchmark datasets to validate the proposed methods. Experimental results exhibit the effectiveness of this new manifold-based clustering and dimensionality reduction method.

Learning robust regression model from high-dimensional corrupted data is an essential and difficult problem in many practical applications. The state-of-the-art methods have studied low-rank regression models that are robust against typical noises (like Gaussian noise and out-sample sparse noise) or outliers, such that a regression model can be learned from clean data lying on underlying subspaces. However, few of the existing low-rank regression methods can handle the outliers/noise lying on the sparsely corrupted disjoint subspaces. To address this issue, we propose a low-rank-sparse subspace representation for robust regression, hereafter referred to as LRS-RR in this paper. The main contribution include the following: (1) Unlike most of the existing regression methods, we propose an approach with two phases of low-rank-sparse subspace recovery and regression optimization being carried out simultaneously,(2) we also apply the linearized alternating direction method with adaptive penalty to solved the formulated LRS-RR problem and prove the convergence of the algorithm and analyze its complexity, (3) we demonstrate the efficiency of our method for the high-dimensional corrupted data on both synthetic data and two benchmark datasets against several state-of-the-art robust methods.

The eye disease such as Diabetic Retinopathy(DR) can be analysed through segmentation of retinal blood vessels. In the last five years, many methods for retinal blood vessels segmentation were proposed. These methods give arise to the improved accuracy, however the sensitivity of low contrast vessels is often ignored. The performance of diagnosis in terms of segmentation of vessels can be degraded due to missing tiny vessels. In this study, we propose a novel algorithm aiming at improving the performance of segmenting small vessels. The proposed approach adopts a morphological and filtering method to handle the background noise and uneven illumination and uses anisotropic diffusion filtering to coherent the vessels and give initial detection of vessels, followed by a double threshold based region growing method.

Learning on Grassmann manifold has become popular in many computer vision tasks, with the strong capability to extract discriminative information for imagesets and videos. However, such learning algorithms particularly on high-dimensional Grassmann manifold always involve with significantly high computational cost, which seriously limits the applicability of learning on Grassmann manifold in more wide areas. In this research, we propose an unsupervised dimensionality reduction algorithm on Grassmann manifold based on the Locality Preserving Projections (LPP) criterion. LPP is a commonly used dimensionality reduction algorithm for vector-valued data, aiming to preserve local structure of data in the dimension-reduced space. The strategy is to construct a mapping from higher dimensional Grassmann manifold into the one in a relative low-dimensional with more discriminative capability. The proposed method can be optimized as a basic eigenvalue problem. The performance of our proposed method is assessed on several classification and clustering tasks and the experimental results show show its clear advantages over other Grassmann based algorithms.

Restricted Boltzmann Machine (RBM) is a particular type of random neural network models modeling vector data based on the assumption of Bernoulli distribution. For multidimensional and non-binary data, it is necessary to vectorize and discretize the information in order to apply the conventional RBM. It is well-known that vectorization would destroy internal structure of data, and the binary units will limit the applying performance due to fickle real data. To address these issues, this paper proposes a Matrix variate Gaussian Restricted Boltzmann Machine (MVGRBM) model for matrix data whose entries follow Gaussian distributions. Compared with some other RBM algorithms, MVGRBM can model real value data better and it has good performance in image classification. To prove that adding Gaussian parameters could model input data well, we compared the reconstruction performance of the Gaussian parameters updating and fixed.

Summary Human detection and localization has attracted much attention in security applications because of the increasing demand of safety and security in different environments, including surveillance systems, secure access control, person recognition, border monitoring, preventing criminal acts, intrusion detection, alarm monitoring, and so on. This article proposes a robust approach for human detection and localization by analyzing and matching corresponding facial features extracted from video sequences. The proposed technique captures the video scenes using a stereo system consisting of two cameras: left and right cameras with similar intrinsic parameters. The system first tracks the human by detecting the face area from the video scenes using an efficient fuzzy face detection algorithm. To localize the human position, the depth information is computed from the extracted face images by using a robust stereo matching algorithm. A neural network is used to match the correspondence pixels between the left and the right face images. Experimental evaluation demonstrates the competence and robustness of the proposed method. The low computation time required for the detection and localization of human objects compared with other methods raises its suitability toward its use in real‐time applications. Copyright © 2016 John Wiley & Sons, Ltd.

Discriminative features are significant for hyper-spectral image (HSI) classification. In this letter, we apply the supervised dimensionality reduction (DR) model termed supervised latent linear Gaussian process latent variable model (SLLGPLVM) for feature extraction. As a semiparametric classification model, the new model has ability in simultaneous feature extraction and classification and demonstrates high classification accuracy with only a small training set. This is therefore suitable for HSI classification. Experimental results on six real HSI data sets show that the proposed SLLGPLVM outperforms several conventional supervised DR models and the support vector machine implemented in the original spectral space.

In multicamera video surveillance, it is challenging to represent videos from different cameras properly and fuse them efficiently for specific applications such as human activity recognition and clustering. In this paper, a novel representation for multicamera video data, namely, the product Grassmann manifold (PGM), is proposed to model video sequences as points on the Grassmann manifold and integrate them as a whole in the product manifold form. In addition, with a new geometry metric on the product manifold, the conventional low rank representation (LRR) model is extended onto PGM and the new LRR model can be used for clustering nonlinear data, such as multicamera video data. To evaluate the proposed method, a number of clustering experiments are conducted on several multicamera video data sets of human activity, including the Dongzhimen Transport Hub Crowd action data set, the ACT 42 Human Action data set, and the SKIG action data set. The experiment results show that the proposed method outperforms many state-of-the-art clustering methods.

In this paper, a new framework of mixed sparse representations (MSRs) is proposed for solving ill-conditioned problems with remote sensing images. In general, it is very difficult to find a common sparse representation for remote sensing images because of complicated ground features. Here we regard a remote sensing image as a combination of subimage of smooth, edges, and point-like components, respectively. Since each domain transformation method is capable of representing only a particular kind of ground object or texture, a group of domain transformations are used to sparsely represent each subimage. To demonstrate the effect of the framework of MSR for remote sensing images, MSR is regarded as a prior for maximum a posteriori when solving ill-conditioned problems such as classification and super resolution (SR), respectively. The experimental results show that not only the new framework of MSR can improve classification accuracy but also it can construct a much better high-resolution image than other common SR methods. The proposed framework MSR is a competitive candidate for solving other remote sensing images-related ill-conditioned problems.

This paper presents a novel Log-Euclidean metric inspired color-to-gray conversion model for faithfully preserving the contrast details of color image, which differs from the traditional Euclidean metric approaches. In the proposed model, motivated by the fact that Log-Euclidean metric has promising invariance properties such as inversion invariant and similarity invariant, we present a Log-Euclidean metric-based maximum function to model the decolorization procedure. The Gaussian-like penalty function consisting of the Log-Euclidean metric between gradients of the input color image and transformed grayscale image is incorporated to better reflect the degree of preserving feature discriminability and color ordering in color-to-gray conversion. A discrete searching algorithm is employed to solve the proposed model with linear parametric and non-negative constraints. Extensive evaluation experiments show that the proposed method outperforms the state-of-the-art methods both quantitatively and qualitatively.

It is a challenging problem to cluster multi- and high-dimensional data with complex intrinsic properties and non-linear manifold structure. The recently proposed subspace clustering method, Low Rank Representation (LRR), shows attractive performance on data clustering, but it generally does with data in Euclidean spaces. In this paper, we intend to cluster complex high dimensional data with multiple varying factors. We propose a novel representation, namely Product Grassmann Manifold (PGM), to represent these data. Additionally, we discuss the geometry metric of the manifold and expand the conventional LRR model in Euclidean space onto PGM and thus construct a new LRR model. Several clustering experimental results show that the proposed method obtains superior accuracy compared with the clustering methods on manifolds or conventional Euclidean spaces.

The probabilistic principal component analysis (PPCA) is built upon a global linear mapping, with which it is insufficient to model complex data variation. This paper proposes a mixture of bilateral-projection probabilistic principal component analysis model (mixB2DPPCA) on 2D data. With multi-components in the mixture, this model can be seen as a 'soft' cluster algorithm and has capability of modeling data with complex structures. A Bayesian inference scheme has been proposed based on the variational EM (Expectation-Maximization) approach for learning model parameters. Experiments on some publicly available databases show that the performance of mixB2DPPCA has been largely improved, resulting in more accurate reconstruction errors and recognition rates than the existing PCA-based algorithms.

Trace-norm regularization plays an important role in many areas such as computer vision and machine learning. When solving general large-scale trace-norm regularized problems, existing methods may be computationally expensive due to many high-dimensional truncated singular value decompositions (SVDs) or the unawareness of matrix ranks. In this paper, we propose a proximal Riemannian pursuit (PRP) paradigm which addresses a sequence of trace-norm regularized subproblems defined on nonlinear matrix varieties. To address the subproblem, we extend the proximal gradient method on vector space to nonlinear matrix varieties, in which the SVDs of intermediate solutions are maintained by cheap low-rank QR decompositions, therefore making the proposed method more scalable. Empirical studies on several tasks, such as matrix completion and low-rank representation based subspace clustering, demonstrate the competitive performance of the proposed paradigms over existing methods.

Sparse subspace clustering (SSC), as one of the most successful subspace clustering methods, has achieved notable clustering accuracy in computer vision tasks. However, SSC applies only 10 vector data in Euclidean space. Unfortunately there is still no satisfactory approach to solve subspace clustering by self-expressive principle f or symmetric positive definite (SPD) matrices which is very useful, in computer vision. In this paper, by embedding the SPD matrices into a Reproducing Kernel Hilbert Space (RKHS), a kernel subspace clustering method is constructed un the SPD manifold through an appropriate Log-Euclidean kernel, termed as kernel sparse subspace clustering on the SPD Riemannian manifold(KSSCR). By exploiting the intrinsic Riemannian geometry within data, KSSCR can effectively characterize the geodesic distance between SPD matrices to uncover the underlying subspace structure. Experimental results Oft several famous datasets demonstrate that the proposed method achieves better clustering results than the state-of-the-art approaches.

Image filtering is a key technology in image processing applications for de-noising corrupted images. Digital images are often polluted by noise during capturing and hence they may not show the features or colors clearly. Image filtering removes the noise in an image and improves the contrast to provide better input for various image processing applications. This paper proposes an efficient image filtering technique using fuzzy logic. The proposed method employs fuzzy membership functions in order to replace the noisy pixels based on the degree of membership of the neighboring pixels within a filter mask. Experimental results confirm that our method is very effective and fast for removing impulsive noise while preserving the small and sharp details in the image.

Matrix manifolds such as Stiefel and Grassmann manifolds have been widely used in modern computer vision. This paper is concerned with the problem of classifying such manifold-valued data, based on the maximum likelihood estimation for the parametric probability density functions defined on the manifolds. By using a new way of computing normalisation constants for the matrix Langevin distribution function, defined for the data on Stiefel manifold, and the Fisher-Bingham density function on Grassmann manifold, we proposed a simple way to estimate the parameters in the distribution and demonstrated on real world datasets that the proposed method is efficient and more accurate. Our method uses asymptotic and Taylor series approximations, with good accuracy estimate.

A novel estimation algorithm is introduced for logistic regression model with nonnegative coefficients constraints. The basic idea is to decompose the model parameters as a vector of convex coefficients (the multinomial manifold) and a scaling parameter, which are then optimized alternatively based on the maximum likelihood cost function. The first and second order Riemannian geometry of the multinomial manifold are utilized in the Riemannian trust-region algorithm. The scaling parameter is solved using the golden section algorithm. Numerical examples are employed to demonstrate that effectiveness of the proposed approach.

Unsupervised learning aims to discovery latent representation embedded in the observation, which is useful for data visualization, dimensionality reduction, and density modeling. Autoencoders have been successfully used to learn the latent variations in data, especially with the recent reintroduction by deep learning. For some specific tasks, there are supervised information or labels that can be used to further guide the unsupervised autoencoder model for finding latent representation. The Non-Parametrically Guided Autoencoder (NPGA) has been proved to be an effective model. It tries to utilize Gaussian Process Regression (GPR) to model the unknown mapping from unknown latent representation to extra supervised information. However for the discrete label information in classification tasks, using GPR could be unwise and inefficient. In this paper, we propose the Non-Parametrically Guided Autoencoder with Laplace Approximation (NPGA-LA) to effectively handle discrete labels. The idea of NPGA-LA is to make use of Gaussian Process Classification (GPC) rather than GPR to model the transformation between the latent space and the discrete label space. The experimental results verify the excellent performance of the newly developed method.

Restricted Boltzmann Machine (RBM) is an important generative model modeling vectorial data. While applying an RBM in practice to images, the data have to be vectorized. This results in high-dimensional data and valuable spatial information has got lost in vectorization. In this paper, a Matrix-Variate Restricted Boltzmann Machine (MVRBM) model is proposed by generalizing the classic RBM to explicitly model matrix data. In the new RBM model, both input and hidden variables are in matrix forms which are connected by bilinear transforms. The MVRBM has much less model parameters while retaining comparable performance as the classic RBM. The advantages of the MVRBM have been demonstrated on three real-world applications: handwritten digit denoising, reconstruction and recognition.

Image processing is an active research area in which medical image processing is a highly challenging field. Medical imaging techniques are used to image the inner portion of human body for medical diagnosis. In this research work, retinal colour fundus images and MRI brain images noise level has been improved. Fundus Fluorescein Angiography (FFA) is the invasive based technique used to give high contrast retinal images but it used contrast injection and other side Magnetic Resonance Imaging (MRI) is a medical used to produce the high contrast image. The biomedical images are mostly suffered from the varied contrast and due to varied contrast, the details of images are not observed properly even after the image enhancement techniques because the presence of noise. In this research, The High-Resolution Fundus (HRF) database is used and it contained 36 images of two pairs (18 good quality images and 18 bad quality images). Oasis MRI brain image database is also used and it contained 30 images. Radial Basis Function (RBF) neural network gave highest PSNR improvement of 53% and 56% in HRF retinal images database and Oasis MRI Brain images database as compared to wavelet technique (18%,35%) and sub space method( 29%,9%). The optimal denoised method is one important step to get better result of contrast normalisation techniques and give accurate results to diagnose the disease progress.

Tensor clustering is an important tool that exploits intrinsically rich structures in real-world multiarray or Tensor datasets. Often in dealing with those datasets, standard practice is to use subspace clustering that is based on vectorizing multiarray data. However, vectorization of tensorial data does not exploit complete structure information. In this paper, we propose a subspace clustering algorithm without adopting any vectorization process. Our approach is based on a novel heterogeneous Tucker decomposition model taking into account cluster membership information. We propose a new clustering algorithm that alternates between different modes of the proposed heterogeneous tensor model. All but the last mode have closed-form updates. Updating the last mode reduces to optimizing over the multinomial manifold for which we investigate second order Riemannian geometry and propose a trust-region algorithm. Numerical experiments show that our proposed algorithm compete effectively with state-of-the-art clustering algorithms that are based on tensor factorization.

Low-rank representation (LRR) has recently attracted a great deal of attention due to its pleasing efficacy in exploring low-dimensional subspace structures embedded in data. For a given set of observed data corrupted with sparse errors, LRR aims at learning a lowest-rank representation of all data jointly. LRR has broad applications in pattern recognition, computer vision and signal processing. In the real world, data often reside on low-dimensional manifolds embedded in a high-dimensional ambient space. However, the LRR method does not take into account the non-linear geometric structures within data, thus the locality and similarity information among data may be missing in the learning process. To improve LRR in this regard, we propose a general Laplacian regularized low-rank representation framework for data representation where a hypergraph Laplacian regularizer can be readily introduced into, i.e., a Non-negative Sparse Hyper-Laplacian regularized LRR model (NSHLRR). By taking advantage of the graph regularizer, our proposed method not only can represent the global low-dimensional structures, but also capture the intrinsic non-linear geometric information in data. The extensive experimental results on image clustering, semi-supervised image classification and dimensionality reduction tasks demonstrate the effectiveness of the proposed method.

In image restoration problems, it is reasonable to add nonnegative constraints because of the physical meaning of images. In general, this problem can be expressed as a quadratic programming problem with nonnegative constraints, which results in a linear complementary problem from the KKT optimization conditions. By reformulating the linear complementary problem as implicit fixed-point equations, a class of modulus-based matrix splitting iteration methods is established. In this paper, for a better computational implementation, we present an inexact iteration process for these modulus-based methods. Convergence properties for this inexact process are analyzed, and some specific implementations for the inner iterations are presented. Numerical experiments for nonnegatively constrained image restorations are presented, and the results show that our methods are comparable and more efficient than the existing projection type methods.

A proper strategy to alleviate overfitting is critical to a deep neural network (DNN). In this paper, we introduce the cross-loss-function regularization for boosting the generalization capability of the DNN, which results in the multi-loss regularized DNN (ML-DNN) framework. For a particular learning task, e.g., image classification, only a single-loss function is used for all previous DNNs, and the intuition behind the multiloss framework is that the extra loss functions with different theoretical motivations (e.g., pairwise loss and LambdaRank loss) may drag the algorithm away from overfitting to one particular single-loss function (e.g., softmax loss). In the training stage, we pretrain the model with the single-core-loss function and then warm start the whole ML-DNN with the convolutional parameters transferred from the pretrained model. In the testing stage, the outputs by the ML-DNN from different loss functions are fused with average pooling to produce the ultimate prediction. The experiments conducted on several benchmark datasets (CIFAR-10, CIFAR-100, MNIST, and SVHN) demonstrate that the proposed ML-DNN framework, instantiated by the recently proposed network in network, considerably outperforms all other state-of-the-art methods.

Many application scenarios involve sequential data, but most existing clustering methods do not well utilize the order information embedded in sequential data. In this paper, we study the subspace clustering problem for sequential data and propose a new clustering method, namely ordered sparse clustering with block-diagonal prior (BD-OSC). Instead of using the sparse normalizer in existing sparse subspace clustering methods, a quadratic normalizer for the data sparse representation is adopted to model the correlation among the data sparse coefficients. Additionally, a block-diagonal prior for the spectral clustering affinity matrix is integrated with the model to improve clustering accuracy. To solve the proposed BD-OSC model, which is a complex optimization problem with quadratic normalizer and block-diagonal prior constraint, an efficient algorithm is proposed. We test the proposed clustering method on several types of databases, such as synthetic subspace data set, human face database, video scene clips, motion tracks, and dynamic 3-D face expression sequences. The experiments show that the proposed method outperforms state-of-the-art subspace clustering methods.

Subspace clustering groups a set of samples from a union of several linear subspaces into clusters, so that the samples in the same cluster are drawn from the same linear subspace. In the majority of the existing work on subspace clustering, clusters are built based on feature information, while sample correlations in their original spatial structure are simply ignored. Besides, original high-dimensional feature vector contains noisy/redundant information, and the time complexity grows exponentially with the number of dimensions. To address these issues, we propose a tensor low-rank representation (TLRR) and sparse coding-based (TLRRSC) subspace clustering method by simultaneously considering feature information and spatial structures. TLRR seeks the lowest rank representation over original spatial structures along all spatial directions. Sparse coding learns a dictionary along feature spaces, so that each sample can be represented by a few atoms of the learned dictionary. The affinity matrix used for spectral clustering is built from the joint similarities in both spatial and feature spaces. TLRRSC can well capture the global structure and inherent feature information of data and provide a robust subspace segmentation from corrupted data. Experimental results on both synthetic and real-world data sets show that TLRRSC outperforms several established stateof- the-art methods.

Exploiting label dependency for multi-label image classification can significantly improve classification performance. Probabilistic Graphical Models are one of the primary methods for representing such dependencies. The structure of graphical models, however, is either determined heuristically or learned from very limited information. Moreover, neither of these approaches scales well to large or complex graphs. We propose a principled way to learn the structure of a graphical model by considering input features and labels, together with loss functions. We formulate this problem into a max-margin framework initially, and then transform it into a convex programming problem. Finally, we propose a highly scalable procedure that activates a set of cliques iteratively. Our approach exhibits both strong theoretical properties and a significant performance improvement over state-of-the-art methods on both synthetic and real-world data sets.

Sequential data are ubiquitous in data analysis. For example hyperspectral data taken from a drill hole in geology, high throughput X-ray diffraction measurements in materials research and EEG brain wave signals in neuroscience. The common feature of sequential data is that they are all acquired subject to one external variable such as location, time or temperature. The data evolve along the direction of that variable through several patterns and the “neighboring” data are very likely to share similar features. The purpose of the segmentation for sequential data is then to identify those sequentially continuous segments/patterns. We approach this problem by adopting the subspace clustering method and propose a novel algorithm called low rank sequential subspace clustering (LRSSC), inspired by another method called spatial subspace clustering (SpatSC). SpatSC finds the subspaces by data self-reconstruction with a sparsity constraint on reconstruction weights and promotes the spatial smoothness of the weights by fusion, the essential part in the fused LASSO. However, the subspace identification capability is limited due to the indeterminacy of the sparse regression in finding suitable samples to linearly reconstruct a given sample. This confuses the graph cut algorithm that produces the final clustering results on the weights. To overcome this drawback, we propose to use the low rank penalty instead of sparsity in learning phase to separate subspaces. This improves the subspace identification as well as the robustness to noise. To demonstrate its effectiveness, we test LRSSC on both simulated and real world data compared with SpatSC and other methods. The proposed algorithm is superior to others when noise level is very high.

A new sparse kernel density estimator is introduced based on the minimum integrated square error criterion combining local component analysis for the finite mixture model. We start with a Parzen window estimator which has the Gaussian kernels with a common covariance matrix, the local component analysis is initially applied to find the covariance matrix using expectation maximization algorithm. Since the constraint on the mixing coefficients of a finite mixture model is on the multinomial manifold, we then use the well-known Riemannian trust-region algorithm to find the set of sparse mixing coefficients. The first and second order Riemannian geometry of the multinomial manifold are utilized in the Riemannian trust-region algorithm. Numerical examples are employed to demonstrate that the proposed approach is effective in constructing sparse kernel density estimators with competitive accuracy to existing kernel density estimators.

Sparse coding aims to find a more compact representation based on a set of dictionary atoms. A well-known technique looking at 2D sparsity is the low rank representation (LRR). However, in many computer vision applications, data often originate from a manifold, which is equipped with some Riemannian geometry. In this case, the existing LRR becomes inappropriate for modeling and incorporating the intrinsic geometry of the manifold that is potentially important and critical to applications. In this paper, we generalize the LRR over the Euclidean space to the LRR model over a specific Rimannian manifold—the manifold of symmetric positive matrices (SPD). Experiments on several computer vision datasets showcase its noise robustness and superior performance on classification and segmentation compared with state-of-the-art approaches.

Recovering intrinsic low-dimensional subspaces from data distributed on them is a key preprocessing step to many applications. In recent years, a lot of work has modeled subspace recovery as low-rank minimization problems. We find that some representative models, such as robust principal component analysis (R-PCA), robust low-rank representation (R-LRR), and robust latent low-rank representation (R-LatLRR), are actually deeply connected. More specifically, we discover that once a solution to one of the models is obtained, we can obtain the solutions to other models in closed-form formulations. Since R-PCA is the simplest, our discovery makes it the center of low-rank subspace recovery models. Our work has two important implications. First, R-PCA has a solid theoretical foundation. Under certain conditions, we could find globally optimal solutions to these low-rank models at an overwhelming probability, although these models are nonconvex. Second, we can obtain significantly faster algorithms for these models by solving R-PCA first. The computation cost can be further cut by applying low-complexity randomized algorithms, for example, our novel l2,1 filtering algorithm, to R-PCA. Although for the moment the formal proof of our l2,1 filtering algorithm is not yet available, experiments verify the advantages of our algorithm over other state-of-the-art methods based on the alternating direction method.

To efficiently deal with the complex nonlinear variations of face images, a novel Lie group (LG) kernel is proposed in this paper to address the facial analysis problems. First, we present a linear dynamic model (LDM)-based face representation to capture both the appearance and spatial information of the face image. Second, the derived LDM can be parameterized as a specially structured upper triangular matrix, the space of which is proved to constitute an LG. An LG kernel is then designed to characterize the similarity between the LDMs for any two face images and the kernel can be fed into classical kernel-based classifiers for different types of facial analysis. Finally, experimental evaluations on face recognition and head pose estimation are conducted on several challenging data sets and the results show that the proposed algorithm outperforms other facial analysis methods.

In this paper, we study discriminative analysis of symmetric positive definite (SPD) matrices on Lie groups (LGs), namely, transforming an LG into a dimension-reduced one by optimizing data separability. In particular, we take the space of SPD matrices, e.g., covariance matrices, as a concrete example of LGs, which has proved to be a powerful tool for high-order image feature representation. The discriminative transformation of an LG is achieved by optimizing the within-class compactness as well as the between-class separability based on the popular graph embedding framework. A new kernel based on the geodesic distance between two samples in the dimension-reduced LG is then defined and fed into classical kernel-based classifiers, e.g., support vector machine, for various visual classification tasks. Extensive experiments on five public datasets, i.e., Scene-15, Caltech101, UIUC-Sport, MIT-Indoor, and VOC07, well demonstrate the effectiveness of discriminative analysis for SPD matrices on LGs, and the state-of-the-art performances are reported.

An efficient data based-modeling algorithm for nonlinear system identification is introduced for radial basis function (RBF) neural networks with the aim of maximizing generalization capability based on the concept of leave-one-out (LOO) cross validation. Each of the RBF kernels has its own kernel width parameter and the basic idea is to optimize the multiple pairs of regularization parameters and kernel widths, each of which is associated with a kernel, one at a time within the orthogonal forward regression (OFR) procedure. Thus, each OFR step consists of one model term selection based on the LOO mean square error (LOOMSE), followed by the optimization of the associated kernel width and regularization parameter, also based on the LOOMSE. Since like our previous state-of-the-art local regularization assisted orthogonal least squares (LROLS) algorithm, the same LOOMSE is adopted for model selection, our proposed new OFR algorithm is also capable of producing a very sparse RBF model with excellent generalization performance. Unlike our previous LROLS algorithm which requires an additional iterative loop to optimize the regularization parameters as well as an additional procedure to optimize the kernel width, the proposed new OFR algorithm optimizes both the kernel widths and regularization parameters within the single OFR procedure, and consequently the required computational complexity is dramatically reduced. Nonlinear system identification examples are included to demonstrate the effectiveness of this new approach in comparison to the well-known approaches of support vector machine and least absolute shrinkage and selection operator as well as the LROLS algorithm.

This paper introduces an L1-norm-based probabilistic principal component analysis model on 2D data (L1-2DPPCA) based on the assumption of the Laplacian noise model. The Laplacian or L1 density function can be expressed as a superposition of an infinite number of Gaussian distributions. Under this expression, a Bayesian inference can be established based on the variational expectation maximization approach. All the key parameters in the probabilistic model can be learned by the proposed variational algorithm. It has experimentally been demonstrated that the newly introduced hidden variables in the superposition can serve as an effective indicator for data outliers. Experiments on some publicly available databases show that the performance of L1-2DPPCA has largely been improved after identifying and removing sample outliers, resulting in more accurate image reconstruction than the existing PCA-based methods. The performance of feature extraction of the proposed method generally outperforms other existing algorithms in terms of reconstruction errors and classification accuracy.

Low-rank representation (LRR) has received considerable attention in subspace segmentation due to its effectiveness in exploring low-dimensional subspace structures embedded in data. To preserve the intrinsic geometrical structure of data, a graph regularizer has been introduced into LRR framework for learning the locality and similarity information within data. However, it is often the case that not only the high-dimensional data reside on a non-linear low-dimensional manifold in the ambient space, but also their features lie on a manifold in feature space. In this paper, we propose a dual graph regularized LRR model (DGLRR) by enforcing preservation of geometric information in both the ambient space and the feature space. The proposed method aims for simultaneously considering the geometric structures of the data manifold and the feature manifold. Furthermore, we extend the DGLRR model to include non-negative constraint, leading to a parts-based representation of data. Experiments are conducted on several image data sets to demonstrate that the proposed method outperforms the state-of-the-art approaches in image clustering.

A new sparse kernel density estimator is introduced based on the minimum integrated square error criterion for the finite mixture model. Since the constraint on the mixing coefficients of the finite mixture model is on the multinomial manifold, we use the well-known Riemannian trust-region (RTR) algorithm for solving this problem. The first- and second-order Riemannian geometry of the multinomial manifold are derived and utilized in the RTR algorithm. Numerical examples are employed to demonstrate that the proposed approach is effective in constructing sparse kernel density estimators with an accuracy competitive with those of existing kernel density estimators.

In order to separate the chromatogram peaks and spectra from the High Performance Liquid Chromatography with Diode Array Detector (HPLC-DAD) data set, a separation model of Generalized Reference Curve Measurement and its solution by multitarget Bare Bones Particle Swarm Optimization (GRCMmBBPSO) is proposed in this paper. Firstly, parameters are constructed which will generate Reference Curves (RCs) for chromatogram peaks. Secondly, the GRCM model is proposed to transform all these parameters to scalar values, which indicate the possibility for the HPLC-DAD data set containing chromatogram peaks similar to the RCs constructed by corresponding parameters. Then, the algorithm of mBBPSO is introduced to calculate the optimal parameters by minimizing the scalar values. Finally, the spectra are estimated. Through simulations and experiments, following conclusions are drawn: (1) the GRCMmBBPSO method can extract chromatogram peaks from the simulation data set without knowing the number of the compounds in advance even when a severe overlap and white noise exist; (2) the GRCMmBBPSO method can be applied to real HPLC-DAD data set.

We propose Ordered Subspace Clustering (OSC) to segment data drawn from a sequentially ordered union of subspaces. Current subspace clustering techniques learn the relationships within a set of data and then use a separate clustering algorithm such as NCut for final segmentation. In contrast our technique, under certain conditions, is capable of segmenting clusters intrinsically without providing the number of clusters as a parameter. Similar to Sparse Subspace Clustering (SSC) we formulate the problem as one of finding a sparse representation but include a new penalty term to take care of sequential data. We test our method on data drawn from infrared hyper spectral data, video sequences and face images. Our experiments show that our method, OSC, outperforms the state of the art methods: Spatial Subspace Clustering (SpatSC), Low-Rank Representation (LRR) and SSC.

In this paper, a novel image blocky artifact removal scheme based on low-rank matrix recovery is proposed. The problem of suppressing blocky artifacts is formulated as recovering a low-rank matrix from corrupted observations. During the deblocking processing, we do not directly recover the whole clean image but only its high-frequency component and then synthesize the clean image by incorporating the low-frequency component of blocky image. To take advantage of the low-rank matrix recovery paradigm, we first cluster the similar patches of the high-frequency component of image via local pixel clustering, then the clean high-frequency component of image is recovered by formulating an optimization problem of the nuclear norm and ℓ1-norm. The experimental results show that the proposed algorithm can achieve competitive performance in terms of both quantitative and subjective quality.

Classical regression methods take vectors as covariates and estimate the corresponding vectors of regression parameters. When addressing regression problems on covariates of more complex form such as multi-dimensional arrays (i.e. Tensors), traditional computational models can be severely compromised by ultrahigh dimensionality as well as complex structure. By exploiting the special structure of tensor covariates, the tensor regression model provides a promising solution to reduce the model's dimensionality to a manageable level, thus leading to efficient estimation. Most of the existing tensor-based methods independently estimate each individual regression problem based on tensor decomposition which allows the simultaneous projections of an input tensor to more than one direction along each mode. As a matter of fact, multi-dimensional data are collected under the same or very similar conditions, so that data share some common latent components but can also have their own independent parameters for each regression task. Therefore, it is beneficial to analyse regression parameters among all the regressions in a linked way. In this paper, we propose a tensor regression model based on Tucker Decomposition, which identifies not only the common components of parameters across all the regression tasks, but also independent factors contributing to each particular regression task simultaneously. Under this paradigm, the number of independent parameters along each mode is constrained by a sparsity-preserving regulariser. Linked multiway parameter analysis and sparsity modeling further reduce the total number of parameters, with lower memory cost than their tensor-based counterparts. The effectiveness of the new method is demonstrated on real data sets.

Asset prices fluctuate up and down chaotically. Traders, investors and fund managers comb the chaos for exploitable patterns with methods such as moving averages from the realm of technical analysis. In this paper we focus on linear moving averages which aim to smooth asset prices removing fluctuations. First, we will develop a method to measure the smoothness for a linear filter. We will also discuss how to measure estimation lag. We will develop a new linear filter and show that, for short to medium time frames, it is smoother than competing filters.

Traditional dictionary learning algorithms are used for finding a sparse representation on high dimensional data by transforming samples into a one-dimensional (ID) vector. This ID model loses the inherent spatial structure property of data. An alternative solution is to employ Tensor Decomposition for dictionary learning on their original structural form - a tensor - by learning multiple dictionaries along each mode and the corresponding sparse representation in respect to the Kronecker product of these dictionaries. To learn tensor dictionaries along each mode, all the existing methods update each dictionary iteratively in an alternating manner. Because atoms from each mode dictionary jointly make contributions to the spar sity of tensor, existing works ignore atoms correlations between different mode dictionaries by treating each mode dictionary independently. In this paper, we propose a joint multiple dictionary learning method for tensor sparse coding, which explores atom correlations for sparse representation and updates multiple atoms from each mode dictionary simultaneously. In this algorithm, the Frequent-Pattern Tree (FP-tree) mining algorithm is employed to exploit frequent atom patterns in the sparse representation. Inspired by the idea of K-SVD, we develop a new dictionary update method that jointly updates elements in each pattern. Experimental results demonstrate our method outperforms other tensor based dictionary learning algorithms.

Subspace clustering groups a set of samples (vectors) into clusters by approximating this set with a mixture of several linear subspaces, so that the samples in the same cluster are drawn from the same linear subspace. In majority of existing works on subspace clustering, samples are simply regarded as being independent and identically distributed, that is, arbitrarily ordering samples when necessary. However, this setting ignores sample correlations in their original spatial structure. To address this issue, we propose a tensor low-rank representation (TLRR) for subspace clustering by keeping available spatial information of data. TLRR seeks a lowest-rank representation over all the candidates while maintaining the inherent spatial structures among samples, and the affinity matrix used for spectral clustering is built from the combination of similarities along all data spatial directions. TLRR better captures the global structures of data and provides a robust subspace segmentation from corrupted data. Experimental results on both synthetic and real-world datasets show that TLRR outperforms several established state-of-the-art methods.

We present a new fuzzy c-means algorithm for image segmentation by introducing a novel spatially constrained Student's t-distribution and a new regularization term. Firstly, considering that conventional distribution models lack spatial information and the multivariate Student's t-distribution is heavily tailed, we propose a new way to incorporate spatial information between neighboring pixels into the Student's t-distribution based on Markov random field (MRF) in order to enhance robustness. Secondly, the new regularization term, inspired by the geodesic active contour (GAC) with a strong ability in capturing boundary, can preserve the details of edges and further enhance its robustness to noise and outliers by capitalizing on the local context information and edge information. Finally, in comparison to other Markov random fields that are complex and computationally expensive, the parameters are easily optimized with the EM algorithm in our proposed method. The proposed algorithm demonstrates the robustness and effectiveness, compared with other state-of-the-art methods on synthetic and real images.

This paper is concerned with tensor clustering with the assistance of dimensionality reduction approaches. A class of formulation for tensor clustering is introduced based on tensor Tucker decomposition models. In this formulation, an extra tensor mode is formed by a collection of tensors of the same dimensions and then used to assist a Tucker decomposition in order to achieve data dimensionality reduction. We design two types of clustering models for the tensors: PCA Tensor Clustering model and Non-negative Tensor Clustering model, by utilizing different regularizations. The tensor clustering can thus be solved by the optimization method based on the alternative coordinate scheme. Interestingly, our experiments show that the proposed models yield comparable or even better performance compared to most recent clustering algorithms based on matrix factorization.

Linear Subspace Learning (LSL) has been widely used in many areas of information processing, such as dimensionality reduction, data mining, pattern recognition and computer vision. Recent years have witnessed several excellent extensions of PCA in LSL. One is the recent L1-norm maximization principal component analysis (L1Max-PCA), which aims at learning linear subspace efficiently. L1Max-PCA simply simulates PCA by replacing the covariance with the so-called L1-norm dispersion in the mapped feature space. However, it is difficult to give an intuitive interpretation. In this paper, a novel subspace learning approach based on sparse dimension reduction is proposed, which enforces the sparsity of the mapped data to better recover cluster structures. The optimization problem is solved efficiently via Alternating Direction Method (ADM). Experimental results show that the proposed method is effective in subspace learning.

BACKGROUND: The 3D chromatogram generated by High Performance Liquid Chromatography-Diode Array Detector (HPLC-DAD) has been researched widely in the field of herbal medicine, grape wine, agriculture, petroleum and so on. Currently, most of the methods used for separating a 3D chromatogram need to know the compounds' number in advance, which could be impossible especially when the compounds are complex or white noise exist. New method which extracts compounds from 3D chromatogram directly is needed. METHODS: In this paper, a new separation model named parallel Independent Component Analysis constrained by Reference Curve (pICARC) was proposed to transform the separation problem to a multi-parameter optimization issue. It was not necessary to know the number of compounds in the optimization. In order to find all the solutions, an algorithm named multi-areas Genetic Algorithm (mGA) was proposed, where multiple areas of candidate solutions were constructed according to the fitness and distances among the chromosomes. RESULTS: Simulations and experiments on a real life HPLC-DAD data set were used to demonstrate our method and its effectiveness. Through simulations, it can be seen that our method can separate 3D chromatogram to chromatogram peaks and spectra successfully even when they severely overlapped. It is also shown by the experiments that our method is effective to solve real HPLC-DAD data set. CONCLUSIONS: Our method can separate 3D chromatogram successfully without knowing the compounds' number in advance, which is fast and effective.

Dimensionality reduction (DR) has been considered as one of the most significant tools for data analysis. One type of DR algorithms is based on latent variable models (LVM). LVM-based models can handle the preimage problem easily. In this paper we propose a new LVM-based DR model, named thin plate spline latent variable model (TPSLVM). Compared to the well-known Gaussian process latent variable model (GPLVM), our proposed TPSLVM is more powerful especially when the dimensionality of the latent space is low. Also, TPSLVM is robust to shift and rotation. This paper investigates two extensions of TPSLVM, i.e., the back-constrained TPSLVM (BC-TPSLVM) and TPSLVM with dynamics (TPSLVM-DM) as well as their combination BC-TPSLVM-DM. Experimental results show that TPSLVM and its extensions provide better data visualization and more efficient dimensionality reduction compared to PCA, GPLVM, ISOMAP, etc.

Many communication signal processing applications involve modeling and inverting complex-valued (CV) Hammerstein systems. We develop a new CV B-spline neural network approach for efficient identification of the CV Hammerstein system and effective inversion of the estimated CV Hammerstein model. In particular, the CV nonlinear static function in the Hammerstein system is represented using the tensor product from two univariate B-spline neural networks. An efficient alternating least squares estimation method is adopted for identifying the CV linear dynamic model's coefficients and the CV B-spline neural network's weights, which yields the closed-form solutions for both the linear dynamic model's coefficients and the B-spline neural network's weights, and this estimation process is guaranteed to converge very fast to a unique minimum solution. Furthermore, an accurate inversion of the CV Hammerstein system can readily be obtained using the estimated model. In particular, the inversion of the CV nonlinear static function in the Hammerstein system can be calculated effectively using a Gaussian-Newton algorithm, which naturally incorporates the efficient De Boor algorithm with both the B-spline curve and first-order derivative recursions. The effectiveness of our approach is demonstrated using the application to equalization of Hammerstein channels.

This paper presents an ordered-patch-based image classification framework integrating the image Grassmannian manifold to address handwritten digit recognition, face recognition, and scene recognition problems. Typical image classification methods explore image appearances without considering the spatial causality among distinctive domains in an image. To address the issue, we introduce an ordered-patch-based image representation and use the autoregressive moving average (ARMA) model to characterize the representation. First, each image is encoded as a sequence of ordered patches, integrating both the local appearance information and spatial relationships of the image. Second, the sequence of these ordered patches is described by an ARMA model, which can be further identified as a point on the image Grassmannian manifold. Then, image classification can be conducted on such a manifold under this manifold representation. Furthermore, an appropriate Grassmannian kernel for support vector machine classification is developed based on a distance metric of the image Grassmannian manifold. Finally, the experiments are conducted on several image data sets to demonstrate that the proposed algorithm outperforms other existing image classification methods.

In order to separate a 3D chromatography, which is generated from High Performance Liquid Chromatography-Diode Array Detector (HPLC-DAD), into chromatograms and spectra, we proposed a model called parallel Independent Component Analysis constrained by Reference Curve (pICARC), which transforms the separation problem to a multi-parameter optimization issue. Then, A new algorithm named multi-areas Genetic Algorithm (mGA) is developed to search multiple solutions in parallel. It was demonstrated that our approach could successfully separate a given HPLC-DAD dataset into chromatograms and spectra with little errors and in short computational time.

Image super-resolution means forming high-resolution images from low-resolution images. In this paper, we develop a new approach based on the deep Restricted Boltzmann Machines (RBM) for image super-resolution. The RBM architecture has ability of learning a set of visual patterns, called dictionary elements from a set of training images. The learned dictionary will be then used to synthesize high resolution images. We test the proposed algorithm on both benchmark and natural images, comparing with several other techniques. The visual quality of the results has also been assessed by both human evaluation and quantitative measurement.

Feature extraction is one of the most fundamental problems in face recognition tasks. In this paper, motivated by low-rank representation (LRR) model on exploring the multiple subspace structures of observation data, we propose a double low-rank matrix recovery method to learn low-rank subspaces from face images, where it takes into account the recovery of row space and column space information simultaneously. Applying Augmented Lagrangian Multiplier (ALM), the optimization problem on minimization of nuclear norm is resolved efficiently. By evaluating on public face databases, experimental results show that our proposed method works much better than existing face recognition methods based on feature extraction. It is more robust to outliers, varying illumination and occlusion.

A novel method called spatial subspace clustering (SpatSC) for 1D hyperspectral data segmentation problem, e.g. hyperspectral data taken from a drill hole, exploring spatial information has been proposed in [1]. The purpose of this exercise is to improve interpretability of the hyperspectral data. The spatial subspace clustering has two major components in its formulation, i.e. data self reconstruction and fused lasso. The first component is mainly to separate different subspaces where data lie on or close to, while the second is to exploit the spatial smoothness based on the observation of stratification of rocks. It produces interpretable and consistent clusters by utilizing the spatial information. However, the implementation of SpatSC requires an optimization of N2variables, where N is the number of samples in the data set. When N is large, for example, tens of thousands for a typical drill hole data set, the algorithm is no longer suitable for personal computers. To alleviate the computational intensity, we propose to run SpatSC on a randomly chosen calibration set from crude spatial clustering, which is only a small proportion of the whole data set. The final clustering result is then propagated combining the crude spatial clustering and SpatSC results on calibration set. By doing so, the computation cost is reduced by an order of two magnitude compare to the original SpatSC. We applied this random spatial subspace clustering algorithm on real thermal infrared drill hole data set to show its effectiveness.

In this paper, we extend our previously proposed line detection method to line segmentation using a so-called unite-and-divide (UND) approach. The methodology includes two phases, namely the union of spectra in the frequency domain, and the division of the sinogram in Radon space. In the union phase, given an image, its sinogram is obtained by parallel 2D multilayer Fourier transforms, Cartesian-to-polar mapping and 1D inverse Fourier transform. In the division phase, the edges of butterfly wings in the neighborhood of every sinogram peak are firstly specified, with each neighborhood area corresponding to a window in image space. By applying the separated sinogram of each such windowed image, we can extract the line segments. The division Phase identifies the edges of butterfly wings in the neighborhood of every sinogram peak such that each neighborhood area corresponds to a window in image space. Line segments are extracted by applying the separated sinogram of each windowed image. Our experiments are conducted on benchmark images and the results reveal that the UND method yields higher accuracy, has lower computational cost and is more robust to noise, compared to existing state-of-the-art methods.

H.264/MVC multi-view video coding provides a better compression rate compared to the simulcast coding using hierarchical B-picture prediction structure exploiting inter- and intra-view redundancy. However, this technique imposes random access frame delay as well as requiring huge computational time. In this paper a novel technique is proposed using 3D motion estimation (3D-ME) to overcome the problems. In the 3D-ME technique, a 3D frame is formed using the same temporal frames of all views and ME is carried out for the current 3D frame using the immediate previous 3D frame as a reference frame. As the correlation among the intra-view images is higher compared to the correlation among the inter-view images, the proposed 3D-ME technique reduces the overall computational time and eliminates the frame delay with comparable rate-distortion (RD) performance compared to H.264/MVC. Another technique is also proposed in the paper where an extra reference 3D frame comprising dynamic background frames (the most common frame of a scene i.e., McFIS) of each view is used for 3D-ME. Experimental results reveal that the proposed 3D-ME-McFIS technique outperforms the H.264/MVC in terms of improved RD performance by reducing computational time and by eliminating the random access frame delay.

Image matting refers to the problem of accurately extracting foreground objects in images and video. The most recent works in natural image matting relies on the local and manifold smoothness assumptions on foreground and background colors on which a cost function is established. In this paper, we present a framework of formulating new regularization for robust solutions and illustrate new algorithms using the standard benchmark images.

Dimensionality reduction (DR) has been considered as one of the most significant tools for data analysis. In this paper we propose a new latent variable model based on the thin plate splines, named Thin Plate Spline Latent Variable Model (TPSLVM). It has strong connection with the so-called Gaussian Process Latent Variable Model (GPLVM). We demonstrate that the proposed TPSLVM can be viewed as the GPLVM with a fairly peculiar covariance function. Moreover, compared to GPLVM, TPSLVM is more powerful especially when the dimensionality of the latent space is very low (e.g., 2D or 3D). One of main purposes of DR algorithms is to visualize data in 2D/3D spaces. Therefore, TPSLVM will benefit this process. Experimental results show that TPSLVM provides better data visualization and more efficient dimensionality reduction than GPLVM.

This paper proposes a new and optimal moving average model that reduces the problems of alternative models. The random (noisy) nature of financial time series creates difficulties when modelling with any method. The most common linear model to deal with this issue of noise is the moving average. These filters come with the drawback of lag, a delay between the model output and the financial data. As more noise reduction is demanded from the models the lag increases. This lag is a hindrance in a market place where individuals are competing for timely and quality information. This paper derives an optimal moving average model which reduces the lag and increases the level of noise reduction. The proposed model was compared against four of the common moving averages and shown to be superior in both lag reduction and noise reduction.

Data pre-processing and cleansing play a vital role in data mining for ensuring good quality of data. Data cleansing tasks include imputation of missing values, and identification and correction of incorrect/noisy data. In this paper, we present a novel approach called Co-appearance based Analysis for Incorrect Records and Attribute-values Detection (CAIRAD). For a data set having incorrect/noisy values CAIRAD separates the noisy records from the clean records. It thereby produces two data sets; a clean data set and a data set having all noisy records. It also reports noisy attribute values of each noisy record. We evaluate CAIRAD on four publicly available natural data sets by comparing its performance with the performance of two high quality existing techniques namely RDCL and EDIR. We use various patterns (of noisy values) each having different noise levels. Several evaluation criteria such as error recall (????), error precision (????), F-measure, record removal ratio (??????), and area under a receiver operating characteristics curve (AUC) are used. Our experimental results indicate that CAIRAD performs significantly better (based on t-test analysis) than RDCL and EDIR.

The Gaussian process (GP) latent variable model (GPLVM) has the capability of learning low-dimensional manifold from highly nonlinear data of high dimensionality. As an unsupervised dimensionality reduction (DR) algorithm, the GPLVM has been successfully applied in many areas. However, in its current setting, GPLVM is unable to use label information, which is available for many tasks; therefore, researchers proposed many kinds of extensions to the GPLVM in order to utilize extra information, among which the supervised GPLVM (SGPLVM) has shown better performance compared with other SGPLVM extensions. However, the SGPLVM suffers in its high computational complexity. Bearing in mind the issues of the complexity and the need of incorporating additionally available information, in this paper, we propose a novel SGPLVM, called supervised latent linear GPLVM (SLLGPLVM). Our approach is motivated by both SGPLVM and supervised probabilistic principal component analysis (SPPCA). The proposed SLLGPLVM can be viewed as an appropriate compromise between the SGPLVM and the SPPCA. Furthermore, it is also appropriate to interpret the SLLGPLVM as a semiparametric regression model for supervised DR by making use of the GP to model the unknown smooth link function. Complexity analysis and experiments show that the developed SLLGPLVM outperforms the SGPLVM not only in the computational complexity but also in its accuracy. We also compared the SLLGPLVM with two classical supervised classifiers, i.e., a GP classifier and a support vector machine, to illustrate the advantages of the proposed model.

In this paper we propose a nonparametric extension to the sparse kernel machine using a beta process prior. The extended beta process sparse kernel machine (BPSKM) allows for a sparse model to be constructed from a set of training data. The recent research on beta process reveals elegant property of Bayesian conjugate prior which is utilized to derive a variational Bayes inference algorithm. The performance of the proposed algorithm has been investigated on both synthetic and real-life data sets.

Dimensionality reduction has been considered as one of the most significant tools for data analysis. In general, supervised information is helpful for dimensionality reduction. However, in typical real applications, supervised information in multiple source tasks may be available, while the data of the target task are unlabeled. An interesting problem of how to guide the dimensionality reduction for the unlabeled target data by exploiting useful knowledge, such as label information, from multiple source tasks arises in such a scenario. In this paper, we propose a new method for dimensionality reduction in the transfer learning setting. Unlike traditional paradigms where the useful knowledge from multiple source tasks is transferred through distance metric, our proposal firstly converts the dimensionality reduction problem into integral regression problems in parallel. Gaussian process is then employed to learn the underlying relationship between the original data and the reduced data. Such a relationship can be appropriately transferred to the target task by exploiting the prediction ability of the Gaussian process model and inventing different kinds of regularizers. Extensive experiments on both synthetic and real data sets show the effectiveness of our method.

Content-based Image Retrieval (CBIR) has been an active area of research for retrieving similar images from large repositories, without the prerequisite of manual labeling. Most current CBIR algorithms can faithfully return a list of images that matches the visual perspective of their inventors, who might decide to use a certain combination of image features like edges, colors and textures of regions as well as their spatial distribution during processing. In practice, however, the retrieved images rarely correspond exactly to the results expected by the users, a problem that has come to be known as the semantic gap. In this paper, we propose a novel and extensible multidimensional approach called matrix of visual perspectives as a solution for addressing this semantic gap. Our approach exploits the dynamic cross-interaction (in other words, mix-and-match) of image features and similarity metrics to produce results that attempt to mimic the mental visual picture of the user. Experimental results on retrieving similar Japanese cultural heritage symbols called kamons by a prototype system confirm that the interaction of visual perspectives in the user can be effectively captured and reflected. The benefits of this approach are broader. They can be equally applicable to the development of CBIR systems for other types of images, whether cultural or noncultural, by adapting to different sets of application specific image features.

In this paper an improved hill climbing algorithm based method is presented to cut character out of the license plate images. Although there are many existing commercial LPR systems, with poor illumination conditions and moving vehicle the accuracy impaired. After examination and comparison of two different types of image segmentation approaches, the hill climbing algorithm based method gave a better image segmentation results. The hill climbing algorithm was modified by introducing automatic parameter determination and smart searching. After modification it efficiently detects the peaks (local maxima) that represent different clusters in the global histogram of an image. The process is successful by getting a clean license plate image removing all unwanted areas. While testing by the OCR software, the experimental results show a high accuracy of image segmentation and significantly higher recognition rate after non-character areas are removed. The recognition rate increased from about 30.6% before our proposed process to about 91.3% after all unwanted non-character areas are removed. Hence, the overall recognition accuracy of LPR was improved.

In recent years, law enforcement personnel have greatly been aided by the deployment of Automated Fingerprint Identification Systems (AFIS). These systems largely operate by matching salient features automatically extracted from fingerprint images for their decision. However, there are two major shortcomings in current systems. First, the result of identification depends primarily on the chosen features and the algorithm that matches them. Second, these systems cannot improve their results by benefiting from interactions with seasoned examiners who often can identify minute differences between fingerprints beyond that is capable of by current systems. In this paper, we propose a system for fingerprint identification that incorporates relevance feedback. We show that a persistent semantic space over the database of fingerprints can be incrementally learned. Here, the learning module makes use of a dimensionality reduction process that returns both a low-dimensional semantic space and an out-of-sample mapping function, achieving a two-fold benefits of data compression and the ability to project novel fingerprints directly onto the semantic space for identification. Experimental results demonstrated the potential of this learning framework for adaptive fingerprint identification.

A new dimensionality reduction method, called relevance units latent variable model (RULVM), is proposed in this paper. RULVM has a close link with the framework of Gaussian process latent variable model (GPLVM) and it originates from a recently developed sparse kernel model called relevance units machine (RUM). RUM follows the idea of relevance vector machine (RVM) under the Bayesian framework but releases the constraint that relevance vectors (RVs) have to be selected from the input vectors. RUM treats relevance units (RUs) as part of the parameters to be learned from the data. As a result, a RUM maintains all the advantages of RVM and offers superior sparsity. RULVM inherits the advantages of sparseness offered by the RUM and the experimental result shows that RULVM algorithm possesses considerable computational advantages over GPLVM algorithm.

Using local data information, the recently proposed local Fisher Discriminant Analysis (LFDA) algorithm18 provides a new way of handling the multimodal issues within classes where the conventional Fisher Discriminant Analysis (FDA) algorithm fails. Like the FDA algorithm (global counterpart), the LFDA suffers when it is applied to the higher dimensional data sets. In this paper, we propose a new formulation by which a robust algorithm can be formed. The new algorithm offers more robust results for higher dimensional data sets when compared with the LFDA in most cases. By extensive simulation studies, we have demonstrated the practical usefulness and robustness of our new algorithm in data visualization.

In many machine vision applications, a set of static and Pan-Tilt-Zoom (PTZ) cameras are used to capture a sequence of high-resolution facial images of a moving person. In this paper, we present our implementation of such a system. We emphasis two novelties in our work; the first one is our efficient PTZ camera calibration technique using hand-drawn gridlines. The second one is our head position estimation technique using the Gaussian Mixture Model (GMM) and variance analysis of the foreground blob regions.

We introduce a robust probabilistic L1-PCA model in which the conventional gaussian distribution for the noise in the observed data was replaced by the Laplacian distribution (or L1 distribution). Due to the heavy tail characteristics of the L1 distribution, the proposed model is supposed to be more robust against data outliers. In this letter, we demonstrate how a variational approximation scheme enables effective inference of key parameters in the probabilistic L1-PCA model. As the L1 density can be expanded as a superposition of infinite number of gaussian densities, we express the L1-PCA model as a marginalized model over the superpositions. By doing so, a tractable Bayesian inference can be achieved based on the variational expectation-maximization-type algorithm.

In most existing dimensionality reduction algorithms, the main objective is to preserve relational structure among objects of the input space in a low dimensional embedding space. This is achieved by minimizing the inconsistency between two similarity/dissimilarity measures, one for the input data and the other for the embedded data, via a separate matching objective function. Based on this idea, a new dimensionality reduction method called Twin Kernel Embedding (TKE) is proposed. TKE addresses the problem of visualizing non-vectorial data that is difficult for conventional methods in practice due to the lack of efficient vectorial representation. TKE solves this problem by minimizing the inconsistency between the similarity measures captured respectively by their kernel Gram matrices in the two spaces. In the implementation, by optimizing a nonlinear objective function using the gradient descent algorithm, a local minimum can be reached. The results obtained include both the optimal similarity preserving embedding and the appropriate values for the hyperparameters of the kernel. Experimental evaluation on real non-vectorial datasets confirmed the effectiveness of TKE. TKE can be applied to other types of data beyond those mentioned in this paper whenever suitable measures of similarity/dissimilarity can be defined on the input data.

MOTIVATION: Kinetic rate in gene expression is a key measurement of the stability of gene products and gives important information for the reconstruction of genetic regulatory networks. Recent developments in experimental technologies have made it possible to measure the numbers of transcripts and protein molecules in single cells. Although estimation methods based on deterministic models have been proposed aimed at evaluating kinetic rates from experimental observations, these methods cannot tackle noise in gene expression that may arise from discrete processes of gene expression, small numbers of mRNA transcript, fluctuations in the activity of transcriptional factors and variability in the experimental environment. RESULTS: In this paper, we develop effective methods for estimating kinetic rates in genetic regulatory networks. The simulated maximum likelihood method is used to evaluate parameters in stochastic models described by either stochastic differential equations or discrete biochemical reactions. Different types of non-parametric density functions are used to measure the transitional probability of experimental observations. For stochastic models described by biochemical reactions, we propose to use the simulated frequency distribution to evaluate the transitional density based on the discrete nature of stochastic simulations. The genetic optimization algorithm is used as an efficient tool to search for optimal reaction rates. Numerical results indicate that the proposed methods can give robust estimations of kinetic rates with good accuracy.

This paper reports a negative result of image recognition algorithm constructed on the bag of pixels, a new representation method of images, and the manifolds of configurations of representative vectors of images which is spanned by using soft permutation matrix. Rather than represented by a single conventional vector of intensity of pixels row by row, an image is described by a column vector whose entries are n-tuples (x, y, pl,...,ps), where pi,i=1...s is the i-th property of the pixel at (x,y) which can be intensity or color information. So the image can be viewed as a point in high dimensional feature space §. On the ground of this representation method and the researches of others, a new recognition algorithm is presented and tested with results far from satisfaction. The main contribution of this paper is that by using the basic analytical method, it provides a tentative explanation which reveals that some faults of the model and optimization algorithm confined its application in recognition.

As part of a larger research project whose objective was to determine the antecedent condition for e-Procurement assimilation, this paper assesses the organizational e-Business capabilities and supplier readiness in the Australian public sector. With the help of an extensive literature review and pilot study, organizational e-Business capabilities and supplier e-Business readiness factors were identified, a research model was developed and hypotheses were formulated. Based on the Resource-based View of the firm, it is argued that organizational e-Business capabilities such as authentication, interoperability, legal framework, open standards, and security and privacy and also supplier e-Business readiness factors positively impact the transactional and strategic assimilation of e-Procurement. A nationwide web-based survey of Procurement/e-Procurement professionals in the Australian public sector is in the final stage of completion at the time of writing of this paper and analysis of the complete set of data will be presented at the conference. Interested readers can also obtain the results of this study by emailing one of the authors of the paper.

There has been an increasing interest in kernel-based techniques, such as support vector techniques, regularization networks, and gaussian processes. There are inner relationships among those techniques, with the kernel function playing a central role. This article discusses a new class of kernel functions derived from the so-called frames in a function Hilbert space.