Seismic fault image segmentation is crucial for interpreting subsurface geological structures, supporting geologists in resource exploration and structural analysis. However, current deep learning models struggle with single-architecture limitations and the distinctive characteristics of seismic faults, which are distinguished by elongated structures with uneven spatial distributions. To address these challenges, we propose TopSUMseg, a novel Topology-Aware Swin Transformer-Mamba framework for 3D seismic fault image segmentation. Our framework combines Swin Transformer’s local feature extraction with Mamba’s efficient sequence modeling, and boosts 3D spatial modeling in Mamba with a newly designed Global-Local Attention module (GLA). Additionally, we design a Topology-Aware Structural Constraint (TASC) to align predictions with ground-truth structures in the feature space, promoting the modeling of complex fault geometries. Experiments on Thebe, the largest public seismic dataset, demonstrate that TopSUMseg achieves state-of-the-art performance with OIS and ODS scores of 0.879 and 0.875, respectively. Trained entirely from scratch, TopSUMseg nonetheless achieves superior performance compared to extensively pre-trained counterparts. In addition, TopSUMseg maintains a significantly lower parameter count while achieving a favorable trade-off between segmentation performance and time complexity, making it a practical and generalizable solution for real-world seismic fault interpretation.

Multimodal large language models (MLLMs) have achieved significant progress in document understanding. However, complex layout reasoning, characterized by concise answers and cross-page integration, remains a challenge. Unlike conventional semantics-oriented tasks, this task demands accurate visual perception of fine-grained structural elements and logical reasoning across multi-page documents. Existing approaches primarily focus on information extraction and semantic understanding, limiting the capacity of fine-tuned autoregressive models to capture short-answer reasoning signals and generalize to complex layout structures. To address this, we propose the Layout-Aware Multi-Source Reasoning Decision Framework (LAMRD), which reframes complex layout reasoning as a decision-making task over multi-source reasoning paths. In the reasoning path construction stage, LAMRD generates layout-aware reasoning paths by integrating internal visual cues and external knowledge from three complementary perspectives: Visual Structural Awareness (VSA), Logical Reasoning Paths (LRP), and External Knowledge Augmentation (EKA). In the reasoning path decision stage, we employ Group Relative Policy Optimization (GRPO) to train a decision model that produces the final answer based on these paths. We conduct comprehensive evaluations using Qwen2.5-VL-7B-Instruct on the CEP-7K dataset, covering layout structure understanding, information extraction, and logical association. Experimental results demonstrate that LAMRD outperforms advanced MLLMs in accuracy, validating its effectiveness for complex document layout understanding.

In handling complex, domain-specific tasks, particularly in the context of state-owned assets and enterprises (SOAEs), general LLMs suffer from the knowledge gap due to insufficient exposure to domain-specific corpora, and the value disagreement, as they are aligned with universal values rather than domain-specific ones. To tackle these challenges, we propose a novel training strategy tailored for the SOAEs domain. This strategy includes a improved domain-adaptive pretraining (DAP) phase with a replay mechanism to mitigate catastrophic forgetting. Following DAP, we utilize a selective portion of domain-specific data for supervised fine-tuning (SFT), and innovatively integrate low-quality data with the remaining SFT data to curate tailored preference datasets, leveraging the Kahneman-Tversky Optimization technique to align our LLMs. Our proposed approach effectively utilizes the data that is often discarded in conventional training procedures, highlighting the substantial improvements in model performance and the importance of training methodologies for domain-specific tasks.

Mathematical reasoning remains an ongoing challenge for AI models, especially for geometry problems, which require both linguistic and visual signals. As the vision encoders of most MLLMs are trained on natural scenes, they often struggle to understand geometric diagrams, performing no better in geometry problem-solving than LLMs that only process text. This limitation is further amplified by the lack of effective methods for representing geometric relationships. To address these issues, we introduce the Diagram Formalization Enhanced Geometry Problem Solver (DFE-GPS), a new framework that integrates visual features, geometric formal language, and natural language representations. Specifically, we propose a novel synthetic data approach and construct a large-scale geometric dataset, SynthGeo228K, annotated with formal and natural language captions, designed to enhance the vision encoder to understand geometric structures better. Our framework improves MLLMs’ ability to process geometric diagrams and extends their application to open-ended tasks on the formalgeo7k dataset.

Combining face-swapping with lip synchronization offers a cost-effective solution for generating customized talking faces. However, directly cascading existing models can introduce significant interference and reduce video clarity due to limited interaction space in the low-level RGB domain. To solve this, we propose SwapTalk, a unified framework that performs face-swapping and lip synchronization within the same latent VQ-embedding space, known for its editability and fidelity. We enhance generalization to unseen identities with identity loss in the face-swapping module and improve synchronization quality with expert discriminator supervision. To better approximate real-world applications, we expand the evaluation scope to asynchronous audio-video scenarios. Furthermore, we introduce a novel identity consistency metric to more comprehensively assess the identity consistency over time series in generated facial videos. Experiments on HDTF show that SwapTalk outperforms existing methods in video quality, lip synchronization accuracy, face-swapping fidelity, and identity consistency.

Transformers have found extensive applications across various domains due to their powerful fitting capabilities. This success can be partially attributed to their inherent nonlinearity. Thus, in addition to the ReLU function employed in the original transformer architecture, researchers have explored alternative modules such as GeLU and SwishGLU to enhance nonlinearity and thereby augment representational capacity. In this paper, we propose a novel category of polynomial composition activations (PolyCom), designed to optimize the dynamics of transformers. Theoretically, we provide a comprehensive mathematical analysis of PolyCom, highlighting its enhanced expressivity and efficacy relative to other activation functions. Notably, we demonstrate that networks incorporating PolyCom achieve the **optimal approximation rate**, indicating that PolyCom networks require minimal parameters to approximate general smooth functions in Sobolev spaces. We conduct empirical experiments on the pre-training configurations of large language models (LLMs), including both dense and sparse architectures. By substituting conventional activation functions with PolyCom, we enable LLMs to capture higher-order interactions within the data, thus improving performance metrics in terms of accuracy and convergence rates. Extensive experimental results demonstrate the effectiveness of our method, showing substantial improvements over other activation functions. Code is available at https://github.com/BryceZhuo/PolyCom.

Transformers have become the de facto architecture for a wide range of machine learning tasks, particularly in large language models (LLMs). Despite their remarkable performance, many challenges remain in training deep transformer networks, especially regarding the position of the layer normalization. While Pre-Norm structures facilitate more stable training owing to their stronger identity path, they often lead to suboptimal performance compared to Post-Norm. In this paper, we propose **HybridNorm**, a simple yet effective hybrid normalization strategy that integrates the advantages of both Pre-Norm and Post-Norm. Specifically, HybridNorm employs QKV normalization within the attention mechanism and Post-Norm in the feed-forward network (FFN) of each transformer block. We provide both theoretical insights and empirical evidence to demonstrate that HybridNorm improves the gradient flow and the model robustness. Extensive experiments on large-scale transformer models, including both dense and sparse variants, show that HybridNorm consistently outperforms both Pre-Norm and Post-Norm approaches across multiple benchmarks. These findings highlight the potential of HybridNorm as a more stable and effective technique for improving the training and performance of deep transformer models. Code is available at https://github.com/BryceZhuo/HybridNorm.

The development of Long-Context Large Language Models (LLMs) has markedly advanced natural language processing by facilitating the process of textual data across long documents and multiple corpora. However, Long-Context LLMs still face two critical challenges: The lost in the middle phenomenon, where crucial middle-context information is likely to be missed, and the distraction issue that the models lose focus due to overly extended contexts. To address these challenges, we propose the Context Filtering Language Model (FltLM), a novel integrated Long-Context LLM which enhances the ability of the model on multi-document question-answering (QA) tasks. Specifically, FltLM innovatively incorporates a context filter with a soft mask mechanism, identifying and dynamically excluding irrelevant content to concentrate on pertinent information for better comprehension and reasoning. Our approach not only mitigates these two challenges, but also enables the model to operate conveniently in a single forward pass. Experimental results demonstrate that FltLM significantly outperforms supervised fine-tuning and retrieval-based methods in complex QA scenarios, suggesting a promising solution for more accurate and reliable long-context natural language understanding applications.

Recent years have witnessed the explosive growth of interaction behaviors in multimedia information systems, where multi-behavior recommender systems have received increasing attention by leveraging data from various auxiliary behaviors such as tip and collect. Among various multi-behavior recommendation methods, non-sampling methods have shown superiority over negative sampling methods. However, two observations are usually ignored in existing state-of-the-art non-sampling methods based on binary regression: (1) users have different preference strengths for different items, so they cannot be measured simply by binary implicit data; (2) the dependency across multiple behaviors varies for different users and items. To tackle the above issue, we propose a novel non-sampling learning framework namedCriterion-guidedHeterogeneousCollaborativeFiltering (CHCF). CHCF introduces both upper and lower thresholds to indicate selection criteria, which will guide user preference learning. Besides, CHCF integrates criterion learning and user preference learning into a unified framework, which can be trained jointly for the interaction prediction of the target behavior. We further theoretically demonstrate that the optimization of Collaborative Metric Learning can be approximately achieved by the CHCF learning framework in a non-sampling form effectively. Extensive experiments on three real-world datasets show the effectiveness of CHCF in heterogeneous scenarios.

Point cloud classification has sparked many researchers’ interest for its cornerstone role in 3D applications. Inheriting the CutMix series augmentation that performs well in 2D images, PointCutMix and RSMix are proposed to generate new samples for 3D point clouds, by replacing partial points of one cloud with those of another. However, the selection of mixed regions is all built on randomness, ignoring the significance of point clouds’ saliency. To address this deficiency, we propose PCSalMix: a novel Saliency-based Mix augmentation for Point Cloud classification. The gradient of classification network on inputs is a natural tool to locate the saliency. Based on this discovery, we extract points with larger gradient values to make more representative samples. Afterward, the soft labels are weighted more accurately by accumulated gradients rather than count ratios of points. The experimental results verify the outperformance of our method on ModelNet40 and ModelNet10 benchmarks in terms of accuracy and robustness against adversarial attacks.

The success of CutMix in image classification has sparked interest in saliency-based mix augmentation methods, which refer to detecting saliency regions to generate more valid images. However, existing mix works either require external tools to locate saliency regions, or rely on additional complex optimization policy for generating new images, which limits their application ranges. To address these deficiencies, we propose Gradient Saliency-based Mix (GradSalMix), a simple yet more general mix augmentation, whose operations are all based on the gradients of the training neural network itself. Specifically, we first locate the saliency regions of two images via their gradients of manifolds, and then directly migrate the region, sampled around the center with a large gradient response value, from one image to another. Afterwards, the labels of images are weighted by their accumulated gradient values for new soft labels, which are shown more accurate than the ones weighted by area ratio. The experimental results show that our proposed method outperforms previous works, in terms of accuracy and robustness against adversarial attacks, on four image classification benchmarks. Moreover, extensive experiments on object detection and point cloud classification also verify the superiority and generality of our method.

Network architecture plays a pivotal role in style transfer. Most existing algorithms use VGG19 as the feature extractor, which incurs a high computational cost. In this work, we conduct an empirical study on the popular network architectures and find that some more efficient networks can replace VGG19 while having comparable style transfer performance. Beyond that, we show that an efficient network can be further accelerated by removing its empty channels via a simple channel pruning method tweaked for style transfer. To prevent the potential performance drop due to using a more lightweight network and obtain better style transfer results, we introduce a more accurate deep feature alignment strategy to improve existing style transfer modules. Taking GoogLeNet as an exemplary efficient network, the pruned GoogLeNet with the improved style transfer module is 2.3 ~ 107.4× faster than the state-of-the-art approaches and can achieve 68.03 FPS on 512×512 images. Extensive experiments demonstrate that VGG19 can be replaced by a more lightweight network with significantly improved efficiency and comparable style transfer quality.

Steerable models can provide very general and flexible equivariance by formulating equivariance requirements in the language of representation theory and feature fields, which has been recognized to be effective for many vision tasks. However, deriving steerable models for 3D rotations is much more difficult than that in the 2D case, due to more complicated mathematics of 3D rotations. In this work, we employ partial differential operators (PDOs) to model 3D filters, and derive general steerable 3D CNNs, which are called PDO-s3DCNNs. We prove that the equivariant filters are subject to linear constraints, which can be solved efficiently under various conditions. As far as we know, PDO-s3DCNNs are the most general steerable CNNs for 3D rotations, in the sense that they cover all common subgroups of SO(3) and their representations, while existing methods can only be applied to specific groups and representations. Extensive experiments show that our models can preserve equivariance well in the discrete domain, and outperform previous works on SHREC’17 retrieval and ISBI 2012 segmentation tasks with a low network complexity.

Graph clustering aiming to partition nodes into several disjoint subsets is a fundamental task for graph-structured learning. Traditional graph clustering methods only consider the adjacency information. In recent years, inspired by the homophily assumption that the adjacent nodes tend to have similar features and labels, most existing graph clustering approaches leverage node attribute information to improve graph clustering performance. These works have mainly focused on node embedding learning via the various combinations of auto-encoder and graph neural networks. As for clustering learning, they introduce a self-optimizing strategy that assumes that all clusters are homogeneous. However, this assumption usually does not hold since the size and variance of different clusters can be quite different, and self-optimizing strategy is incompetent in dealing with this heterogeneous clusters. In this work, we propose a novel method named Adaptive Harmony Learning and Optimization (AHLO) for attributed graph clustering, which models the node embeddings with the mixture of von Mises-Fisher distributions on the unit hypersphere and develops an alternating learning strategy. Specifically, we take the node embeddings as the supervisory signals for the update of the mixture parameters, and the mixture distribution as the supervisory signals for the update of the node embeddings. To prevent small clusters from annexing by large clusters, we develop the regularized harmony loss to enhance the prediction on small clusters. In the mixture parameter optimization stage, we utilize EM algorithm and heuristically design a center update scheme with consideration of the posterior probability confidence and the impact of other centers. Hence, AHLO can simultaneously improve the intra-cluster compactness and inter-cluster separability. Extensive experiments on four benchmark attributed graph datasets have demonstrated the effectiveness of our proposed AHLO.

Abstract In the variational learning process of a Bayesian Hidden Markov model, the forward‐backward algorithm is heuristically applied without theoretical justification. This is potentially problematic, because the original derivation of the forward‐backward algorithm implicitly requires the parameters to be normalized, which does not hold in the variational learning process of Bayesian HMM. In this paper, we prove that such a requirement is not necessary for the forward‐backward algorithm to obtain the correct result. We prove the result from two perspectives. The first proof straightforwardly verifies that implementing the forward‐backward algorithm with the unnormalised parameters is equivalent to implementing it with the normalized parameters. The second proof provides a new derivation of the forward‐backward algorithm without hidden Markov assumptions and probabilistic meanings of the parameters. As a result, we justify that applying the forward‐backward algorithm is theoretically correct and reasonable in the variational learning of Bayesian hidden Markov models.

This paper studies learning from positive and unlabeled examples, known as PU learning. It proposes a novel PU learning method called Predictive Adversarial Networks (PAN) based on GAN (Generative Adversarial Networks). GAN learns a generator to generate data (e.g., images) to fool a discriminator which tries to determine whether the generated data belong to a (positive) training class. PU learning can be casted as trying to identify (not generate) likely positive instances from the unlabeled set to fool a discriminator that determines whether the identified likely positive instances from the unlabeled set are indeed positive. However, directly applying GAN is problematic because GAN focuses on only the positive data. The resulting PU learning method will have high precision but low recall. We propose a new objective function based on KL-divergence. Evaluation using both image and text data shows that PAN outperforms state-of-the-art PU learning methods and also a direct adaptation of GAN for PU learning.

Continual learning (CL) incrementally learns a sequence of tasks while solving the catastrophic forgetting (CF) problem. Existing methods mainly try to deal with CF directly. In this paper, we propose to avoid CF by considering the features of each class holistically rather than only the discriminative information for classifying the classes seen so far. This latter approach is prone to CF because the discriminative information for old classes may not be sufficiently discriminative for the new class to be learned. Consequently, in learning each new task, the network parameters for previous tasks have to be revised, which causes CF. With the holistic consideration, after adding new tasks, the system can still do well for previous tasks. The proposed technique is called Per-class Continual Learning (PCL). PCL has two key novelties. (1) It proposes a one-class learning based technique for CL, which considers features of each class holistically and represents a new approach to solving the CL problem. (2) It proposes a method to extract discriminative information after training to further improve the accuracy. Empirical evaluation shows that PCL markedly outperforms the state-of-the-art baselines for one or more classes per task. More tasks also result in more gains.

Spherical signals exist in many applications, e.g., planetary data, LiDAR scans and digitalization of 3D objects, calling for models that can process spherical data effectively. It does not perform well when simply projecting spherical data into the 2D plane and then using planar convolution neural networks (CNNs), because of the distortion from projection and ineffective translation equivariance. Actually, good principles of designing spherical CNNs are avoiding distortions and converting the shift equivariance property in planar CNNs to rotation equivariance in the spherical domain. In this work, we use partial differential operators (PDOs) to design a spherical equivariant CNN, PDO-eS2CNN, which is exactly rotation equivariant in the continuous domain. We then discretize PDO-eS2CNNs, and analyze the equivariance error resulted from discretization. This is the first time that the equivariance error is theoretically analyzed in the spherical domain. In experiments, PDO-eS2CNNs show greater parameter efficiency and outperform other spherical CNNs significantly on several tasks.

Rapid and accurate prediction of user preferences is the ultimate goal of today’s recommender systems. More and more researchers pay attention to multi-behavior recommender systems which utilize the auxiliary types of user-item interaction data, such as page view and add-to-cart to help estimate user preferences. Recently, graph-based methods were proposed to showcase an advanced capability in representation learning and capturing collaborative signals. However, we argue that these methods ignore the intrinsic difference between the two types of nodes in the bipartite graph and aggregate information from neighboring nodes with the same functions. Besides, these models do not fully explore the collaborative signals implied by the meta-path across different types of behavior, which causes a huge loss of the potential semantic information across behaviors. To address the above limitations, we present a unified graph model named SaGCN (short for Semantic-aware Graph Convolutional Networks). Specifically, we construct separate user-user and item-item graphs by meta-path, and apply separate aggregation and transformation functions to propagate user and item information. To perform better semantic propagation, we design a relation composition function and a semantic propagation architecture for heterogeneous collaborative filtering signals learning. Extensive experiments on two real-world datasets show that SaGCN outperforms a wide range of state-of-the-art methods in multi-behavior scenarios.

Hashing has been widely used in approximate nearest neighbor search recently. Deep supervised hashing methods are not widely-used because of the lack of labeled data, especially when the domain is transferred. Meanwhile, unsupervised deep hashing models can hardly achieve satisfactory performance due to the lack of reliable similarity signals. Here, we propose a novel deep unsupervised hashing method, namely Distilled Smooth Guidance (DSG), which can learn a distilled dataset consisting of similarity signals as well as smooth confidence signals. Specifically, we obtain the similarity confidence weights based on the initial noisy similarity signals learned from local structures and construct a priority loss function for smooth similarity-preserving learning. Besides, global information based on clustering is utilized to distill the image pairs by removing contradictory similarity signals. Extensive experiments on three widely used bench-mark datasets show that the proposed DSG consistently out-performs the state-of-the-art search methods.

Hashing is widely-used in approximate nearest neighbor search for its computational efficiency. Most of the existing unsupervised hashing methods are based on local consistency that the Hamming distance between two images should be small if their features are similar. However, many false similar pairs may be included for the insufficient representation of features. Here we proposed deep unsupervised hashing by Global and Local Consistency (GLC). Specifically, GLC has two components named semantic information generating and semantic consistency learning, and each component is conducted from both global and local views. From local view, GLC introduces reliable graph and penalty graph to capture local signals with high confidence to preserve the semantic structure. From global view, GLC includes a distribution loss to capture the global consistency with cluster signals. Extensive experimental results on three widely-used benchmark datasets show that GLC performs better than existing state- of-the-art methods.

Recently, hashing is widely used in approximate nearest neighbor search for its storage and computational efficiency. Most of the unsupervised hashing methods learn to map images into semantic similarity-preserving hash codes by constructing local semantic similarity structure from the pre-trained model as the guiding information, i.e., treating each point pair similar if their distance is small in feature space. However, due to the inefficient representation ability of the pre-trained model, many false positives and negatives in local semantic similarity will be introduced and lead to error propagation during the hash code learning. Moreover, few of the methods consider the robustness of models, which will cause instability of hash codes to disturbance. In this paper, we propose a new method named Comprehensive sImilarity Mining and cOnsistency learNing (CIMON). First, we use global refinement and similarity statistical distribution to obtain reliable and smooth guidance. Second, both semantic and contrastive consistency learning are introduced to derive both disturb-invariant and discriminative hash codes. Extensive experiments on several benchmark datasets show that the proposed method outperforms a wide range of state-of-the-art methods in both retrieval performance and robustness.

With the increasing scale and diversification of interaction behaviors in E-commerce, more and more researchers pay attention to multi-behavior recommender systems which utilize interaction data of other auxiliary behaviors. However, all existing models ignore two kinds of intrinsic heterogeneity which are helpful to capture the difference of user preferences and the difference of item attributes. First (intra-heterogeneity), each user has multiple social identities with otherness, and these different identities can result in quite different interaction preferences. Second (inter-heterogeneity), each item can transfer an item-specific percentage of score from low-level behavior to high-level behavior for the gradual relationship among multiple behaviors. Thus, the lack of consideration of these heterogeneities damages recommendation rank performance. To model the above heterogeneities, we propose a novel method named intrA- and inteR-heteroGeneity recOmmendation model (ARGO). Specifically, we embed each user into multiple vectors representing the user's identities, and the maximum of identity scores indicates the interaction preference. Besides, we regard the item-specific transition percentage as trainable transition probability between different behaviors. Extensive experiments on two real-world datasets show that ARGO performs much better than the state-of-the-art in multi-behavior scenarios.

Many representation methods were built for gray image textures. However, they are not effective for color textures in general. To alleviate this problem, in this paper we propose a novel Compact Interchannel Sampling Difference Descriptor (CISDD) for color texture classification. In particular, considering sampling-based method can capture more directional information, we first use a heavy-tailed distribution, t-distribution to generate sample points in the image patch to calculate the micro-block difference. Then we model the interchannel relationship of color texture image by using dense micro-block differences. Furthermore, we utilize principal component analysis (PCA) to reduce the dimensions of the features encoded by the Fisher vector, and construct a Compact Interchannel Sampling Difference Descriptor (CISDD) for representing color texture image. Finally, experimental results on five published standard texture datasets (KTH-TIPS, VisTex, CUReT, USPTex and Colored Brodatz) reveal that CISDD is effective and outperforms thirteen representative color texture classification methods.

Images or videos always contain multiple objects or actions. Multi-label recognition has been witnessed to achieve pretty performance attribute to the rapid development of deep learning technologies. Recently, graph convolution network (GCN) is leveraged to boost the performance of multi-label recognition. However, what is the best way for label correlation modeling and how feature learning can be improved with label system awareness are still unclear. In this paper, we propose a label graph superimposing framework to improve the conventional GCN+CNN framework developed for multi-label recognition in the following two aspects. Firstly, we model the label correlations by superimposing label graph built from statistical co-occurrence information into the graph constructed from knowledge priors of labels, and then multi-layer graph convolutions are applied on the final superimposed graph for label embedding abstraction. Secondly, we propose to leverage embedding of the whole label system for better representation learning. In detail, lateral connections between GCN and CNN are added at shallow, middle and deep layers to inject information of label system into backbone CNN for label-awareness in the feature learning process. Extensive experiments are carried out on MS-COCO and Charades datasets, showing that our proposed solution can greatly improve the recognition performance and achieves new state-of-the-art recognition performance.

Understanding neural models is a major topic of interest in the deep learning community. In this paper, we propose to interpret a general neural model comparatively. Specifically, we study the sequence-to-sequence (Seq2Seq) model in the contexts of two mainstream NLP tasks–machine translation and dialogue response generation–as they both use the seq2seq model. We investigate how the two tasks are different and how their task difference results in major differences in the behaviors of the resulting translation and dialogue generation systems. This study allows us to make several interesting observations and gain valuable insights, which can be used to help develop better translation and dialogue generation models. To our knowledge, no such comparative study has been done so far.

Sparsity is regarded as a desirable property of representations, especially in terms of explanation.However, its usage has been limited due to the gap with dense representations.Most research progresses in NLP in recent years are based on dense representations.Thus the desirable property of sparsity cannot be leveraged.Inspired by Fourier Transformation, in this paper, we propose a novel Semantic Transformation method to bridge the dense and sparse spaces, which can facilitate the NLP research to shift from dense spaces to sparse spaces or to jointly use both spaces.Experiments using classification tasks and natural language inference task show that the proposed Semantic Transformation is effective.

Recent research has shown that incorporating equivariance into neural network architectures is very helpful, and there have been some works investigating the equivariance of networks under group actions. However, as digital images and feature maps are on the discrete meshgrid, corresponding equivariance-preserving transformation groups are very limited. In this work, we deal with this issue from the connection between convolutions and partial differential operators (PDOs). In theory, assuming inputs to be smooth, we transform PDOs and propose a system which is equivariant to a much more general continuous group, the $n$-dimension Euclidean group. In implementation, we discretize the system using the numerical schemes of PDOs, deriving approximately equivariant convolutions (PDO-eConvs). Theoretically, the approximation error of PDO-eConvs is of the quadratic order. It is the first time that the error analysis is provided when the equivariance is approximate. Extensive experiments on rotated MNIST and natural image classification show that PDO-eConvs perform competitively yet use parameters much more efficiently. Particularly, compared with Wide ResNets, our methods result in better results using only 12.6% parameters.

Tensor completion and robust principal component analysis have been widely used in machine learning while the key problem relies on the minimization of a tensor rank that is very challenging. A common way to tackle this difficulty is to approximate the tensor rank with l1-norm of the singular values solved by the Tensor Singular Value Decomposition (T-SVD). Besides, the sparsity of a tensor is also measured with l1- norm. However, the l1penalty is essentially biased and thus the result will deviate. In order to sidestep the bias, we propose a novel non-convex tensor rank surrogate function and a novel non-convex sparsity measure. In this new setting by using the concavity instead of the convexity, a majorization minimization algorithm is further designed for tensor completion and robust principal component analysis. Furthermore, we analyze its theoretical properties. Finally, the experiments on both natural and hyperspectral images demonstrate the efficacy and efficiency of our proposed method.

Existing neural network based one-class learning methods mainly use various forms of auto-encoders or GAN style adversarial training to learn a latent representation of the given one class of data. This paper proposes an entirely different approach based on a novel regularization, called holistic regularization (or H-regularization), which enables the system to consider the data holistically, not to produce a model that biases towards some features. Combined with a proposed 2-norm instance-level data normalization, we obtain an effective one-class learning method, called HRN. To our knowledge, the proposed regularization and the normalization method have not been reported before. Experimental evaluation using both benchmark image classification and traditional anomaly detection datasets show that HRN markedly outperforms the state-of-the-art existing deep/non-deep learning models.

As an analytic tool in medicine, particularly in radiology, deep learning is gaining much attention and opening a new way for disease diagnosis. Nonetheless, it is rather challenging to acquire large‐scale detailed labelled datasets in the field of medical imaging. In fact, transfer learning provides a possible way to resolve this issue to a certain extent such that the parameter learning of a neural network starts with its pre‐trained weights learned from a large‐scale dataset of certain similar task, and fine‐tunes on a small comprehensively annotated dataset for the particular target task. The main aim of this study is to apply the deep learning model to detect the synovial fluid of human knee joint from magnetic resonance images. A specialized convolutional neural network architecture is proposed for automated detection of human knee joint's synovial fluid. Two independent datasets are used in the training, development, and evaluation of the proposed model. It is demonstrated by the experimental results that the proposed model obtains high sensitivity, specificity, precision, and accuracy to the detection of human knee joint's synovial fluid. As a result, this proposed approach provides a novel and feasible way for automating and expediting the synovial fluid analysis.

Compared with standard supervised learning, the key difficulty in semi-supervised learning is how to make full use of the unlabeled data. A recently proposed method, virtual adversarial training (VAT), smartly performs adversarial training without label information to impose a local smoothness on the classifier, which is especially beneficial to semi-supervised learning. In this work, we propose tangent-normal adversarial regularization (TNAR) as an extension of VAT by taking the data manifold into consideration. The proposed TNAR is composed by two complementary parts, the tangent adversarial regularization (TAR) and the normal adversarial regularization (NAR). In TAR, VAT is applied along the tangent space of the data manifold, aiming to enforce local invariance of the classifier on the manifold, while in NAR, VAT is performed on the normal space orthogonal to the tangent space, intending to impose robustness on the classifier against the noise causing the observed data deviating from the underlying data manifold. Demonstrated by experiments on both artificial and practical datasets, our proposed TAR and NAR complement with each other, and jointly outperforms other state-of-the-art methods for semi-supervised learning.

Fashion landmark detection is a challenging task even using the current deep learning techniques, due to the large variation and non-rigid deformation of clothes. In order to tackle these problems, we propose Spatial-Aware Non-Local (SANL) block, an attentive module in the deep neural network which can utilize spatial and semantic information while capturing global dependency. The attention maps are generated by Grad-CAM or a human parsing segmentation model and then fed into the SANL blocks via attention mechanism. We then establish our fashion landmark detection framework on feature pyramid network, equipped with four SANL blocks in the backbone. It is demonstrated by the experimental results on two large-scale fashion datasets that our proposed fashion landmark detection approach with the SANL blocks outperforms the current state-of-the-art methods considerably. Some supplementary experiments on fine-grained image classification also show the effectiveness of the proposed SANL block.

Understanding the behavior of stochastic gradient descent (SGD) in the context of deep neural networks has raised lots of concerns recently. Along this line, we study a general form of gradient based optimization dynamics with unbiased noise, which unifies SGD and standard Langevin dynamics. Through investigating this general optimization dynamics, we analyze the behavior of SGD on escaping from minima and its regularization effects. A novel indicator is derived to characterize the efficiency of escaping from minima through measuring the alignment of noise covariance and the curvature of loss function. Based on this indicator, two conditions are established to show which type of noise structure is superior to isotropic noise in term of escaping efficiency. We further show that the anisotropic noise in SGD satisfies the two conditions, and thus helps to escape from sharp and poor minima effectively, towards more stable and flat minima that typically generalize well. We systematically design various experiments to verify the benefits of the anisotropic noise, compared with full gradient descent plus isotropic diffusion (i.e. Langevin dynamics).

Existing neural models for dialogue response generation assume that utterances are sequentially organized. However, many real-world dialogues involve multiple interlocutors (i.e., multi-party dialogues), where the assumption does not hold as utterances from different interlocutors can occur ``in parallel.'' This paper generalizes existing sequence-based models to a Graph-Structured neural Network (GSN) for dialogue modeling. The core of GSN is a graph-based encoder that can model the information flow along the graph-structured dialogues (two-party sequential dialogues are a special case). Experimental results show that GSN significantly outperforms existing sequence-based models.

Eye image recognition from a face image acquired at a distant is a promising physical biometric technique to use for human identification. This contemporary field of research depends on image preprocessing, feature extraction, and reliable classification techniques. In this work, we separate eye images from an image of the entire face of a subject and then extract features from these eye images utilizing a convolutional neural network (CNN) model. In general, CNN models convolve images in different layers to extract effective features and then use the softmax function to produce a probability output in the final layer. In our approach, we use CNN features and a kernel extreme learning machine (KELM) classifier instead of softmax to modify the original CNN model. The modified CNN-KELM model has been verified using the publicly available CASIA.v4 distance image database. The experimental results demonstrate that our proposed approach obtains a satisfactory recognition result when compared with several current state-of-the-art human identification approaches.

The mixture of Gaussian processes (GPs) is capable of learning any general stochastic process based on a given set of (sample) curves for the regression and prediction problems. However, it is ineffective for curve clustering and prediction, when the sample curves are derived from different stochastic processes as independent sources linearly mixed together. In this paper, we propose a two-layer mixture model of GP functional regressions (GPFRs) to describe such a mixture of general stochastic processes or independent sources, especially for curve clustering and prediction. Specifically, in the lower layer, the mixture of GPFRs (MGPFRs) is developed for a cluster (or class) of curves within the input space. In the higher layer, the mixture of MGPFRs is further established to divide the curves into clusters according to its components in the output space. For the parameter estimation of the two-layer mixture of GPFRs, we develop a Monte Carlo EM algorithm based on a Monte Carlo Markov chain (MCMC) method, in short, the MCMC EM algorithm. We validate the hierarchical mixture of GPFRs and MCMC EM algorithm using synthetic and real-world data sets. Our results show that our new model outperforms the conventional mixture models in curve clustering and prediction.

This paper presents effective combination models with certain combination features for human detection. In the past several years, many existing features/models have achieved impressive progress, but their performances are still limited by the biases rooted in their self-structures, that is, a particular kind of feature/model may work well for some types of human bodies, but not for all the types. To tackle this difficult problem, we combine certain complementary features/models together with effective organization/fusion methods. Specifically, the HOG features, color features and bar-shape features are combined together with a cell-based histogram structure to form the so-called HOG-III features. Moreover, the detections from different models are fused together with the new proposed weighted-NMS algorithm, which enhances the probable “true” activations as well as suppresses the overlapped detections. The experiments on PASCAL VOC datasets demonstrate that, both the HOG-III features and the weighted-NMS fusion algorithm are effective (obvious improvement for detection performance) and efficient (relatively less computation cost): When applied to human detection task with the Grammar model and Poselet model, they can boost the detection performance significantly; Also, when extended to detection of the whole VOC 20 object categories with the deformable part-based model and deep CNN-based model, they still show competitive improvements.

In this paper, we propose a competitive learning approach to image segmentation by coupling the Mumford-Shah (MS) model and the Distance Sensitive Rival Penalized Competitive Learning (DSRPCL) mechanism, being denoted as the DBMS model. Actually, the DBMS model with the evolution of the level set function can get highly accurate segmentation of the image by automatically detecting the appropriate number of segmented regions and overcoming the problems of vacuum and overlap. It is demonstrated by experimental results on BSDS500 that our DBMS approach can obtain the state-of-the-art segmentation result under the evaluation of ODS index.

For the learning of mixtures of Gaussian processes, model selection is an important but difficult problem. In this paper, we develop an automatic model selection algorithm for mixtures of Gaussian processes in the light of the reversible jump Markov chain Monte Carlo framework for Gaussian mixtures. In this way, the component number and the parameters are updated according the five types of random moves and model selection can be made automatically. The key idea is that the moves of component splitting or merging preserve the zeroth, first and second moments of the components so that the covariance parameters of the new components can be related to the origin ones. It is demonstrated by the simulation experiments that this automatic model selection algorithm is feasible and effective.

The mixture of Gaussian processes (MGP) is a powerful statistical learning model for regression and prediction and the EM algorithm is an effective method for its parameter learning or estimation. However, the feasible EM algorithms for MGPs are certain approximations of the real EM algorithm since Q-function cannot be computed efficiently in this situation. To overcome this problem, we propose an MCMC based EM algorithm for MGPs where Q-function is alternatively estimated on a set of simulated samples via the Markov Chain Monte Carlo (MCMC) method. It is demonstrated by the experiments on both the synthetic and real-world datasets that our proposed MCMC based EM algorithm is more effective than the other three EM algorithms for MGPs.

The Mixture of Gaussian Processes (MGP) is a powerful statistical learning framework in machine learning. For the learning of MGP on a given dataset, it is necessary to solve the model selection problem, i.e., to determine the number C of actual GP components in the mixture. However, the current learning algorithms for MGPs cannot solve this problem effectively. In this paper, we propose an effective model selection criterion, called the Synchronously Balancing or SB criterion for MGPs. It is demonstrated by the experimental results that this SB criterion is feasible and even outperforms two classical criterions: AIC and BIC, for model selection on MGPs. Moreover, it is found that there exists a feasible interval of the penalty coefficient for correct model selection.

Community discovery is very important for understanding the organization or structure of a network or social system. However, it is still a very challenging problem, especially for a large-scale network. In fact, current community mining algorithms generally aim at a special kind of networks and cannot be applied to the general cases. Moreover, they are generally time consuming. This paper proposes a Modularity Dominated Density Based Merge Search (MDDBMS) algorithm which is a further approach to the density based merge search to community mining by considering all the network as a graph of vertexes with the densities as their degrees. In fact, certain criteria are modified and the modularity is used to check whether the merge operation is needed. The experimental results on several datasets of social and protein-protein interaction (PPI) networks demonstrate that our proposed MDDBMS algorithm can obtain competitive results in comparison with current state-of-the-art community mining algorithms with much lower time consumption.

Statistical modeling of wavelet subbands has frequently been used for image recognition and retrieval. However, traditional wavelets are unsuitable for use with images containing distributed discontinuities, such as edges. Shearlets are a newly developed extension of wavelets that are better suited to image characterization. Here, we propose novel texture classification and retrieval methods that model adjacent shearlet subband dependences using linear regression. For texture classification, we use two energy features to represent each shearlet subband in order to overcome the limitation that subband coefficients are complex numbers. Linear regression is used to model the features of adjacent subbands; the regression residuals are then used to define the distance from a test texture to a texture class. Texture retrieval consists of two processes: the first is based on statistics in contourlet domains, while the second is performed using a pseudo-feedback mechanism based on linear regression modeling of shearlet subband dependences. Comprehensive validation experiments performed on five large texture datasets reveal that the proposed classification and retrieval methods outperform the current state-of-the-art.

In this paper, a dynamically regularized harmony learning (DRHL) algorithm is proposed for Gaussian mixture learning with a favourite feature of both adaptive model selection and consistent parameter estimation. Specifically, under the framework of Bayesian Ying-Yang (BYY) harmony learning, we utilize the average Shannon entropy of the posterior probability per sample as a regularization term being controlled by a scale factor to the harmony function on Gaussian mixtures increasing from 0 to 1 dynamically. It is demonstrated by the experiments on both synthetic and real-world datasets that the DRHL algorithm can not only select the correct number of actual Gaussians in the dataset, but also obtain the maximum likelihood (ML) estimators of the parameters in the actual mixture. Moreover, the DRHL algorithm is scalable and can be implemented on a big dataset.

MOTIVATION: Currently there are no curative anticancer drugs, and drug resistance is often acquired after drug treatment. One of the reasons is that cancers are complex diseases, regulated by multiple signaling pathways and cross talks among the pathways. It is expected that drug combinations can reduce drug resistance and improve patients' outcomes. In clinical practice, the ideal and feasible drug combinations are combinations of existing Food and Drug Administration-approved drugs or bioactive compounds that are already used on patients or have entered clinical trials and passed safety tests. These drug combinations could directly be used on patients with less concern of toxic effects. However, there is so far no effective computational approach to search effective drug combinations from the enormous number of possibilities. RESULTS: In this study, we propose a novel systematic computational tool DRUGCOMBORANKER: to prioritize synergistic drug combinations and uncover their mechanisms of action. We first build a drug functional network based on their genomic profiles, and partition the network into numerous drug network communities by using a Bayesian non-negative matrix factorization approach. As drugs within overlapping community share common mechanisms of action, we next uncover potential targets of drugs by applying a recommendation system on drug communities. We meanwhile build disease-specific signaling networks based on patients' genomic profiles and interactome data. We then identify drug combinations by searching drugs whose targets are enriched in the complementary signaling modules of the disease signaling network. The novel method was evaluated on lung adenocarcinoma and endocrine receptor positive breast cancer, and compared with other drug combination approaches. These case studies discovered a set of effective drug combinations top ranked in our prediction list, and mapped the drug targets on the disease signaling network to highlight the mechanisms of action of the drug combinations. AVAILABILITY AND IMPLEMENTATION: The program is available on request.

BACKGROUND: Gene fusions, which result from abnormal chromosome rearrangements, are a pathogenic factor in cancer development. The emerging RNA-Seq technology enables us to detect gene fusions and profile their features. RESULTS: In this paper, we proposed a novel fusion detection tool, FusionQ, based on paired-end RNA-Seq data. This tool can detect gene fusions, construct the structures of chimerical transcripts, and estimate their abundances. To confirm the read alignment on both sides of a fusion point, we employed a new approach, "residual sequence extension", which extended the short segments of the reads by aggregating their overlapping reads. We also proposed a list of filters to control the false-positive rate. In addition, we estimated fusion abundance using the Expectation-Maximization algorithm with sparse optimization, and further adopted it to improve the detection accuracy of the fusion transcripts. Simulation was performed by FusionQ and another two stated-of-art fusion detection tools. FusionQ exceeded the other two in both sensitivity and specificity, especially in low coverage fusion detection. Using paired-end RNA-Seq data from breast cancer cell lines, FusionQ detected both the previously reported and new fusions. FusionQ reported the structures of these fusions and provided their expressions. Some highly expressed fusion genes detected by FusionQ are important biomarkers in breast cancer. The performances of FusionQ on cancel line data still showed better specificity and sensitivity in the comparison with another two tools. CONCLUSIONS: FusionQ is a novel tool for fusion detection and quantification based on RNA-Seq data. It has both good specificity and sensitivity performance. FusionQ is free and available at http://www.wakehealth.edu/CTSB/Software/Software.htm.

As a newly developed 2-D extension of the wavelet transform using multiscale and directional filter banks, the contourlet transform can effectively capture the intrinsic geometric structures and smooth contours of a texture image that are the dominant features for texture classification. In this paper, we propose a novel Bayesian texture classifier based on the adaptive model-selection learning of Poisson mixtures on the contourlet features of texture images. The adaptive model-selection learning of Poisson mixtures is carried out by the recently established adaptive gradient Bayesian Ying-Yang harmony learning algorithm for Poisson mixtures. It is demonstrated by the experiments that our proposed Bayesian classifier significantly improves the texture classification accuracy in comparison with several current state-of-the-art texture classification approaches.

Interactions between cancer cells and factors within the tumor microenvironment (mE) are essential for understanding tumor development. The spatial relationships between blood vessel cells and cancer cells, e.g. tumor initiating cells (TICs), are an important parameter. Accurate segmentation of blood vessel is necessary for the quantization of their spatial relationships. However, this remains an open problem due to uneven intensity and low signal to noise ratio (SNR). To overcome these challenges, we propose a novel approach that integrates an oriented hidden Markov random field model (Ori-HMRF) with local clustering. The local clustering delineates boundaries of blood vessel segments with low SNR. Then blood vessel segments are viewed as random variables in the Ori-HMRF and their spatial dependence is defined based on directional information. The Ori-HMRF model suppresses noise and generates accurate blood vessel segmentation results. Experimental validations were conducted on both normal mammary and breast cancer tissues.

Mixture of experts (ME) is a modular neural network architecture for supervised classification. The double-loop expectation-maximization (EM) algorithm has been developed for learning the parameters of the ME architecture, and the iteratively reweighted least squares (IRLS) algorithm and the Newton-Raphson algorithm are two popular schemes for learning the parameters in the inner loop or gating network. In this letter, we investigate asymptotic convergence properties of the EM algorithm for ME using either the IRLS or Newton-Raphson approach. With the help of an overlap measure for the ME model, we obtain an upper bound of the asymptotic convergence rate of the EM algorithm in each case. Moreover, we find that for the Newton approach as a specific Newton-Raphson approach to learning the parameters in the inner loop, the upper bound of asymptotic convergence rate of the EM algorithm locally around the true solution Θ* is [Formula: see text], where ϵ>0 is an arbitrarily small number, o(x) means that it is a higher-order infinitesimal as x → 0, and e(Θ*) is a measure of the average overlap of the ME model. That is, as the average overlap of the true ME model with large sample tends to zero, the EM algorithm with the Newton approach to learning the parameters in the inner loop tends to be asymptotically superlinear. Finally, we substantiate our theoretical results by simulation experiments.

In this letter, we propose an efficient one-nearest-neighbor classifier of texture via the contrast of local energy histograms of all the wavelet subbands between an input texture patch and each sample texture patch in a given training set. In particular, the contrast is realized with a discrepancy measure which is just a sum of symmetrized Kullback-Leibler divergences between the input and sample local energy histograms on all the wavelet subbands. It is demonstrated by various experiments that our proposed method obtains a satisfactory texture classification accuracy in comparison with several current state-of-the-art texture classification approaches.

For the separation of linear instantaneous mixtures of independent sources, many Independent Component Analysis (ICA) algorithms can learn the separating matrix by optimizing some objective functions derived from various criteria. The Maximum Likelihood (ML) principle, with hypothesized model pdf's, provides an objective function which is commonly used. It is generally considered that the ML approach leads to a separating solution as long as the kurtosis signs of the model pdf's can correspond and equal to those of the sources, respectively, in some order, which is referred to as the one-bit-matching condition. In this letter, we present an experimental analysis on spurious solution of the ML approach and show that spurious maximum of the objective function really exists in certain cases even if the one-bit-matching condition is satisfied.

Automated cell segmentation and tracking are critical for quantitative analysis of cell cycle behavior using time-lapse fluorescence microscopy. However, the complex, dynamic cell cycle behavior poses new challenges to the existing image segmentation and tracking methods. This paper presents a fully automated tracking method for quantitative cell cycle analysis. In the proposed tracking method, we introduce a neighboring graph to characterize the spatial distribution of neighboring nuclei, and a novel dissimilarity measure is designed based on the spatial distribution, nuclei morphological appearance, migration, and intensity information. Then, we employ the integer programming and division matching strategy, together with the novel dissimilarity measure, to track cell nuclei. We applied this new tracking method for the tracking of HeLa cancer cells over several cell cycles, and the validation results showed that the high accuracy for segmentation and tracking at 99.5% and 90.0%, respectively. The tracking method has been implemented in the cell-cycle analysis software package, DCELLIQ, which is freely available.

In this paper, we show that most multiple term queries include more than one topic and users usually reformulate their queries by topics instead of terms. In order to provide empirical evidence on user's reformulation behavior and to help search engines better handle the query reformulation problem, we focus on detecting internal topics in the original query and analyzing users. reformulation to those topics. Particularly, we utilize the Interaction Information (II) to measure the degree of one sub-query being a topic based on the local search results. The experimental results on query log show that: most users reformulate query at the topical level; and our proposed II-based algorithm is a good method to detect topics from original queries.

In this paper, the complex-valued ICA problem is studied in the context of blind complex-source separation. We formulate the complex ICA problem in a general setting, and define the superadditive functional that may be used for constructing a contrast function for circular complex sources separation. We propose several contrast functions and study their properties. Finally, we also discuss relevant issues and present the convex analysis of a specific contrast function.

From the Bayesian Ying-Yang (BYY) harmony learning theory, a harmony function has been developed for finite mixtures with a novel property that its maximization can make model selection automatically during parameter learning. In this paper, we make a theoretical analysis on the harmony function and prove that the global maximization of the harmony function leads to the automated model selection property when there is no or weak overlap between the actual components in the sample data. Moreover, it is proved that the estimates of the parameters through maximizing the harmony function are generally biased, but the deviation error is dominated by the average overlap measure between the actual components in the mixture.

Rival penalized competitive learning (RPCL) has been shown to be a useful tool for clustering on a set of sample data in which the number of clusters is unknown. However, the RPCL algorithm was proposed heuristically and is still in lack of a mathematical theory to describe its convergence behavior. In order to solve the convergence problem, we investigate it via a cost-function approach. By theoretical analysis, we prove that a general form of RPCL, called distance-sensitive RPCL (DSRPCL), is associated with the minimization of a cost function on the weight vectors of a competitive learning network. As a DSRPCL process decreases the cost to a local minimum, a number of weight vectors eventually fall into a hypersphere surrounding the sample data, while the other weight vectors diverge to infinity. Moreover, it is shown by the theoretical analysis and simulation experiments that if the cost reduces into the global minimum, a correct number of weight vectors is automatically selected and located around the centers of the actual clusters, respectively. Finally, we apply the DSRPCL algorithms to unsupervised color image segmentation and classification of the wine data.

Under the Bayesian Ying–Yang (BYY) harmony learning theory, a harmony function has been developed on a BI-directional architecture of the BYY system for Gaussian mixture with an important feature that, via its maximization through a general gradient rule, a model selection can be made automatically during parameter learning on a set of sample data from a Gaussian mixture. This paper further proposes the conjugate and natural gradient rules to efficiently implement the maximization of the harmony function, i.e. the BYY harmony learning, on Gaussian mixture. It is demonstrated by simulation experiments that these two new gradient rules not only work well, but also converge more quickly than the general gradient ones.

The one-bit-matching conjecture for independent component analysis (ICA) has been widely believed in the ICA community. Theoretically, it has been proved that under the assumption of zero skewness for the model probability density functions, the global maximum of a cost function derived from the typical objective function on the ICA problem with the one-bit-matching condition corresponds to a feasible solution of the ICA problem. In this note, we further prove that all the local maximums of the cost function correspond to the feasible solutions of the ICA problem in the two-source case under the same assumption. That is, as long as the one-bit-matching condition is satisfied, the two-source ICA problem can be successfully solved using any local descent algorithm of the typical objective function with the assumption of zero skewness for all the model probability density functions.

Finding informative genes from microarray data is an important research problem in bioinformatics research and applications. Most of the existing methods rank features according to their discriminative capability and then find a subset of discriminative genes (usually top k genes). In particular, t-statistic criterion and its variants have been adopted extensively. This kind of methods rely on the statistics principle of t-test, which requires that the data follows a normal distribution. However, according to our investigation, the normality condition often cannot be met in real data sets.To avoid the assumption of the normality condition, in this paper, we propose a rank sum test method for informative gene discovery. The method uses a rank-sum statistic as the ranking criterion. Moreover, we propose using the significance level threshold, instead of the number of informative genes, as the parameter. The significance level threshold as a parameter carries the quality specification in statistics. We follow the Pitman efficiency theory to show that the rank sum method is more accurate and more robust than the t-statistic method in theory.To verify the effectiveness of the rank sum method, we use support vector machine (SVM) to construct classifiers based on the identified informative genes on two well known data sets, namely colon data and leukemia data. The prediction accuracy reaches 96.2% on the colon data and 100% on the leukemia data. The results are clearly better than those from the previous feature ranking methods. By experiments, we also verify that using significance level threshold is more effective than directly specifying an arbitrary k.

Organizing Web search results into clusters facilitates users' quick browsing through search results. Traditional clustering techniques are inadequate since they don't generate clusters with highly readable names. In this paper, we reformalize the clustering problem as a salient phrase ranking problem. Given a query and the ranked list of documents (typically a list of titles and snippets) returned by a certain Web search engine, our method first extracts and ranks salient phrases as candidate cluster names, based on a regression model learned from human labeled training data. The documents are assigned to relevant salient phrases to form candidate clusters, and the final clusters are generated by merging these candidate clusters. Experimental results verify our method's feasibility and effectiveness.

Gaussian mixture modeling is a powerful approach for data analysis and the determination of the number of Gaussians, or clusters, is actually the problem of Gaussian mixture model selection which has been investigated from several respects. This paper proposes a new kind of automated model selection algorithm for Gaussian mixture modeling via an entropy penalized maximum-likelihood estimation. It is demonstrated by the experiments that the proposed algorithm can make model selection automatically during the parameter estimation, with the mixing proportions of the extra Gaussians attenuating to zero. As compared with the BYY automated model selection algorithms, it converges more stably and accurately as the number of samples becomes large.

We investigate the memory structure and retrieval of the brain and propose a hybrid neural network of addressable and content-addressable memory which is a special database model and can memorize and retrieve any piece of information (a binary pattern) both addressably and content-addressably. The architecture of this hybrid neural network is hierarchical and takes the form of a tree of slabs which consist of binary neurons with the same array. Simplex memory neural networks are considered as the slabs of basic memory units, being distributed on the terminal vertexes of the tree. It is shown by theoretical analysis that the hybrid neural network is able to be constructed with Hebbian and competitive learning rules, and some other important characteristics of its learning and memory behavior are also consistent with those of the brain. Moreover, we demonstrate the hybrid neural network on a set of ten binary numeral patters

This paper presents a new neural network approach to real-time pattern recognition on a given set of binary (or bipolar) sample patterns. The perceptive neuron of a binary pattern is defined and constructed as a binary neuron with a neighborhood perceptive field. Letting its hidden units be the respective perceptive neurons of the patterns, a three-layer forward neural network is constructed to recognize these patterns with minimum error probability in a noisy environment. The theoretical and simulation analyses show that the network is effective for pattern recognition and can be under strict real-time constraints.

It is well known that the convergence rate of the expectation-maximization (EM) algorithm can be faster than those of convention first-order iterative algorithms when the overlap in the given mixture is small. But this argument has not been mathematically proved yet. This article studies this problem asymptotically in the setting of gaussian mixtures under the theoretical framework of Xu and Jordan (1996). It has been proved that the asymptotic convergence rate of the EM algorithm for gaussian mixtures locally around the true solution Theta* is o(e(0. 5-epsilon)(Theta*)), where epsilon > 0 is an arbitrarily small number, o(x) means that it is a higher-order infinitesimal as x --> 0, and e(Theta*) is a measure of the average overlap of gaussians in the mixture. In other words, the large sample local convergence rate for the EM algorithm tends to be asymptotically superlinear when e(Theta*) tends to zero.

In this work, groupwise successive interference cancellation scheme for multirate CDMA signals is proposed. The proposed detector operation is based on the minimum mean square error (MMSE) criterion. It has the promise of decision feedback interference cancellation for superior performance compared to the linear MMSE detection at the cost of a moderate increase in complexity. In groupwise detection, users are grouped according to their processing gain or data rate, i.e., users with the same processing gain or data rate are grouped together, then detected. Users with the lowest processing gain or highest data rate are detected first, neglecting the presence of the other users in the system. Interference between the groups is cancelled in a successive order. The BER of the groupwise successive MMSE interference canceler is compared with the BER multirate linear MMSE detector and multirate decorrelating detector. The results show that the groupwise successive MMSE interference canceler yields better performance than the multirate linear MMSE detector and multirate decorrelating detector. When the system is highly loaded, the groupwise successive MMSE interference canceler can offer a better performance than the multirate linear MMSE detector and the multirate conventional detector.

This work is an extension of the multi-rate LMMSE detectors (Ge 1997) for asynchronous multi-rate CDMA systems. The proposed LMMSE detectors can work on dual-rate mode according to different quality-of-service (QoS) requirements and computational constraints. In using the proposed multirate LMMSE detectors, only the signature and timing information of the desired user are required. The proposed detectors can achieve better detection performance with less computation than the dual-rate decorrelating detectors. Adaptive schemes of implementing the proposed detectors can handle both synchronous and asynchronous channels as long as the signature and timing information on the desired user is available.

A LMMSE multi-rate detector for synchronous CDMA frequency selective Rayleigh fading channels is proposed. A comparative study between the conventional multi-rate RAKE receiver and suboptimal multi-rate detector is presented. The conventional multi-rate RAKE receiver is severely limited by the near-far effects in the multiuser environment. Both suboptimal multi-rate receiver and LMMSE multi-rate detector have the near-far resistance while preserving the multipath diversity gain. The simulation results show that the proposed LMMSE multi-rate detector can improve the performance compared to the proposed suboptimal multirate receiver.