With advances in sensing technology, large volumes of spatio-temporal data are collected in applications such as imaging, video, and traffic monitoring. These datasets often contain outliers caused by sensor malfunctions or anomalous activity. While such outliers are commonly modeled as sparse noise, they frequently exhibit structure, resulting in grouped anomalies that cannot be fully captured by ℓ 1 -norm sparsity. We introduce a robust tensor decomposition method in which the normal component is modeled as low-rank, and the anomalous component is structured through topological regularizers. Specifically, we employ two classes of regularizers: latent overlapping group norms to capture anomaly contiguity, and graph total variation to enforce smoothness. The framework incorporates geometric information via node- and edge-based groupings derived from the underlying graph of the spatio-temporal data. The resulting optimization problem is solved using two-block ADMM, and the method is evaluated on both simulated and real spatio-temporal datasets, demonstrating its ability to detect structured anomalies effectively.

Anomaly detection in spatiotemporal data is a challenging problem encountered in a variety of applications including urban traffic monitoring. Existing anomaly detection methods mostly focus on point anomalies and cannot deal with temporal and spatial dependencies that arise in spatiotemporal data. Tensor-based anomaly detection methods have been proposed to address this problem. While these methods are able to capture the dependencies across the different modes, they are mostly supervised and do not take the particular nature of anomalies into account. In this paper, we introduce an unsupervised tensor-based anomaly detection method that simultaneously considers the sparse and spatiotemporally smooth nature of anomalies. The anomaly detection problem is formulated as a regularized robust low-rank + sparse tensor decomposition where the spatiotemporal smoothness of the anomalies is quantified by the graph total variation with respect to the underlying spatial and temporal graphs. This minimization ensures that the extracted anomalies are temporally persistent and spatially smooth. The proposed framework is evaluated on both synthetic and real spatiotemporal urban traffic data.

Subgroup identification is a fundamental step in precision medicine. Recent research applying data-driven methods such as independent component/vector analysis to multi-subject functional magnetic resonance imaging (fMRI) data has effectively revealed meaningful subgroups. These methods typically focus on single-dimensional information, such as individual functional networks or assuming uniform subgroup structures across networks. Given the complex nature of psychiatric disorders, considering the relationships among subjects across different functional networks can offer valuable insights into diagnostic heterogeneity. We introduce a novel subgroup identification method that leverages multiplex community detection to identify subgroups from multi-subject resting-state fMRI data. The proposed method models subject correlations across functional networks as a multiplex network and identifies common communities across multiple networks and unique communities specific to each functional network. Results from applying the proposed method to 464 psychotic patients show that the identified subgroups exhibit significant group differences on multiple meaningful functional networks as well as the clinical scores, which demonstrate the effectiveness of our method on identifying meaningful subgroups.

In recent work , we developed a fast, memory-efficient, and sample-efficient solution to the Low Rank column-wise Compressive Sensing (LRcCS) problem: recover an n × q LR matrix from m under-sampled linear projections of each of its columns. Here, undersampled means m ≪ n,q . The matrix LR model and the corresponding algorithms have two important limitations. First, for real image sequences, the required memory complexity is prohibitive. Secondly, for image or volume image sequences, it requires vectorizing the image or volume as one column of a matrix and this ignores the inherent 2D or 3D structure of the images or volumes. To address these limitations, in this work, we explore the use of a tensor LR model on the image sequence along with developing a fast and memory-efficient gradient descent (GD) based recovery algorithm and evaluating it experimentally.

An important problem in graph signal processing (GSP) is to infer the topology of an unknown graph from a set of observations on the nodes of the graph, i.e. graph signals. Recently, graph learning (GL) approaches have been extended to learn dynamic graphs from temporal graph signals. However, existing work primarily focuses on unsigned graphs and cannot learn signed graphs, which are important data structures that can represent the similarity and dissimilarity of the nodes. In this paper, we propose a dynamic signed GL (dynSGL) method based on the assumptions that (i) at each time point signals are smooth with respect to the signed graph, i.e. signal values at two nodes connected with a positive (negative) edge are similar (dissimilar) and (ii) evolution of the graph structures is smooth across time. The performance of dynSGL is evaluated on simulated data and shown to have higher accuracy compared to static signed and dynamic unsigned GL techniques. Application of the proposed method to a financial dataset gives important insights to the time-varying changes to the interactions between stocks.

Many real-world data are represented through the relations between data samples, i.e., a graph structure. Although many datasets come with a pre-existing graph, there is still a large number of applications where the graph structure is not readily available. An essential task for such cases is graph learning (GL), which infers the graph structure from a set of graph signals. Existing GL techniques mostly focus on learning a single graph structure; however, samples are usually connected in multiple different ways. Furthermore, existing works can only handle unsigned graphs, while contemporary tasks require inference of signed graphs, which are better at representing similarity and dissimilarity of samples. In this paper, we propose a framework (mvSGL) for joint estimation of multiple related signed graphs. mvSGL optimizes the total variation of graph signals with respect to graphs while ensuring that the graphs are similar to each other through a consensus graph. mvSGL is employed in the inference of multiple gene regulatory networks (GRN) from single cell datasets that include multiple cell types. Performance evaluation using simulated and real datasets demonstrates the effectiveness of mvSGL in the inference of multiple related GRNs.

BACKGROUND: Characterizing the topology of gene regulatory networks (GRNs) is a fundamental problem in systems biology. The advent of single cell technologies has made it possible to construct GRNs at finer resolutions than bulk and microarray datasets. However, cellular heterogeneity and sparsity of the single cell datasets render void the application of regular Gaussian assumptions for constructing GRNs. Additionally, most GRN reconstruction approaches estimate a single network for the entire data. This could cause potential loss of information when single cell datasets are generated from multiple treatment conditions/disease states. RESULTS: To better characterize single cell GRNs under different but related conditions, we propose the joint estimation of multiple networks using multiple signed graph learning (scMSGL). The proposed method is based on recently developed graph signal processing (GSP) based graph learning, where GRNs and gene expressions are modeled as signed graphs and graph signals, respectively. scMSGL learns multiple GRNs by optimizing the total variation of gene expressions with respect to GRNs while ensuring that the learned GRNs are similar to each other through regularization with respect to a learned signed consensus graph. We further kernelize scMSGL with the kernel selected to suit the structure of single cell data. CONCLUSIONS: scMSGL is shown to have superior performance over existing state of the art methods in GRN recovery on simulated datasets. Furthermore, scMSGL successfully identifies well-established regulators in a mouse embryonic stem cell differentiation study and a cancer clinical study of medulloblastoma.

Networks provide a powerful tool to model complex systems. Recently, there has been a growing interest in multiplex networks as they can represent the interactions between a pair of nodes through multiple types of links, each reflecting a distinct type of interaction. One of the important tools in understanding network topology is community detection. Existing work on multiplex community detection mostly focuses on learning a common community structure across layers without taking the heterogeneity of the different layers into account. In this paper, we introduce a new multiplex community detection approach that can identify communities that are common across layers as well as those that are unique to each layer. The proposed algorithm employs Orthogonal Non-Negative Matrix Tri-Factorization to model each layer’s adjacency matrix as the sum of two low-rank matrix factorizations, corresponding to the common and private communities, respectively. The proposed algorithm is evaluated on both synthetic and real multiplex networks and compared to state-of-the-art techniques.

Graph learning (GL) aims to infer the topology of an unknown graph from a set of observations on its nodes, i.e., graph signals. While most of the existing GL approaches focus on homogeneous datasets, in many real world applications, data is heterogeneous, where graph signals are clustered and each cluster is associated with a different graph. In this paper, we address the problem of learning multiple graphs from heterogeneous data by formulating an optimization problem for joint graph signal clustering and graph topology inference. In particular, our approach extends spectral clustering by partitioning the graph signals not only based on their pairwise similarities but also their smoothness with respect to the graphs associated with the clusters. The proposed method also learns the representative graph for each cluster using the smoothness of the graph signals with respect to the graph topology. The resulting optimization problem is solved with an efficient block-coordinate descent algorithm and results on simulated and real data indicate the effectiveness of the proposed method.

MOTIVATION: Elucidating the topology of gene regulatory networks (GRNs) from large single-cell RNA sequencing datasets, while effectively capturing its inherent cell-cycle heterogeneity and dropouts, is currently one of the most pressing problems in computational systems biology. Recently, graph learning (GL) approaches based on graph signal processing have been developed to infer graph topology from signals defined on graphs. However, existing GL methods are not suitable for learning signed graphs, a characteristic feature of GRNs, which are capable of accounting for both activating and inhibitory relationships in the gene network. They are also incapable of handling high proportion of zero values present in the single cell datasets. RESULTS: To this end, we propose a novel signed GL approach, scSGL, that learns GRNs based on the assumption of smoothness and non-smoothness of gene expressions over activating and inhibitory edges, respectively. scSGL is then extended with kernels to account for non-linearity of co-expression and for effective handling of highly occurring zero values. The proposed approach is formulated as a non-convex optimization problem and solved using an efficient ADMM framework. Performance assessment using simulated datasets demonstrates the superior performance of kernelized scSGL over existing state of the art methods in GRN recovery. The performance of scSGL is further investigated using human and mouse embryonic datasets. AVAILABILITY AND IMPLEMENTATION: The scSGL code and analysis scripts are available on https://github.com/Single-Cell-Graph-Learning/scSGL. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

With the advance of sensor technology, it is becoming more commonplace to collect multi-mode data, i.e., tensors, with high dimensionality. To deal with the large amounts of redundancy in tensorial data, different dimensionality reduction methods such as low-rank tensor decomposition have been developed. While low-rank decompositions capture the global structure, there is a need to leverage the underlying local geometry through manifold learning methods. Manifold learning methods have been widely considered in tensor factorization to incorporate the low-dimensional geometry of the underlying data. However, existing techniques focus on only one mode of the data and exploit correlations among the features to reduce the dimension of the feature vectors. Recently, multiway graph signal processing approaches that exploit the correlations among all modes of a tensor have been proposed to learn low-dimensional representations. Inspired by this idea, in this letter we propose a graph regularized robust tensor-train decomposition method where the graph regularization is applied across each mode of the tensor to incorporate the local geometry. As the resulting optimization problem is computationally prohibitive due to the high dimensionality of the graph regularization terms, an equivalence between mode-$n$canonical unfolding and regular mode-$n$unfolding is derived resulting in a computationally efficient optimization algorithm. The proposed method is evaluated on both synthetic and real tensors for denoising and tensor completion.

Phase-amplitude coupling (PAC), which quantifies the coupling between the amplitude of a fast oscillation and the phase of a slow oscillation, is reported as a possible mechanism that controls the flow of information in the brain. Although there is ample evidence suggesting that neural interactions are directional, conventional PAC measures mostly quantify the cross-frequency coupling, failing to provide information on the direction of interactions. In this paper, we introduce a Granger causality (GC) based approach to estimate the direction of PAC. This approach infers the directionality of cross-frequency coupling by computing GC between the instantaneous phase and amplitude components extracted from the signal through a complex time-frequency (t-f) distribution, known as the Reduced Interference Distribution (RID)-Rihaczek. The method is evaluated on both simulated and real electroencephalogram (EEG) signals. The results demonstrate that the proposed GC based directional PAC measure can infer the direction of neural interactions across frequency bands.

Anomaly detection in spatiotemporal data is a challenging problem encountered in a variety of applications including hyperspectral imaging, video surveillance, and urban traffic monitoring. Existing anomaly detection methods are most suited for point anomalies in sequence data and cannot deal with temporal and spatial dependencies that arise in spatiotemporal data. In recent years, tensor-based methods have been proposed for anomaly detection to address this problem. These methods rely on conventional tensor decomposition models, not taking the structure of the anomalies into account, and are supervised or semi-supervised. We introduce an unsupervised tensor-based anomaly detection method that takes the sparse and temporally continuous nature of anomalies into account. In particular, the anomaly detection problem is formulated as a robust low-rank + sparse tensor decomposition with a regularization term that minimizes the temporal variation of the sparse part, so that the extracted anomalies are temporally persistent. We also approximate rank minimization with graph total variation minimization to reduce the complexity of the optimization algorithm. The resulting optimization problem is convex, scalable, and is shown to be robust against missing data and noise. The proposed framework is evaluated on both synthetic and real spatiotemporal urban traffic data and compared with baseline methods.

Higher-order data with high dimensionality arise in a diverse set of application areas such as computer vision, video analytics and medical imaging. Tensors provide a natural tool for representing these types of data. Two major challenges that confound current tensor based supervised learning algorithms are storage complexity and computational efficiency. In this paper, we address these problems by introducing a multi-branch tensor network structure. The multi-branch structure is a general tensor decomposition that includes Tucker and tensor-train (TT) as special cases and takes advantage of the flexibility of the tensor network to provide a better balance between storage and computational complexity. We then introduce a supervised discriminative tensor-train subspace learning approach referred to as tensor-train discriminant analysis (TTDA), and its implementations using the multi-branch tensor network structure. Multi-branch implementations of TTDA are shown to achieve lower storage and computational complexity while providing improved classification performance with respect to both Tucker and TT based supervised learning methods.

Networks are useful representations of many systems with interacting entities, such as social, biological and physical systems. Characterizing the meso-scale organization, i.e. the community structure, is an important problem in network science. Community detection aims to partition the network into sets of nodes that are densely connected internally but sparsely connected to other dense sets of nodes. Current work on community detection mostly focuses on static networks. However, many real world networks are dynamic, i.e. their structure and properties change with time, requiring methods for dynamic community detection. In this paper, we propose a new stochastic block model (SBM) for modeling the evolution of community membership. Unlike existing SBMs, the proposed model allows each community to evolve at a different rate. This new model is used to derive a maximum a posteriori estimator for community detection, which can be written as a constrained spectral clustering problem. In particular, the transition probabilities for each community modify the graph adjacency matrix at each time point. This formulation provides a relationship between statistical network inference and spectral clustering for dynamic networks. The proposed method is evaluated on both simulated and real dynamic networks.

Long-distance neuronal communication in the brain is enabled by the interactions across various oscillatory frequencies. One interaction that is gaining importance during cognitive brain functions is phase amplitude coupling (PAC), where the phase of a slow oscillation modulates the amplitude of a fast oscillation. Current techniques for calculating PAC provide a numerical index that represents an average value across a pre-determined time window. However, there is growing empirical evidence that PAC is dynamic, varying across time. Current approaches to quantify time-varying PAC relies on computing PAC over sliding short time windows. This approach suffers from the arbitrary selection of the window length and does not adapt to the signal dynamics. In this paper, we introduce a data-driven approach to quantify dynamic PAC. The proposed approach relies on decomposing the signal using matching pursuit (MP) to extract time and frequency localized atoms that best describe the given signal. These atoms are then used to compute PAC across time and frequency. As the atoms are time and frequency localized, we only compute PAC across time and frequency regions determined by the selected atoms rather than the whole time-frequency range. The proposed approach is evaluated on both simulated and real electroencephalogram (EEG) signals.

With the advances in data acquisition technology, tensor objects are collected in a variety of applications including multimedia, medical and hyperspectral imaging. As the dimensionality of tensor objects is usually very high, dimensionality reduction is an important problem. Most of the current tensor dimensionality reduction methods rely on finding low-rank linear representations using different generative models. However, it is well-known that high-dimensional data often reside in a low-dimensional manifold. Therefore, it is important to find a compact representation, which uncovers the low-dimensional tensor structure while respecting the intrinsic geometry. In this paper, we propose a graph regularized tensor train (GRTT) decomposition that learns a low-rank tensor train model that preserves the local relationships between tensor samples. The proposed method is formulated as a non-convex optimization problem on the Stiefel manifold and an efficient algorithm is proposed to solve it. The proposed method is compared to existing tensor based dimensionality reduction methods as well as tensor manifold embedding methods for unsupervised learning applications.

Many real-world systems can be represented by networks. One common approach to characterizing the organization of networks is community detection. A lot of work has been conducted in community detection of static networks. However, most real systems are time-dependent and modeled by temporal networks with a structure that evolves across time. In this paper, a low-rank approximation based evolutionary clustering approach is introduced to detect and track the community structure of temporal networks. The proposed approach provides robustness to outliers and results in smoothly evolving cluster assignments through joint low-rank approximation and subspace learning. Moreover, a cost function is introduced to track changes in the community structure across time. The performance of the proposed approach is validated on both simulated and real temporal networks.

Interaction of neuronal oscillations across different frequency bands plays an important role in perception, attention, and memory. One particular form of interaction is the modulation of the amplitude of high-frequency oscillations by the phase of low-frequency oscillations, known as phase-amplitude coupling (PAC). Current methods for quantifying PAC mostly rely on Hilbert transform which assumes that brain activity is stationary and narrowband. Moreover, these methods are limited to quantifying bivariate PAC and cannot capture multivariate cross-frequency coupling between different brain regions. This paper presents a new complex time-frequency based high resolution PAC measure and its extension to the multivariate case using PARAFAC (Parallel Factor) model. The proposed approach is evaluated on both simulated and real electroencephalogram (EEG) data.

The rapid development of information technology is making it possible to collect massive amounts of multidimensional, multimodal data with high dimensionality in a diverse set of science and engineering disciplines. Although there has been a lot of recent work in the area of unsupervised tensor learning, extensions to supervised learning, feature extraction and classification are still limited. Moreover, most of the existing supervised tensor learning approaches are based on the Tucker model. However, this model has some limitations for large tensors including high memory and execution time costs. In this paper, we introduce a supervised learning approach for tensor classification based on the tensor-train model. In particular, we introduce two computationally efficient implementations of tensor-train discriminant analysis (TT-DA). The proposed approaches are evaluated on image classification tasks with respect to computation time, storage cost and classification accuracy.

Oscillatory phenomena and linear/nonlinear interactions across different oscillatory frequencies are commonly seen between the elements of complex systems. One particular form of interaction that is widely encountered is the modulation of the amplitude of broadband high-frequency oscillations by the phase of low-frequency oscillations, defined as phase-amplitude coupling (PAC). Conventional methods for assessing PAC mostly rely on the Hilbert transform. Previous studies have shown that this method produces biased or spurious PAC estimates as it is highly dependent on the parameters of the bandpass filter used to extract narrowband signals from broadband signals. Recently, wavelet-based methods have been proposed to address some of these limitations. In this letter, we provide a thorough comparison between wavelet-based PAC measures and our recently introduced Reduced Interference Rihaczek (RID-Rihaczek) distribution based PAC both analytically and through simulations. The proposed method is shown to yield higher frequency resolution PAC estimates compared to wavelet-based methods.

Recent work on resting-state functional magnetic resonance imaging (rs-fMRI) suggests that functional connectivity (FC) is dynamic. A variety of machine learning and signal processing tools have been applied to the study of dynamic functional connectivity networks (dFCNs) of the brain, by identifying a small number of network states that describe the dynamics of connectivity during rest. Recently, deep learning (DL) methods have been applied to neuroimaging data for learning generative models. In this paper, we employ the restricted Boltzmann machine (RBM), to learn FC states from resting-state dFCNs. Unlike previous applications of DL to neuroimaging data that focus on feature extraction based on the voxel level activation data, the current work employs RBM to learn connectivity patterns, where the input to RBMs are a collection of windowed covariances across time and subjects. The extracted FC states are evaluated based on their occurrence rate as well as modularity.

The widespread use of multisensor technology and the emergence of big data sets have brought the necessity to develop more versatile tools to represent higher order data with multiple aspects and high dimensionality. Data in the form of multidimensional arrays, also referred to as tensors, arise in a variety of applications including chemometrics, hyperspectral imaging, high-resolution videos, neuroimaging, biometrics, and social network analysis. Early multiway data analysis approaches reformatted such tensor data as large vectors or matrices and then resorted to dimensionality reduction methods developed for classical two-way analysis such as principal component analysis (PCA). However, one cannot discover hidden components within multiway data using conventional PCA. To this end, tensor decomposition methods which are flexible in the choice of the constraints and that extract more general latent components have been proposed. In this paper, we review the major tensor decomposition methods with a focus on problems targeted by classical PCA. In particular, we present tensor methods that aim to solve three important challenges typically addressed by PCA: dimensionality reduction, i.e., low-rank tensor approximation; supervised learning, i.e., learning linear subspaces for feature extraction; and robust low-rank tensor recovery. We also provide experimental results to compare different tensor models for both dimensionality reduction and supervised learning applications.

Many systems from human brain to the networks on social media, can be modeled as graphs. The network structure helps us understand, predict and optimize the behavior of dynamical systems. One of the important tools in understanding network topology is community detection. Even though community detection methods are well developed for static networks, the extensions to the dynamic case are more limited. In this paper, we introduce two tensor based frameworks, windowed and running time, for identifying and tracking the network community structure across time. The frameworks take the history of the networks into account. The proposed approach relies on determining the subspace across time using the Tucker decomposition of a tensor constructed from networks across time. We also propose a computationally efficient way to update the subspace estimates across time to track changes in community structure. The proposed approach is evaluated on both simulated and real dynamic networks.

Event-related potentials (ERP)s are electrophysiological responses that are commonly used for detecting the brain response to external stimuli. In this paper, we propose to use the sparse common component and innovations model (SCCI) to extract ERPs from multiple EEG signals recorded across closely located electrodes. This model finds the sparse representation of the common component of the signals and their innovation components with respect to pre-determined common and innovation dictionaries, where the common component refer to an event captured by adjacent electrodes such as ERPs. However, different stimuli may produce different responses and predetermining the dictionary may not always be optimal. Therefore, we introduce a structured dictionary learning method to simultaneously learn the two dictionaries from training data. The proposed method is applied to a study of error monitoring where two different types of brain responses are elicited corresponding to the decision made by the subject. The learned dictionaries can discriminate between the response types and extract the ERP corresponding to the two responses.

Advances in information technology are making it possible to collect increasingly massive amounts of multidimensional, multi-modal neuroimaging data such as functional magnetic resonance imaging (fMRI). Current fMRI datasets involve multiple variables including multiple subjects, as well as both temporal and spatial data. These high dimensional datasets pose a challenge to the signal processing community to develop data reduction methods that can exploit their rich structure and extract meaningful summarizations. In this paper, we propose a tensor-based framework for data reduction and low-dimensional structure learning with a particular focus on reducing high dimensional fMRI data sets into physiologically meaningful network components. We develop a multiscale tensor factorization method for higher order data inspired by hybrid linear modeling and subspace clustering techniques. In particular, we develop a multi-scale HoSVD approach where a given tensor is first permuted and then partitioned into several sub-tensors each of which can be represented more efficiently. This multi-scale framework is applied to resting state fMRI data to identify the default mode network from compressed data.

Graph signal processing extends the notion of frequency from signals in the time domain to signals defined on graphs. Graph signals arise in many applications including brain signals defined on functional connectivity networks. Most of the current work on graph signal processing focuses on static graphs. However, functional connectivity networks are dynamic and the signals on these networks change with time. In this paper, we introduce a new transform for dynamic networks named as Dynamic Graph Fourier Transform (DGFT). The proposed approach extends the notion of graph Laplacian from the static case to the dynamic case through the network Laplacian tensor. The basis functions for the transform are obtained through the Tucker decomposition of this Laplacian tensor. The proposed method detects nonstationary activity in the network structure and allows us to obtain information about the regions in the brain that contribute to different frequency contents in a cognitive control experiment.

Electric discharge machining current causes a significant amount of damage to bearings, producing pits in the rotating elements of the bearing, and ultimately leading to bearing failure. Although this relationship is well known and studied, little work has been done to relate bearing current discharge events to bearing vibrations for bearing failure prognosis. This paper proposes both experimental and computational approaches for remaining useful life (RUL) estimation of bearings by taking advantage of the relationship between current discharge events and the vibration signal. A test bed, which induces accelerated aging via applied electrical stress, is introduced to better understand the relationship between bearing currents, vibrations, and failure. Over the course of the experiments, multiple sensor data are collected from start to failure in order to correlate current data as well as vibration data to bearing failure. Finally, a computational framework that determines critical events from the current discharge events and uses the timing of these events to estimate RUL from vibration data is proposed.

Recent developments In Information technology have enabled us to collect and analyze high dimensional and higher order data such as tensors. High dimensional data usually lies in a lower dimensional subspace and identifying this low-dimensional structure is important in many signal and information processing applications. Traditional subspace estimation approaches have been limited to vector-type data and cannot effectively deal with these high order datasets. Moreover, most of the existing methods are batch algorithms which can't handle streaming data. In this paper, we propose a new tensor subspace tracking approach to identify changes in dynamic networks. The proposed approach recursively estimates low-rank subspace of higher order data and decomposes it into low-rank and sparse components. The proposed approach is evaluated on both simulated and real dynamic networks.

Functional connectivity brain networks have been shown to demonstrate interesting complex network behavior such as small-worldness. Transforming networks to time series has provided an alternative way of characterizing the structure of complex networks. However, previously proposed deterministic methods are limited to unweighted graphs. In this paper, we propose to employ the resistance distance matrix of weighted graphs as the distance matrix for transforming networks to signals based on classical multidimensional scaling. We present a framework for obtaining information about the network's structure through the mapped signals and recovering the original network using properties of the resistance matrix. Finally, the proposed method is applied to characterizing functional connectivity networks constructed from electroencephalogram data.

With the advances in high resolution neuroimaging, there has been a growing interest in the detection of functional brain connectivity. Complex network theory has been proposed as an attractive mathematical representation of functional brain networks. However, most of the current studies of functional brain networks have focused on the computation of graph theoretic indices for static networks, i.e. long-time averages of connectivity networks. It is well-known that functional connectivity is a dynamic process and the construction and reorganization of the networks is key to understanding human cognition. Therefore, there is a growing need to track dynamic functional brain networks and identify time intervals over which the network is quasi-stationary. In this paper, we present a tensor decomposition based method to identify temporally invariant `network states' and find a common topographic representation for each state. The proposed methods are applied to electroencephalogram (EEG) data during the study of error-related negativity (ERN).

In recent years, there has been a growing interest in diagnosis and prognosis of motors and electrical drives. Effective and accurate prognosis and diagnosis of systems will eventually lead to condition based maintenance, which will decrease maintenance costs and system downtime. Much work has been done in diagnosing the state of a motor, however prediction of the health state at future times, and ultimately the prediction of the system's remaining useful life (RUL), still proves to be a challenge. One of the challenges to efficient prognosis is that in many applications, there is no labeled training data and the different health states of the system are not known a priori. In this paper, we propose an approach for learning the hidden health states of a bearing from vibration signals. The proposed approach is based on extracting multiple features from sensor signals and identifying change points in the state of the system based on these features.

Phase synchronization has been proposed as a plausible mechanism to quantify both linear and nonlinear relationships between neuronal populations and to assess functional brain connectivity. However, bivariate phase synchrony is not sufficient for complex system analysis such as the brain where the bivariate relationships do not always reflect the underlying network structure. Recently, multivariate extensions of bivariate phase synchrony has been of interest in investigating the interactions within a group of oscillators. Current extensions are based on either averaging all possible pairwise synchrony values or eigen decomposition of a matrix of bivariate synchronization indices to estimate multivariate synchrony using the entropy of the normalized eigenvalues. All of these approaches are sensitive to the accuracy of the bivariate synchrony indices, cause loss of information, computationally complex and are indirect ways to quantify the multivariate synchrony. In this paper, we propose a novel and direct measure to estimate the multivariate phase synchrony by forming direction vectors in a multidimensional hyperspherical coordinate system. The proposed method is evaluated through application to electroencephalogram (EEG) data containing error-related negativity (ERN) related to cognitive control. We compare the new measure with existing methods and show its effectiveness in quantifying multivariate synchronization of different brain regions.

Human brain is known to be one of the most complex biological systems and understanding the functional connectivity patterns to distinguish between normal and disrupted brain behavior still remains a challenge. Previous studies focus on analyzing functional connectivity averaged over a certain time and frequency window which is generally not sufficient to address the time-varying evolution of the connectivity patterns. In this paper, we propose a framework to describe the dynamic properties of functional connectivity in the brain. The proposed approach is based on constructing time-varying connectivity graphs from multichannel electroencephalogram (EEG) data, using subspace analysis to detect network-wide changes, identifying key event intervals and then extracting representative networks that describe the connectivity in each event interval. This framework is evaluated for EEG data, containing error-related negativity (ERN) component related to cognitive control. For each time interval, the statistically significant connectivity patterns are presented to illustrate the dynamic nature of functional connectivity.

The development of portable devices that make the reliable diagnosis of faults in electric motors possible has become a challenge for many researchers and maintenance enterprises. These machines intervene in a huge amount of processes and applications and their eventual failure may imply important costs in terms of time and money. However, the aforementioned issue remains unsolved because most of the developed fault diagnosis techniques rely on the user expertise, since they are based on a qualitative interpretation of the results. This complicates the implementation of these methodologies in condition monitoring systems or devices. The objective of this paper is to propose an integral methodology that is able to diagnose the presence of rotor bar failures in an automatic way. The proposed algorithm combines the Discrete Wavelet Transform with the scale transform for feature extraction and correlation coefficient for pattern recognition. The algorithm is applied to both small and large motors operating in a wide range of conditions. The results illustrate the validity and generality of the approach for automatic condition monitoring of electric motors.

Determining the functional relationships between nodes in complex networks such as the neuronal networks is important. In recent years, graph theory has been employed to characterize the functional net work structure of the brain from neurophysiological data such as the electroencephalogram (EEG). Current work on graph theoretic analysis of brain networks focuses on global characteristics of the network such as small world network measures. However, it is as important to be able to extract local features of the graph and quantify the vulnerability and robustness of different brain regions. In this paper, we explore how a well-known measure in signal processing, energy, can be extended toward understanding the functional role of neural assemblies in the brain network as represented by a graph. For this purpose, we introduce the Laplacian-Huckel Energy to quantify the local contribution of the nodes to the organization of any scale-free graph and determine anomalies in the graph. The pro posed measure is evaluated for both the well-known Zachery karate network and a brain network constructed from an electroencephalogram study.

Functional connectivity has been widely used to reveal the dependencies between signals in complex networks such as neural networks observed from electroencephalogram (EEG) data. The interactions among neural oscillations are known to be nonlinear and non-stationary. Classical measures for quantifying these interactions only capture the linear relationships, are mostly defined in either the time or frequency domain, and are limited to pairwise relationships. In this paper, we propose a multi-scale multi-information measure to quantify the interdependencies among multiple variables in both time and frequency domains. Multivariate empirical mode decomposition (MEMD) is employed to decompose signals into different frequency bands and multi-information is used to quantify the dependencies between these signals across time and frequency. The proposed measure is applied to both simulated data and EEG data to evaluate its effectiveness.

Directed Information (DI) has recently been introduced to quantify the causality between two signals. However, one major remaining issue with DI is the computational complexity which increases dramatically with the length of the signal. Current simplified DI computation methods are either model dependent or focus on short-time intervals losing most of the causal dependencies. In this paper, we introduce the time-lagged DI to reduce the computational complexity while still quantifying the causal relationships.

The concepts of modulation frequency along with modulation spectra are originally encountered in acoustics, speech and audio processing. The modulation spectrum, a function of acoustic frequency and modulation frequency, has been proposed and widely used in the speech processing community. However, modulation spectrum, much like time-frequency distribution is a representation of an individual signal and does not quantify the modulation effects between two signals. In this paper, we introduce cross frequency-spectral lag representation based on the Wigner distribution to represent the modulation relationships between two signals. The performance of the proposed distribution is illustrated for simulated signals as well as for electroencephalogram (EEG) signals.

One particular challenge in the study of the brain as a complex system is the identification of dynamic functional networks underlying observed neural activity. In this study, we focus on inferring the functional connectivity of the brain and the underlying network patterns from electroencephalography (EEG) data. The interactions between the different neuronal populations are quantified through a dynamic measure of phase synchrony. These interactions are then analyzed by applying a graph clustering algorithm known as the Cluster-Overlap Newman Girvan Algorithm (CONGA) and generating a three dimensional model relating modularity, degree, and number of clusters. The importance of each electrode in forming clusters is quantified using a `participation score' and an optimal clustering arrangement is found with respect to the degree, number of clusters and the `participation score'. The proposed measures are applied to an EEG study containing the error-related negativity (ERN) to determine the organization of the brain during a decision making task.

The concept of mutual information (MI) has been widely used for inferring complex networks such as genetic regulatory networks. However, the MI based methods cannot infer directed or dynamic networks. In this paper, we propose a new network inference algorithm to infer directed acyclic networks which can determine both the connectivity and causality between different nodes based on the concept of directed information (DI) and conditional directed information. The proposed method is applied to both simulated data and Electroencephalography (EEG) data to evaluate its effectiveness.

The quantification of phase synchrony is important for the study of large-scale interactions in the brain. Current methods for computing phase synchrony are limited to the estimation of the stability of the phase difference between pairs of signals over a time window, within successive frequency bands. These approaches cannot quantify the synchrony across a group of electrodes and over time-varying frequency regions from multiple trials. In this paper, we address this issue by quantifying the frequency locking between groups of electrodes using a time-frequency based estimation of the instantaneous frequency. The instantaneous frequency maps of individual electrodes are combined to obtain the instantaneous frequency histogram as an estimate of the amount of frequency locking across electrodes. This analysis is then extended to the estimation of frequency locking across multiple electrodes and trials. Results are shown for both synthetic signal models and electroencephalogram (EEG) data collected from control and schizophrenic subjects.

Human brain activity can be measured with high temporal resolution by recording the electric potentials on the scalp surface using imaging methods such as the electroencephalogram (EEG). The analysis of EEG data is difficult due to the fact that multiple neurons may be simultaneously active and the potentials from these sources are superimposed on the limited sensors. It is desirable to unmix the data into signals representing the behavior of the original individual neurons. This is a problem of underdetermined blind source separation (UBSS). Since EEG signals are non-stationary, in this paper a two-stage UBSS approach is proposed for the separation of EEG signals by taking advantage of the high resolution of time-frequency distributions. Experimental results indicate the effectiveness of the introduced approach at separating EEG signals in the time-frequency domain compared with independent component analysis (ICA).

Two directed information measures have been used extensively in literature to evaluate the direction of information flow between two time series. The relationship between these two measures, however, has not been established as to date. In this letter, we derive a formula that defines the relationship between these two measures. We also offer some insights into the interpretation of these measures and their relationship, which are verified through simulations.

Energy distribution over wavelet subbands is a widely used feature for wavelet packet based texture classification. Due to the overcomplete nature of the wavelet packet decomposition, feature selection is usually applied for a better classification accuracy and a compact feature representation. The majority of wavelet feature selection algorithms conduct feature selection based on the evaluation of each subband separately, which implicitly assumes that the wavelet features from different subbands are independent. In this paper, the dependence between features from different subbands is investigated theoretically and simulated for a given image model. Based on the analysis and simulation, a wavelet feature selection algorithm based on statistical dependence is proposed. This algorithm is further improved by combining the dependence between wavelet feature and the evaluation of individual feature component. Experimental results show the effectiveness of the proposed algorithms in incorporating dependence into wavelet feature selection.

The functional integration between the different parts of the brain is usually quantified through a measure of coherence. Most of the existing measures define coherence based on the spectral energy distribution of the signals rather than the phase, and therefore cannot be reliably used as measures of neural synchrony. Moreover, the most common methods for quantifying coherence are formulated in the frequency domain and thus, do not take into account the time-varying nature of brain activity. Recently, coherence measures have been extended to account for the energy and the phase relationships between the given signals and the time-varying nature of the signals using the wavelet transform. In this paper, we extend this idea by introducing a new time-varying phase coherence measure based on Cohen's class of time-frequency distributions. This new measure is applied to both synthesized signals and electroencephalogram (EEG) data to show the effectiveness of the proposed measure in estimating phase changes and in quantifying the neural synchrony in the brain.

Underdetermined blind source separation (UBSS) is a challenging problem that has recently been formulated in the time-frequency domain. Previous work in the area of UBSS problem focuses on using sparse representations of signals, such as matching pursuit and wavelet packet decomposition, for identifying the sources. However, these methods are in general computationally expensive and rely on the choice of an appropriate basis function for obtaining a sparse representation. In this paper, we propose a new approach based on Cohen's class of distributions. The new approach takes advantage of the high resolution of time-frequency distributions for obtaining a sparse representation, and separates the sources by a simple clustering algorithm followed by a convex optimization problem. Compared to other time-frequency based separation methods, the presented approach is characterized by its simplicity and ease of implementation. Experimental results indicate the effectiveness of the proposed approach at separating the sparse signals in the time-frequency domain.

Blind source separation aims at recovering the original source signals given only observations of their mixtures. Some common approaches to the source separation problem include second or higher order statistics based methods, and independent component analysis. Most of these methods are developed in the time domain, and thus, inherently assume the stationarity of the underlying signals. Since most real life signals of interest are non–stationary, there have been efforts to perform source separation in the time–frequency domain. In this paper, we propose a new approach for source separation on the time–frequency plane using an information–theoretic cost function. Jensen–Rényi divergence, as adapted to time–frequency distributions, is introduced as an effective cost function to extract sources that are disjoint on the time–frequency plane. The sources are extracted through a series of Givens rotations and the optimal rotation angle is found using the steepest descent algorithm. The performance of the proposed method is illustrated and quantified through examples.

The features extracted from traditional wavelet transform have been successfully applied to texture classification. However, most wavelet features are not invariant to image rotation. This paper proposes a new rotation invariant feature based on the combination of ridgelet, a directional non-separable wavelet transform, and Fourier transforms. The ridgelet transform is applied to the rotated image, transforming the rotation angle to shifts in the ridgelet domain. Changes caused by the shift is eliminated by using the magnitude of the Fourier transform in the ridgelet domain. The rotation invariance is proved theoretically and verified by experimental results.

In this paper, a new watermarking scheme in the joint time-frequency domain is introduced. Wigner distribution is used to transform an image into the spatial-spectral domain. The proposed method selects the time-frequency cells to be watermarked based on the particular image's energy distribution in the joint domain. This approach ensures the imperceptibility of the embedded watermark. It is shown that embedding in the time-frequency domain is equivalent to a nonlinear embedding function in the spatial domain. A corresponding watermark detection algorithm is also introduced. The performance of the proposed watermarking algorithm under possible attacks, such as noise, re-sampling, rotation, filtering, and JPEG compression is illustrated

In this paper, we introduce a new robust image watermarking technique based on the discrete wavelet transform (DWT). The proposed method extends the concept of image denoising to watermarking. A spatially adaptive wavelet thresholding method is used to select the coefficients to be watermarked. A multi-bit watermark is embedded into the discrete wavelet coefficients of the host image. A semi-blind watermark extraction algorithm is presented and the threshold for a given probability of false alarm is derived. The simulation results show that the proposed method outperforms a well-known DWT based watermarking method under most attacks including JPEG compression

In this paper, application of sparse representation (factorization) of signals over an overcomplete basis (dictionary) for signal classification is discussed. Search- ing for the sparse representation of a signal over an overcomplete dictionary is achieved by optimizing an objective function that includes two terms: one that measures the signal reconstruction error and another that measures the sparsity. This objective function works well in applications where signals need to be recon- structed, like coding and denoising. On the other hand, discriminative methods, such as linear discriminative analysis (LDA), are better suited for classification tasks. However, discriminative methods are usually sensitive to corruption in sig- nals due to lacking crucial properties for signal reconstruction. In this paper, we present a theoretical framework for signal classification with sparse representa- tion. The approach combines the discrimination power of the discriminative meth- ods with the reconstruction property and the sparsity of the sparse representation that enables one to deal with signal corruptions: noise, missing data and outliers. The proposed approach is therefore capable of robust classification with a sparse representation of signals. The theoretical results are demonstrated with signal classification tasks, showing that the proposed approach outperforms the standard discriminative methods and the standard sparse representation in the case of cor- rupted signals.

Information-theoretic characterization of time-frequency distributions have been successful at quantifying the complexity of nonstationary signals. Information measures such as entropy and divergence have been adapted to the time-frequency domain for counting the number of signal components, evaluating the performance of different kernels and discriminating between signals based on their information content. Inspired by the success of these measures and in order to develop a more comprehensive information processing theory on the time-frequency plane, we introduce a mutual information measure for time-frequency distributions. The properties of this measure are derived and its application to signal classification problems is illustrated with examples.

In wavelet packet based image processing and classification, the proper selection of a subset of subbands can achieve a good representation of the image with a small number of subbands. Various algorithms have been proposed to address the subband selection problem. However, these algorithms evaluate the representation power of each subband separately and subsequently choose a set of subbands based on this representation power. Such a process implicitly assumes the independence between different subbands, which seldom holds and thus degrades performance. To address the limit of the existing algorithms, we propose a mutual information based subband selection algorithm for image classification. We also introduce a practical method for computing mutual information in high dimensional space. Our experiments show that the proposed subband selection algorithm effectively improves the accuracy of wavelet packet based image classification.

Image denoising is a well-known problem in signal processing. Wavelet decomposition based approaches have been applied successfully to the image denoising problem. The majority of wavelet thresholding methods do not take the spatial correlation between wavelet coefficients into account. A new image denoising approach is presented; it incorporates the intra-scale dependencies between the wavelet coefficients into the thresholding algorithm. The cooccurrence matrix of the wavelet coefficients and their neighbors is constructed to represent the spatial dependencies. An information-theoretic criterion, the 2D joint entropy of the wavelet cooccurrence matrix, is used as the cost function to determine the optimal threshold. Experimental results indicate that the proposed approach yields significant improvement over universal thresholding, both in visual quality and mean square error.

In wavelet packet based image processing and classification, the proper selection of a subset of subbands is crucial for efficient representation of the image with a small number of subbands. Various algorithms have been proposed to address the subband selection problem. However, these algorithms evaluate the representation power of each subband separately based on a pre-defined cost function and subsequently choose a set of subbands based on these representation powers. This process implicitly assumes the independence between different subbands, which seldom holds and thus, degrades the classification performance. Past experience shows that the subbands with high energy play an important role in classifying texture. To incorporate both the dependence between subbands and the individual power of each subband into the subband selection process, we propose a subband grouping and selection (SGS) algorithm for selecting subbands for texture classification. Our experiments show that the proposed subband selection algorithm effectively outperforms the traditional algorithms by achieving higher classification accuracy with a fewer number of subbands.

Re/spl acute/nyi entropy has been proposed as an effective measure of signal information content and complexity on the time-frequency plane. The previous work concerning Re/spl acute/nyi entropy in the time-frequency plane has focused on measuring the complexity of a given deterministic signal. In this paper, the properties of Re/spl acute/nyi entropy for random signals are examined. The upper and lower bounds on the expected value of Re/spl acute/nyi entropy are derived and ways of minimizing the entropy of time-frequency distributions by putting constraints on the time-frequency kernel are explored. It is proven that the quasi-Wigner kernel has the minimum entropy among all positive time-frequency kernels with finite time-support and correct marginals. A general class of minimum entropy kernels is presented. The performance of minimum entropy kernels in signal representation and component counting is also demonstrated.

Time-frequency analysis is a major tool in representing the energy distribution of time-varying signals. There has been much research on various properties of these representations. However, there is a general lack of quantitative analysis in describing the amount of information encoded into a time-frequency distribution. Recently, entropy based measures have been applied to the time-frequency plane to quantify the information content of signals. The paper aims to extend this approach to include other information theoretic measures, such as divergence measures, to quantify how time-frequency distributions discriminate signals in an information theoretic framework. Different distance measures, such as Kullback-Leibler distance, Renyi distance, and Jensen difference based measures, are adapted to the time-frequency plane. The robustness of different distance measures under an additive perturbation model is derived. The performance of different distance measures in quantifying the differences in information between signals is demonstrated. Finally, the proposed distance measures are applied to a set of event related brain potentials to discriminate different subject groups.

Fingerprint matching has been deployed in a variety of security related applications. Traditional minutiae detection based identification algorithms do not utilize the rich discriminatory texture structure of fingerprint images. Furthermore, minutiae detection requires substantial improvement of image quality and is thus error-prone. In this paper, we propose a new algorithm for fingerprint identification using wavelet packet analysis and best basis selection. Each fingerprint is decomposed using two dimensional wavelet packet family corresponding to different scales. The energy distribution of the fingerprint in each subband is extracted as a feature for identification. Wavelet packet decomposition yields a redundant representation of the image. For this reason, several algorithms for selecting the best basis from this redundant representation have been investigated. In this paper, we propose a new method for choosing best basis in wavelet packets for fingerprint matching. Experiments show that our new algorithm improves the accuracy of fingerprint matching.

A comprehensive theory for time-frequency based signal detection has been developed during the past decade. The time-frequency detectors proposed in the literature are linear structures operating on the time-frequency representation of the signals and are equivalent to quadratic receivers that are defined in the time domain. We introduce the concept of entropy based detection on the time-frequency plane. In recent years, Renyi entropy has been proposed as an effective measure for quantifying signal complexity on the time-frequency plane and some important properties of this measure have been proven. A new approach that uses the entropy functional as the test statistic for signal detection is developed. A minimum error detection algorithm is derived and the performance of this new signal detection method is demonstrated through examples.

Renyi entropy has been proposed as one of the methods for measuring signal information content and complexity on the time-frequency plane. It provides a quantitative measure for the uncertainty of the signal. All of the previous work concerning Renyi entropy in the time-frequency plane has focused on determining the number of signal components in a given deterministic signal. We discuss the behaviour of Renyi entropy when the signal is random, more specifically white complex Gaussian noise. We present the bounds on the expected value of Renyi entropy and discuss ways to minimize the uncertainty by deriving conditions on the time-frequency kernel. The performance of minimum entropy kernels in determining the number of signal elements is demonstrated. Finally, some possible applications of Renyi entropy for signal detection are discussed.

Time-frequency distributions (TFDs) are used for representing the energy density of a signal simultaneously in time and frequency. The frequency marginal is defined as the density of frequency at a particular time. When the frequency marginal is satisfied, it is given by the classical periodogram formulation. In spectral estimation literature, the periodogram is not considered to be a good spectral estimate because of the high variance it exhibits. For this reason, several modified periodogram methods including Thomson's (1982) spectral estimation method have been introduced. In this paper, a new kernel design method to obtain a smoothed spectrum as the frequency marginal is introduced. The necessary conditions on the time-frequency kernel are derived. The consequences of this new kernel design method in signal analysis are illustrated with examples.