Graph-based data present unique challenges and opportunities for machine learning. Graph Neural Networks (GNNs), and especially those algorithms that capture graph topology through message passing for neighborhood aggregation, have been a leading solution. However, these networks often require substantial computational resources and may not optimally leverage the information contained in the graph’s topology, particularly for large-scale or complex graphs.We propose Topology Coordinate Neural Network (TCNN) and Directional Virtual Coordinate Neural Network (DVCNN) as novel and efficient alternatives to message passing GNNs, that directly leverage the graph’s topology, sidestepping the computational challenges presented by competing algorithms. Our proposed methods can be viewed as a reprise of classic techniques for graph embedding for neural network feature engineering, but they are novel in that our embedding techniques leverage ideas in Graph Coordinates (GC) that are lacking in current practice.Experimental results, benchmarked against the Open Graph Benchmark Leaderboard, demonstrate that TCNN and DVCNN achieve competitive or superior performance to message passing GNNs. For similar levels of accuracy and ROC-AUC, TCNN and DVCNN need far fewer trainable parameters than contenders of the OGBN Leaderboard. The proposed TCNN architecture requires fewer parameters than any neural network method currently listed in the OGBN Leaderboard for both OGBN-Proteins and OGBN-Products datasets. Conversely, our methods achieve higher performance for a similar number of trainable parameters. These results hold across diverse datasets and edge features, underscoring the robustness and generalizability of our methods. By providing an efficient and effective alternative to message passing GNNs, our work expands the toolbox of techniques for graph-based machine learning. A significantly lower number of tunable parameters for a given evaluation metric makes TCNN and DVCNN especially attractive for resource limited IoT/mobile devices and for reducing power consumption of ML models.

Estimating intrinsic dimension of financial markets informs the number of assets in portfolio selection models and can help archetypal Mean-Variance (MV) single portfolio optimization model pioneered by Markowitz, especially when intrinsic dimension is introduced as a cardinality constraint in the model. The balance between portfolio concentration risk and portfolio diversification benefit is a quintessential active portfolio management problem. Portfolio variance-covariance matrix, a second order dataset derived from first order portfolio constituents return time series, is an essential tool to quantify risk in a portfolio. Our goal in this paper is to estimate financial markets non-linear intrinsic dimension using second order inputs such as distances (dissimilarity) between asset returns. Second- order data is widely used in financial applications such as portfolio optimization, hierarchical clustering, and risk metrics. We demonstrate the effectiveness of our techniques on synthetic data, benchmark image problems, and, most importantly, financial markets.

In this paper, we propose a novel method to make distance predictions in real-world social networks. As predicting missing distances is a difficult problem, we take a two-stage approach. Structural parameters for families of synthetic networks are first estimated from a small set of measurements of a real-world network and these synthetic networks are then used to pre-train the predictive neural networks. Since our model first searches for the most suitable synthetic graph parameters which can be used as an “oracle” to create arbitrarily large training data sets, we call our approach “Oracle Search Pre-training” (OSP). Such an approach enables us to evaluate the robustness of the autoencoder to artificial training data while making predictions on real-world social networks.For example, many real-world networks exhibit a Power law structure in their node degree distribution, so a Power law model can provide a foundation for the desired oracle to generate synthetic pre-training networks, if the appropriate Power law graph parameters can be estimated. Accordingly, we conduct experiments on real-world Facebook, Email, and Train Bombing networks and show that OSP outperforms models without pre-training, models pre-trained with inaccurate parameters, and other distance prediction schemes such as Low-rank Matrix Completion. In particular, we achieve a prediction error of less than one hop with only 1% of sampled distances from the social network. OSP can be easily extended to other domains such as random networks by choosing an appropriate model to generate synthetic training data, and therefore promises to impact many different network learning problems.

Detecting anomalous returns in Exchange Traded Fund(ETF) constituents helps us maintain industry sector exposure while at the same time tuning sector ETF holdings by augmenting portfolio with dollar neutral and sector neutral trades. We demonstrate how Robust Principal Component Analysis (RPCA) can be used to tease out observed ETF constituent returns X into L, a low dimensional manifold, and S, an anomalous matrix containing abnormal returns. Abnormality of returns is determined by checking entries in anomalous matrix against public filings in U.S. Securities and Exchange Commission EDGAR database. Additionally, we show how sparsity in S can be controlled by a tuning parameter that gives us improved f1_scores. Moreover, we build several portfolios with daily and weekly turnover to show how trading in sector ETF anomalies can alter the portfolio return profile.

The US government imposes reporting requirements on healthcare providers as health metrics are important for assessing and improving the US healthcare system. However, proposed health metrics requirements can be unnecessarily burdensome if the preliminary analysis does not consider possible redundancies between the various metrics. Accordingly, if some subset of the proposed metrics could be demonstrated to contain nearly all the information of the full set of metrics, then the reporting burden could be substantially lightened with minimum impact. However, such an analysis is complicated by anomalies in the collected metrics. Accordingly, we propose a machine learning approach which simultaneously identifies redundancies in collected healthcare metrics and identifies anomalies in those collected metrics.

Financial markets are comprised of many instruments, are complex, and are constantly changing. However, it is interesting to consider what, if any, commonalities persist in markets over long time horizons. In particular, one can study financial markets under nominal and stressed market conditions, and attempt to discern which market parameters remain invariant and which market parameters change. Herein, we study financial markets from the perspective of low-dimensional manifolds that capture the inherent characteristics of the high-dimensional data that represent the market performance. Using Russe113000 constituents, we estimate intrinsic dimension of the US equity market over 30 years (1986-2016) and analyze those times where the change in dimension is abnormal. In particular, our focus in on novel applications of nonlinear techniques such as Isomap and autoencoders, as opposed to linear technique such as principal component analysis (PCA). Such ideas have many applications, including portfolio diversification and market crash analysis.

Tracking of trajectories of mutually interacted collectively moving agents such as fish, birds, insects, and even humans is an active field in computer vision. However, the trajectories produced by multi-object tracking methods might consist of unconstructed segments of trajectories due to the natural phenomena such as occlusion, change of illumination, etc., which require robust tracking methods. Some tracking methods employ computationally expensive approximation schemes to connect these segments. In this work, we utilize mutual interactions and dependencies between the agents to reconstruct the missing segments of the trajectories. Due to these interactions, the coordinate matrix representing the particles' trajectories of collective motion is often low-rank. Thus, we utilize a low-rank matrix completion technique to reconstruct incomplete trajectories. We apply this approach for two representative self-propelled particle swarms, simulated by the classic Vicsek model, that imitate two real-life collective motion scenarios and use low-rank approximations to analyze their incomplete trajectories.

This paper introduces a novel crowdsourcing consensus model and inference algorithm — which we call PICA (Permutation-Invariant Crowdsourcing Aggregation) — that is designed to recover the ground-truth labels of a dataset while being invariant to the class permutations enacted by the different annotators. This is particularly useful for settings in which annotators may have systematic confusions about the meanings of different classes, as well as clustering problems (e.g., dense pixel-wise image segmentation) in which the names/numbers assigned to each cluster have no inherent meaning.The PICA model is constructed by endowing each annotator with a doubly-stochastic matrix (DSM), which models the probabilities that an annotator will perceive one class and transcribe it into another. We conduct simulations and experiments to show the advantage of PICA compared to two baselines (Majority Vote, and an "unpermutation" heuristic) for three different clustering/labeling tasks. We also explore the conditions under which PICA provides better inference accuracy compared to a simpler but related model based on right-stochastic matrices. Finally, we show that PICA can be used to crowdsource responses for dense image segmentation tasks, and provide a proof-of-concept that aggregating responses in this way could improve the accuracy of this labor-intensive task.

Deep autoencoders, and other deep neural networks, have demonstrated their effectiveness in discovering non-linear features across many problem domains. However, in many real-world problems, large outliers and pervasive noise are commonplace, and one may not have access to clean training data as required by standard deep denoising autoencoders. Herein, we demonstrate novel extensions to deep autoencoders which not only maintain a deep autoencoders' ability to discover high quality, non-linear features but can also eliminate outliers and noise without access to any clean training data. Our model is inspired by Robust Principal Component Analysis, and we split the input data X into two parts, $X = L_{D} + S$, where $L_{D}$ can be effectively reconstructed by a deep autoencoder and $S$ contains the outliers and noise in the original data X. Since such splitting increases the robustness of standard deep autoencoders, we name our model a "Robust Deep Autoencoder (RDA)". Further, we present generalizations of our results to grouped sparsity norms which allow one to distinguish random anomalies from other types of structured corruptions, such as a collection of features being corrupted across many instances or a collection of instances having more corruptions than their fellows. Such "Group Robust Deep Autoencoders (GRDA)" give rise to novel anomaly detection approaches whose superior performance we demonstrate on a selection of benchmark problems.