While it is relatively easier to train humanoid robots to mimic specific locomotion skills, it is more challenging to learn from various motions and adhere to continuously changing commands. These robots must accurately track motion instructions, seamlessly transition between a variety of movements, and master intermediate motions not present in their reference data. In this work, we propose a novel approach that integrates human-like motion transfer with precise velocity tracking by a series of improvements to classical imitation learning. To enhance generalization, we employ the Wasserstein divergence criterion (WGAN-div). Furthermore, a Hybrid Internal Model provides structured estimates of hidden states and velocity to enhance mobile stability and environment adaptability, while a curiosity bonus fosters exploration. Our comprehensive method promises highly human-like locomotion that adapts to varying velocity requirements, direct generalization to unseen motions and multitasking, as well as zero-shot transfer to the simulator and the real world across different terrains. These advancements are validated through simulations across various robot models and extensive real-world experiments.

Recent advancements in spatial imaging technologies have revolutionized the acquisition of high-resolution multichannel images, gene expressions, and spatial locations at the single-cell level. Our study introduces xSiGra, an interpretable graph-based AI model, designed to elucidate interpretable features of identified spatial cell types, by harnessing multimodal features from spatial imaging technologies. By constructing a spatial cellular graph with immunohistology images and gene expression as node attributes, xSiGra employs hybrid graph transformer models to delineate spatial cell types. Additionally, xSiGra integrates a novel variant of gradient-weighted class activation mapping component to uncover interpretable features, including pivotal genes and cells for various cell types, thereby facilitating deeper biological insights from spatial data. Through rigorous benchmarking against existing methods, xSiGra demonstrates superior performance across diverse spatial imaging datasets. Application of xSiGra on a lung tumor slice unveils the importance score of cells, illustrating that cellular activity is not solely determined by itself but also impacted by neighboring cells. Moreover, leveraging the identified interpretable genes, xSiGra reveals endothelial cell subset interacting with tumor cells, indicating its heterogeneous underlying mechanisms within complex cellular interactions.

OBJECTIVE: Our objective is to develop and validate TrajVis, an interactive tool that assists clinicians in using artificial intelligence (AI) models to leverage patients' longitudinal electronic medical records (EMRs) for personalized precision management of chronic disease progression. MATERIALS AND METHODS: We first perform requirement analysis with clinicians and data scientists to determine the visual analytics tasks of the TrajVis system as well as its design and functionalities. A graph AI model for chronic kidney disease (CKD) trajectory inference named DisEase PrOgression Trajectory (DEPOT) is used for system development and demonstration. TrajVis is implemented as a full-stack web application with synthetic EMR data derived from the Atrium Health Wake Forest Baptist Translational Data Warehouse and the Indiana Network for Patient Care research database. A case study with a nephrologist and a user experience survey of clinicians and data scientists are conducted to evaluate the TrajVis system. RESULTS: The TrajVis clinical information system is composed of 4 panels: the Patient View for demographic and clinical information, the Trajectory View to visualize the DEPOT-derived CKD trajectories in latent space, the Clinical Indicator View to elucidate longitudinal patterns of clinical features and interpret DEPOT predictions, and the Analysis View to demonstrate personal CKD progression trajectories. System evaluations suggest that TrajVis supports clinicians in summarizing clinical data, identifying individualized risk predictors, and visualizing patient disease progression trajectories, overcoming the barriers of AI implementation in healthcare. DISCUSSION: The TrajVis system provides a novel visualization solution which is complimentary to other risk estimators such as the Kidney Failure Risk Equations. CONCLUSION: TrajVis bridges the gap between the fast-growing AI/ML modeling and the clinical use of such models for personalized and precision management of chronic diseases.

Popular clustering methods, such as the k-means and the Gaussian mixture model (GMM), are composed of an initialization stage and an iteration stage. Although this is well known, the role of the two stages has not been well studied for existing clustering methods yet. To understand this issue, the current research investigates well-known existing k-means and the GMM methods. The finding is that the initialization stage is more critical than the iteration stage. Based on this issue, the research develops improved k-means and GMM methods by using a nice initialization, which significantly enhances the performance of the corresponding methods. Experiments based on Monte Carlo simulations show that the proposed improved k-means and GMM methods can completely identify the true clusters if they are well separated from each, but this cannot be achieved by the existing k-means and GMM methods. Ex-periments based on a real-world dataset show that the proposed methods can be efficiently combined with dimension-reduction techniques for clustering high-dimensional massive data.

Spatial cellular authors heterogeneity contributes to differential drug responses in a tumor lesion and potential therapeutic resistance. Recent emerging spatial technologies such as CosMx, MERSCOPE and Xenium delineate the spatial gene expression patterns at the single cell resolution. This provides unprecedented opportunities to identify spatially localized cellular resistance and to optimize the treatment for individual patients. In this work, we present a graph-based domain adaptation model, SpaRx, to reveal the heterogeneity of spatial cellular response to drugs. SpaRx transfers the knowledge from pharmacogenomics profiles to single-cell spatial transcriptomics data, through hybrid learning with dynamic adversarial adaption. Comprehensive benchmarking demonstrates the superior and robust performance of SpaRx at different dropout rates, noise levels and transcriptomics coverage. Further application of SpaRx to the state-of-the-art single-cell spatial transcriptomics data reveals that tumor cells in different locations of a tumor lesion present heterogenous sensitivity or resistance to drugs. Moreover, resistant tumor cells interact with themselves or the surrounding constituents to form an ecosystem for drug resistance. Collectively, SpaRx characterizes the spatial therapeutic variability, unveils the molecular mechanisms underpinning drug resistance and identifies personalized drug targets and effective drug combinations.

Cell-cell communications are vital for biological signalling and play important roles in complex diseases. Recent advances in single-cell spatial transcriptomics (SCST) technologies allow examining the spatial cell communication landscapes and hold the promise for disentangling the complex ligand-receptor (L-R) interactions across cells. However, due to frequent dropout events and noisy signals in SCST data, it is challenging and lack of effective and tailored methods to accurately infer cellular communications. Herein, to decipher the cell-to-cell communications from SCST profiles, we propose a novel adaptive graph model with attention mechanisms named spaCI. spaCI incorporates both spatial locations and gene expression profiles of cells to identify the active L-R signalling axis across neighbouring cells. Through benchmarking with currently available methods, spaCI shows superior performance on both simulation data and real SCST datasets. Furthermore, spaCI is able to identify the upstream transcriptional factors mediating the active L-R interactions. For biological insights, we have applied spaCI to the seqFISH+ data of mouse cortex and the NanoString CosMx Spatial Molecular Imager (SMI) data of non-small cell lung cancer samples. spaCI reveals the hidden L-R interactions from the sparse seqFISH+ data, meanwhile identifies the inconspicuous L-R interactions including THBS1-ITGB1 between fibroblast and tumours in NanoString CosMx SMI data. spaCI further reveals that SMAD3 plays an important role in regulating the crosstalk between fibroblasts and tumours, which contributes to the prognosis of lung cancer patients. Collectively, spaCI addresses the challenges in interrogating SCST data for gaining insights into the underlying cellular communications, thus facilitates the discoveries of disease mechanisms, effective biomarkers and therapeutic targets.

A/B testing is a powerful tool for a company to make informed decisions about their services and products. A limitation of A/B tests is that they do not easily extend to measure post-experiment (long-term) differences. In this talk, we study a different approach inspired by recent advances in off-policy evaluation in reinforcement learning (RL). The basic RL approach assumes customer behavior follows a stationary Markovian process, and estimates the average engagement metric when the process reaches the steady state. However, in realistic scenarios, the stationary assumption is often violated due to weekly variations and seasonality effects. To tackle this challenge, we propose a variation by relaxing the stationary assumption. We empirically tested both stationary and nonstationary approaches in a synthetic dataset and an online store dataset.

Identifying brain abnormalities in autism spectrum disorder (ASD) is critical for early diagnosis and intervention. To explore brain differences in ASD and typical development (TD) individuals by detecting structural features using T1-weighted magnetic resonance imaging (MRI), we developed a deep learning-based approach, three-dimensional (3D)-ResNet with inception (I-ResNet), to identify participants with ASD and TD and propose a gradient-based backtracking method to pinpoint image areas that I-ResNet uses more heavily for classification. The proposed method was implemented in a preschool dataset with 110 participants and a public autism brain imaging data exchange (ABIDE) dataset with 1099 participants. An extra epilepsy dataset with 200 participants with clear degeneration in the parahippocampal area was applied as a verification and an extension. Among the datasets, we detected nine brain areas that differed significantly between ASD and TD. From the ROC in PASD and ABIDE, the sensitivity was 0.88 and 0.86, specificity was 0.75 and 0.62, and area under the curve was 0.787 and 0.856. In a word, I-ResNet with gradient-based backtracking could identify brain differences between ASD and TD. This study provides an alternative computer-aided technique for helping physicians to diagnose and screen children with an potential risk of ASD with deep learning model.

We consider off-policy evaluation (OPE), which evaluates the performance of a new policy from observed data collected from previous experiments, without requiring the execution of the new policy. This finds important applications in areas with high execution cost or safety concerns, such as medical diagnosis, recommendation systems and robotics. In practice, due to the limited information from off-policy data, it is highly desirable to construct rigorous confidence intervals, not just point estimation, for the policy performance. In this work, we propose a new variational framework which reduces the problem of calculating tight confidence bounds in OPE into an optimization problem on a feasible set that catches the true state-action value function with high probability. The feasible set is constructed by leveraging statistical properties of a recently proposed kernel Bellman loss (Feng et al., 2019). We design an efficient computational approach for calculating our bounds, and extend it to perform post-hoc diagnosis and correction for existing estimators. Empirical results show that our method yields tight confidence intervals in different settings.

Aerial imagery has been increasingly adopted in mission-critical tasks, such as traffic surveillance, smart cities, and disaster assistance. However, identifying objects from aerial images faces the following challenges: 1) objects of interests are often too small and too dense relative to the images; 2) objects of interests are often in different relative sizes; and 3) the number of objects in each category is imbalanced. A novel network structure, Points Estimated Network (PENet), is proposed in this work to answer these challenges. PENet uses a Mask Resampling Module (MRM) to augment the imbalanced datasets, a coarse anchor-free detector (CPEN) to effectively predict the center points of the small object clusters, and a fine anchor-free detector FPEN to locate the precise positions of the small objects. An adaptive merge algorithm Non-maximum Merge (NMM) is implemented in CPEN to address the issue of detecting dense small objects, and a hierarchical loss is defined in FPEN to further improve the classification accuracy. Our extensive experiments on aerial datasets visDrone [1] and UAVDT [2] showed that PENet achieved higher precision results than existing state-of-the-art approaches. Our best model achieved 8.7% improvement on visDrone and 20.3% on UAVDT.

Off-policy evaluation provides an essential tool for evaluating the effects of different policies or treatments using only observed data. When applied to high-stakes scenarios such as medical diagnosis or financial decision-making, it is essential to provide provably correct upper and lower bounds of the expected reward, not just a classical single point estimate, to the end-users, as executing a poor policy can be very costly. In this work, we propose a provably correct method for obtaining interval bounds for off-policy evaluation in a general continuous setting. The idea is to search for the maximum and minimum values of the expected reward among all the Lipschitz Q-functions that are consistent with the observations, which amounts to solving a constrained optimization problem on a Lipschitz function space. We go on to introduce a Lipschitz value iteration method to monotonically tighten the interval, which is simple yet efficient and provably convergent. We demonstrate the practical efficiency of our method on a range of benchmarks.

Categorical variables are nominal variables that classify observations by groups. The treatment of categorical variables in regression is a well-studied yet vital problem, with the most popular solution to perform a one hot encoding. However, challenges arise if a categorical variable has millions of levels. It will cause the memory needed for the computation far exceeds the total available memory in a given computer system or even a computer cluster. Thus, it is fair to state that one hot encoding approach has its limitations when a categorical variable has a large number of levels. The common workaround is the sparse matrix approach because it requires much fewer resources to cache the dummy variables. However, existing sparse matrix approaches are still not sufficient to handle extreme cases when a categorical variable has millions of levels. For instance, the number of subnets in network traffic analyses can easily exceeds tens of millions. In this paper, we proposed an innovative approach called sparse block regression (SBR) to address this challenge. SBR constructs a sparse block matrix using sufficient statistics. The benefits include but not limited to: 1) overcome the memory barrier issue caused by one hot encoding, 2) obtain multiple models with a single scan of data stored in the secondary storage; and 3) update the models with simple matrix operations. The study compared proposed SBR against conventional sparse matrix approaches. The experiments proved that SBR can efficiently and accurately solve the regression problem with large category number. Compared to the sparse matrix approach, SBR saved 90% memory in size during the computation.

Regression problems that have closed-form solutions are well understood and can be easily implemented when the dataset is small enough to be all loaded into the RAM. Challenges arise when data are too big to be stored in RAM to compute the closed form solutions. Many techniques were proposed to overcome or alleviate the memory barrier problem but the solutions are often local optima. In addition, most approaches require loading the raw data to the memory again when updating the models. Parallel computing clusters are often expected in practice if multiple models need to be computed and compared. We propose multiple learning approaches that utilize an array of sufficient statistics (SS) to address the aforementioned big data challenges. The memory oblivious approaches break the memory barrier when computing regressions with closed-form solutions, including but not limited to linear regression, weighted linear regression, linear regression with Box-Cox transformation (Box-Cox regression) and ridge regression models. The computation and update of the SS arrays can be handled at per row level or per mini-batch level. And updating a model is as easy as matrix addition and subtraction. Furthermore, the proposed approaches also enable the computational parallelizability of multiple models because multiple SS arrays for different models can be computed simultaneously with a single pass of slow disk I/O access to the dataset. We implemented our approaches on Spark and evaluated over the simulated datasets. Results showed our approaches can achieve exact solutions of multiple models. The training time saved compared to the traditional methods is proportional to the number of models need to be investigated.

Stein variational gradient descent (SVGD) is a particle-based inference algorithm that leverages gradient information for efficient approximate inference. In this work, we enhance SVGD by leveraging preconditioning matrices, such as the Hessian and Fisher information matrix, to incorporate geometric information into SVGD updates. We achieve this by presenting a generalization of SVGD that replaces the scalar-valued kernels in vanilla SVGD with more general matrix-valued kernels. This yields a significant extension of SVGD, and more importantly, allows us to flexibly incorporate various preconditioning matricesto accelerate the exploration in the probability landscape. Empirical results show that our method outperforms vanilla SVGD and a variety of baseline approaches over a range of real-world Bayesian inference tasks.

We consider the off-policy estimation problem of estimating the expected reward of a target policy using samples collected by a different behavior policy. Importance sampling (IS) has been a key technique to derive (nearly) unbiased estimators, but is known to suffer from an excessively high variance in long-horizon problems. In the extreme case of in infinite-horizon problems, the variance of an IS-based estimator may even be unbounded. In this paper, we propose a new off-policy estimation method that applies IS directly on the stationary state-visitation distributions to avoid the exploding variance issue faced by existing estimators.Our key contribution is a novel approach to estimating the density ratio of two stationary distributions, with trajectories sampled from only the behavior distribution. We develop a mini-max loss function for the estimation problem, and derive a closed-form solution for the case of RKHS. We support our method with both theoretical and empirical analyses.