Accurate simulation of turbulent flows is of immense importance in a variety of scientific and engineering fields. Within the realm of turbulent flow simulation, direct numerical simulation (DNS) is widely considered to be the most reliable approach, but it is prohibitively expensive and thus has limited applicability to long-term and fine-scale simulation over various configurations. Given the pressing need for efficient simulation, there is an increasing interest in building machine learning models for simulating turbulence, either by reconstructing DNS from alternative low-fidelity simulations or sequentially predicting DNS from historical data. However, conventional machine learning models are not designed for capturing complex spatio-temporal characteristics of turbulent flows. This results in their limited performance and generalizability, especially when applied to complex flow data and various flow configurations. This paper presents a novel physics-enhanced neural operator (PENO) that efficiently models the complex flow dynamics while leveraging physical knowledge of partial differential equations (PDEs) to enhance the simulation process. We further introduce a self-augmentation mechanism to reduce the accumulated errors in long-term simulations. The proposed method is evaluated on multiple turbulent flow datasets, showcasing the model's capability to reconstruct high-resolution DNS data, maintain the inherent physical properties of flow transport, and transfer across various resolution settings and simulation configurations. These encouraging results confirm its applicability to a wide range of real-world scenarios in which extensive simulations are needed under diverse settings.

Methane (CH4) is the second most powerful greenhouse gas after carbon dioxide and plays a crucial role in climate change due to its high global warming potential. Accurately modeling CH4 fluxes across the globe and at fine temporal scales is essential for understanding its spatial and temporal variability and developing effective mitigation strategies. In this work, we introduce the first-of-its-kind cross-scale global wetland methane benchmark dataset (X-MethaneWet), which synthesizes physics-based model simulation data from TEM-MDM and the real-world observation data from FLUXNET-CH4. This dataset can offer opportunities for improving global wetland CH4 modeling and science discovery with new AI algorithms. To set up AI model baselines for methane flux prediction, we evaluate the performance of various sequential deep learning models on X-MethaneWet. Furthermore, we explore four different transfer learning techniques to leverage simulated data from TEM-MDM to improve the generalization of deep learning models on real-world FLUXNET-CH4 observations. Our extensive experiments demonstrate the effectiveness of these approaches, highlighting their potential for advancing methane emission modeling and identifying new opportunities for developing more accurate and scalable AI-driven climate models.

Isotopic ratios of hydrogen and oxygen in water serve as powerful tracers of the Earth's hydrological cycle, offering insights into the origins of water vapor, large-scale atmospheric circulation, and moisture transport dynamics. However, integrating water isotopes into fully coupled global climate models (GCMs) is both scientifically and technically challenging due to the complex interactions between water isotopes and the atmosphere, hydrosphere, and cryosphere, as well as the extensive modifications to model physics and dynamics. As a result, most GCMs lack support for isotopes and even the few existing isotope-enabled GCMs still remain highly expensive to run, significantly limiting their usability. Machine learning (ML) offers promising opportunities to emulate the complex process as powerful mathematical approximators. The water isotope fields from the emulators bring potential for applications in isotope-unenabled GCMs. However, the absence of a publicly available ML-ready dataset has hindered the development of robust ML-based emulators. To address this gap, we introduce IsoSim, the first ML-ready benchmark dataset designed to facilitate the development of ML emulators for water isotopes in GCMs. This dataset includes global climate variables and water isotope fields across three spatial dimensions (latitude, longitude, and height) from isotope-enabled GCM simulations, spanning 500 years at a monthly resolution. We also include different climatic scenarios and a diverse set of learning-based emulators to carry out extensive evaluations and build the benchmarks. The dataset and results serve as reference points to compare machine learning models' ability in approximating complex physical relationships.

Probabilistic time-series forecasting plays a vital role in decisionmaking under uncertainty, especially in applications like solar energy, where forecast reliability directly impacts energy planning and grid stability. While recent models have improved in generating predictive distributions rather than single-point estimates, existing evaluations often focus on average performance and overlook how model quality varies across different real-world scenarios. In solar energy monitoring, for example, the difficulty of forecasting can change significantly due to atmospheric variability, sensor types, and climate conditions. This work addresses the need for scenario-aware evaluation of probabilistic models by benchmarking state-of-the-art forecasting methods using SolarCube-a large-scale solar radiation dataset spanning diverse regions, cloud regimes, and environmental conditions. We define structured "easy" and "hard" cases across four scenarios and examine how different probabilistic model families (e.g., diffusion, VAE, flow-based) capture uncertainty under these conditions. Our goal is to move beyond overall metrics and reveal how model reliability changes across scenarios that are critical for downstream applications.

Understanding environmental ecosystems is vital for the sustainable management of our planet. However, existing physics-based and data-driven models often fail to generalize to varying spatial regions and scales due to the inherent data heterogeneity presented in real environmental ecosystems. This generalization issue is further exacerbated by the limited observation samples available for model training. To address these issues, we propose Geo-STARS, a geo-aware spatio-temporal modeling framework for predicting stream water temperature across different watersheds and spatial scales. The major innovation of Geo-STARS is the introduction of geo-aware embedding, which leverages geographic information to explicitly capture shared principles and patterns across spatial regions and scales. We further integrate the geo-aware embedding into a gated spatio-temporal graph neural network. This design enables the model to learn complex spatial and temporal patterns guided by geographic and hydrological context, even with sparse or no observational data. We evaluate Geo-STARS's efficacy in predicting stream water temperature, which is a master factor for water quality. Using real-world datasets spanning 37 years across multiple watersheds along the eastern coast of the United States, Geo-STARS demonstrates its superior generalization performance across both regions and scales, outperforming state-of-the-art baselines. These results highlight the promise of Geo-STARS for scalable, data-efficient environmental monitoring and decision-making.

The discovery of environmental knowledge depends on labeled task-specific data, but is often constrained by the high cost of data collection. Existing machine learning approaches usually struggle to generalize in data-sparse or atypical conditions. To this end, we propose an Augmentation-Adaptive Self-Supervised Learning (A2SL) framework, which retrieves relevant observational samples to enhance modeling of the target ecosys-tem. Specifically, we introduce a multi-level pairwise learning loss to train a scenario encoder that captures varying degrees of similarity among scenarios. These learned similarities drive a retrieval mechanism that supplements a target scenario with relevant data from different locations or time periods. Further-more, to better handle variable scenarios, particularly under atypical or extreme conditions where traditional models struggle, we design an augmentation-adaptive mechanism that selectively enhances these scenarios through targeted data augmentation. Using freshwater ecosystems as a case study, we evaluate A2SL in modeling water temperature and dissolved oxygen dynamics in real-world lakes. Experimental results show that A2SL signif-icantly improves predictive accuracy and enhances robustness in data-scarce and atypical scenarios. Although this study focuses on freshwater ecosystems, the A2SL framework offers a broadly applicable solution in various scientific domains. Code-https://github.com/shiyuanlsy/A2sl

Environmental ecosystems exhibit complex and evolving dynamics over time, making the modeling of non-stationary processes critically important. However, traditional methods often rely on static models trained on entire datasets, failing to capture the non-stationary and drastically fluctuating characteristics. Dynamically adjusting models to evolving data is challenging, as they can easily either lag behind new trends or overfit newly received data. To address these challenges, we propose Domain-Adaptive Continual Meta-Learning (DACM) method, aiming to automatically detect distribution shifts and adapt to newly emergent domains. In particular, while DACM continuously explores the sequential temporal data, it also exploits historical data that are similar in distribution to the current observations. By striking a balance between temporal exploration and distributional exploitation, DACM quickly adjusts the model to stay up-to-date with new trends while maintaining generalization ability to data with similar distributions. We demonstrate the effectiveness of DACM on a real-world water temperature prediction dataset, where it outperforms diverse baseline models and shows strong adaptability and predictive performance in non-stationary environments.

“AI for Science” has become a prominent yet controversial research frontier, eliciting both acclaim and criticism. While celebrated for its potential to revolutionize scientific discovery, concerns persist regarding the reliability, interpretability, and validation of AI-generated knowledge. This paper explores core challenges, including the opacity of AI insights, the difficulty of evaluating unverifiable outputs, and the in-adequacy of traditional frameworks like the Turing Test. Although addressing these issues may seem like a distant goal, this paper proposes a Blue Sky Idea aimed at redefining AI’s role in scientific exploration and paving the way for transformative progress.

Spatial pattern mining (SPM) aims to detect geographic locations or areas that present interesting, nontrivial, and potentially useful patterns. Traditional formulations of point-based SPM tasks are mainly based on true observations, which tend to have limited spatial coverage, availability, and timeliness. While machine learning (ML) has the potential to extend the range of usable data, the uncertainty of model-predicted labels presents new challenges for their usability in the SPM context. This paper formulates the task of ML-enabled SPM using predicted labels by ML models. Given the ever-expanding family of spatial patterns, we consider four widely-adopted patterns – hotspots, co-locations, mixture patterns, and spatial outliers – to scope our study to make the discussion concrete. We develop soft-label versions of SPM algorithms that can directly execute on uncertain predictions generated by ML models. Additionally, we evaluate the ML-enabled SPM results for both categorical and real-valued datasets across a spectrum of prediction quality. The results show that certain spatial patterns such as multinomial scan statistic-based mixture patterns and normal-model-based hotspots can more robustly maintain the detection quality at different error levels, while others such as spatial outliers are more sensitive to incorrect predictions. This provides helpful guidance on using learning-based predictions for SPM.

Accurate long-term predictions are the foundations for many machine learning applications and decision-making processes. Traditional time series approaches for prediction often focus on either autoregressive modeling, which relies solely on past observations of the target “endogenous variables”, or forward modeling, which considers only current covariate drivers “exogenous variables”. However, effectively integrating past endogenous and past exogenous with current exogenous variables remains a significant challenge. In this paper, we propose ExoTST, a novel transformer-based framework that effectively incorporates current exogenous variables alongside past context for improved time series prediction. To integrate exogenous information efficiently, ExoTST leverages the strengths of attention mechanisms and introduces a novel cross-temporal modality fusion module. This module enables the model to jointly learn from both past and current exogenous series, treating them as distinct modalities. By considering these series separately, ExoTST provides robustness and flexibility in handling data uncertainties that arise from the inherent distribution shift between historical and current exogenous variables. Extensive experiments on real-world carbon flux datasets and time series benchmarks demonstrate ExoTST's superior performance compared to state-of-the-art baselines, with improvements of up to 10% in prediction accuracy. Moreover, ExoTST exhibits strong robustness against missing values and noise in exogenous drivers, maintaining consistent performance in real-world situations where these imperfections are common.

This paper introduces a Process-Guided Learning (Pril) framework that integrates physical models with recurrent neural networks (RNNs) to enhance the prediction of dissolved oxygen (DO) concentrations in lakes, which is crucial for sus-taining water quality and ecosystem health. Unlike traditional RNNs, which may deliver high accuracy but often lack physical consistency and broad applicability, the Pril method incorporates differential DO equations for each lake layer, modeling it as a first-order linear solution using a forward Euler scheme with a daily timestep. However, this method is sensitive to numerical instabilities. When drastic fluctuations occur, the numerical integration is neither mass-conservative nor stable. Especially during stratified conditions, exogenous fluxes into each layer cause significant within-day changes in DO concentrations. To address this challenge, we further propose an Adaptive Process-Guided Learning (April) model, which dynamically adjusts timesteps from daily to sub-daily intervals with the aim of mitigating the discrepancies caused by variations in entrainment fluxes. April uses a generator-discriminator architecture to identify days with significant DO fluctuations and employs a multi-step Euler scheme with sub-daily timesteps to effectively manage these variations. We have tested our methods on a wide range of lakes in the Midwestern USA, and demonstrated robust capability in predicting DO concentrations even with limited training data. While primarily focused on aquatic ecosystems, this approach is broadly applicable to diverse scientific and engineering disciplines that utilize process-based models, such as power engineering, climate science, and biomedicine.

Training machine learning (ML) models for scientific problems is often challenging due to limited observation data. To overcome this challenge, prior works commonly pre-train ML models using simulated data before having them fine-tuned with small real data. Despite the promise shown in initial research across different domains, these methods cannot ensure improved performance after fine-tuning because (i) they are not designed for extracting generalizable physics-aware features during pre-training, (ii) the features learned from pre-training can be distorted by the fine-tuning process. In this paper, we propose a new learning method for extracting, preserving, and adapting physics-aware features. We build a knowledge-guided neural network (KGNN) model based on known dependencies amongst physical variables, which facilitate extracting physics-aware feature representation from simulated data. Then we fine-tune this model by alternately updating the encoder and decoder of the KGNN model to enhance the prediction while preserving the physics-aware features learned through pre-training. We further propose to adapt the model to new testing scenarios via a teacher-student learning framework based on the model uncertainty. The results demonstrate that the proposed method outperforms many baselines by a good margin, even using sparse training data or under out-of-sample testing scenarios.

Human-generated Spatial-Temporal Data (HSTD), represented as trajectory sequences, has undergone a data revolution, thanks to advances in mobile sensing, data mining, and AI. Previous studies have revealed the effectiveness of employing attention mechanisms to analyze massive HSTD. However, traditional attention models face challenges when managing lengthy and noisy trajectories as their computation comes with large memory overheads. Furthermore, attention scores within HSTD trajectories are sparse (i.e., most of the scores are zeros), and clustered with varying lengths (i.e., consecutive tokens clustered with similar scores). To address these challenges, we introduce an innovative strategy named Memory-efficient Trajectory Attention (MeTA). We leverage complicated spatial-temporal features (e.g., traffic speed, proximity to PoIs) and design an innovative feature-based trajectory partition technique to shrink trajectory length. Additionally, we present a learnable dynamic sorting mechanism, with which attention is only computed between sub-trajectories that have prominent correlations. Empirical validations using real-world HSTD demonstrate that our approach not only yields competitive results but also significantly lowers memory usage compared with state-of-the-art methods. Our approach presents innovative solutions for memory-efficient trajectory attention, offering valuable insights for handling HSTD efficiently.

Accurate prediction of water flow is of utmost importance, particularly for ensuring water supply and informing early actions for floods and droughts. Existing flow prediction methods rely on the input of weather drivers, which hinders their applicability to monitoring small headwater streams due to the limited spatial resolution of existing weather datasets. This paper introduces a new dataset with frequent imagery on streams for water monitoring tasks. Our objective is to automatically predict streamflow for each stream site using frequent images taken at a sub-hourly scale. To overcome the challenge of limited labels for certain stream sites, we employ knowledge transfer from well-observed sites to poorly-observed sites via domain adaptation. As each stream site involves highly variable time series data over long periods, we introduce a novel method STCGAN (Spatial-Temporal Cycle Generative Adversarial Network), which incorporates temporal context by conditioning on the sequence's time and learns overall trends of stream flow variation. It integrates the predictive modeling of streamflow with the cyclic generative process and enhances the prediction with data augmentation using generated synthetic samples. Our experiments demonstrate superior performance of the proposed method using data collected from the West Brook area located in western Massachusetts, US. The proposed method can be further extended to selectively combine information from multiple well-observed stream sites, leading to improved overall performance.

Accurate and timely crop mapping is essential for yield estimation, insurance claims, and conservation efforts. Over the years, many successful machine learning models for crop mapping have been developed that use just the multi-spectral imagery from satellites to predict crop type over the area of interest. However, these traditional methods do not account for the physical processes that govern crop growth. At a high level, crop growth can be envisioned as physical parameters, such as weather and soil type, acting upon the plant leading to crop growth which can be observed via satellites. In this paper, we propose Weather-based Spatio-Temporal segmentation network with ATTention (WSTATT), a deep learning model that leverages this understanding of crop growth by formulating it as an inverse model that combines weather (Daymet) and satellite imagery (Sentinel-2) to generate accurate crop maps. We show that our approach provides significant improvements over existing algorithms that solely rely on spectral imagery by comparing segmentation maps and F1 classification scores. Furthermore, effective use of attention in WSTATT architecture enables detection of crop types earlier in the season (up to 5 months in advance), which is very useful for improving food supply projections. We finally discuss the impact of weather by correlating our results with crop phenology to show that WSTATT is able to capture physical properties of crop growth.

Accurate simulation of turbulent flows is of crucial importance in many branches of science and engineering. Direct numerical simulation (DNS) provides the highest fidelity means of capturing all intricate physics of turbulent transport. However, the method is computationally expensive because of the wide range of turbulence scales that must be accounted for in such simulations. Large eddy simulation (LES) provides an alternative. In such simulations, the large scales of the flow are resolved, and the effects of small scales are modelled. Reconstruction of the DNS field from the low-resolution LES is needed for a wide variety of applications. Thus the construction of super-resolution methodologies that can provide this reconstruction has become an area of active research. In this work, a new physics-guided neural network is developed for such a reconstruction. The method leverages the partial differential equation that underlies the flow dynamics in the design of spatio-temporal model architecture. A degradation-based refinement method is also developed to enforce physical constraints and to further reduce the accumulated reconstruction errors over long periods. Detailed DNS data on two turbulent flow configurations are used to assess the performance of the model.

Fairness related to locations (i.e., “where”) is critical for the use of machine learning in a variety of societal domains involving spatial datasets (e.g., agriculture, disaster response, urban planning). Spatial biases incurred by learning, if left unattended, may cause or exacerbate unfair distribution of resources, social division, spatial disparity, etc. The goal of this work is to develop statistically-robust formulations and model-agnostic learning strategies to understand and promote spatial fairness. The problem is challenging as locations are often from continuous spaces with no well-defined categories (e.g., gender), and statistical conclusions from spatial data are fragile to changes in spatial partitionings and scales. Existing studies in fairness-driven learning have generated valuable insights related to non-spatial factors including race, gender, education level, etc., but research to mitigate location-related biases still remains in its infancy, leaving the main challenges unaddressed. To bridge the gap, we first propose a robust space-as-distribution (SPAD) representation of spatial fairness to reduce statistical sensitivity related to partitionings and scales in continuous space. Furthermore, we propose a new SPAD-based stochastic strategy to efficiently optimize over an extensive distribution of fairness criteria, and a bi-level training framework to enforce fairness via adaptive adjustment of priorities among locations. Finally, we extend this framework with a similarity-based training strategy to improve the computational efficiency. Experiments conducted on two real-world problems, crop monitoring in the US and palm oil plantation mapping in Indonesia, show that SPAD can effectively reduce sensitivity in fairness evaluation and the stochastic bi-level training framework can greatly improve the fairness. Controlled experiments also show that similarity-based acceleration can greatly reduce the training time while keeping the prediction performance and fairness results at the same level.

This paper proposes a meta-transfer-learning method for predicting daily maximum water temperature in stream networks with explicit modeling of extreme events. Accurate prediction of these extreme events is challenging because of their sparsity in the training data and their distinct responses to external drivers when compared to non-extreme observations. To overcome these challenges, we propose a sample reweighting strategy to escalate the importance of extreme events in the training process while preserving the predictive performance in normal time periods. The sample weight for each training data point is estimated as the similarity with the target test data point using contextual information and physical simulation. The obtained sample weight values are then used to fine-tune the initial model to transfer it to the test data. This method is further enhanced by an extreme value theory-based loss function to enforce the distribution of extreme data points and accelerated by a clustering algorithm based on the estimated similarities. Additionally, we introduce an online learning strategy to further refine the predictive model using newly collected observed data. The experimental results using real stream data from the Delaware River Basin over the past 36 years demonstrate that our meta-transfer-learning method produces more accurate predictions in both normal and extreme time periods when compared to baselines without the sample re-weighting scheme. The similarity learning method can reveal meaningful relationships amongst data points. We also show that the clustering algorithm can be used to accelerate the prediction while not compromising the predictive performance. The online learning strategy is shown to further improve predictive performance using recently observed data.

Ecosystem services, such as carbon sequestration, biodiversity, and climate regulation, play essential roles in combating climate change. Projection of ecosystem dynamics under various scenarios is critical in understanding potential impacts and informing policies and mitigation strategies. Ecosystem Demography (ED) model is a major mechanistic model for ecosystem dynamics projection, but its computational cost has been a major bottleneck in performing large-scale (e.g., global, national) projections at very high spatial resolution. We aim to approximate the ED model using deep neural networks at operational high accuracy to assist large-scale climate studies. The deep approximation is non-trivial due to challenges by long-term error accumulation (e.g., 40 years), highly diverse scenarios, and high cost in training data generation. We propose a Deep-ED approximation model to address the challenges with a multi-scale cumulative loss reduction structure, significance-based scenario partitioning, self-guided forwarding, and physics-aware active learning strategies. Experiment results in the northeastern US demonstrate the high accuracy of Deep-ED and its potential in large-scale ecosystem projection.

With the recent rapid advances of revolutionary AI models such as ChatGPT, foundation models have become a main topic for the discussion of future AI. Despite the excitement, the success is still limited to specific types of tasks. Particularly, ChatGPT and similar foundation models have unique characteristics that are difficult to replicate for most geospatial tasks. This paper envisions several major challenges and opportunities in the creation of geospatial foundation (geo-foundation) models, as well as potential future adoption scenarios. We also expect that a major success story is necessary for geo-foundation models to take off in the long term.

Advancements in mobile sensing, data mining, and artificial intelligence have revolutionized the collection and analysis of Human-generated Spatial-Temporal Data (HSTD), paving the way for diverse applications across multiple domains. However, previous works have primarily focused on designing task-specific models for different problems, which lack transferability and generalizability when confronted with diverse HSTD. Additionally, these models often require a large amount of labeled data for optimal performance. While pre-trained models in Natural Language Processing (NLP) and Computer Vision (CV) domains have showcased impressive transferability and generalizability, similar efforts in the spatial-temporal data domain have been limited. In this paper, we take the lead and introduce the Spatial-Temporal Pre-Training model, $i.e$., STPT, which is connected with a self-supervised learning task, to address these limitations. STPT enables the creation of robust and versatile representations of HSTD. We validate our framework using real-world data and demonstrate its efficacy through two downstream tasks, $i.e$., trajectory classification and driving activity identification $(e.g$., identifying seeking $vs$. serving behaviors in taxi trajectories). Our results achieve an accuracy of 83.125% (16.2% higher than the average baseline) for human mobility identification and an accuracy of 77.88% (13.0% higher than the average baseline) for the human activity identification task. These outcomes underscore the potential of our pre-trained model for diverse downstream applications within the spatial-temporal data domain.

Accurate long-term predictions are the foundations for many machine learning applications and decision-making processes. However, building accurate long-term prediction models remains challenging due to the limitations of existing temporal models like recurrent neural networks (RNNs), as they capture only the statistical connections in the training data and may fail to learn the underlying dynamics of the target system. To tackle this challenge, we propose a novel machine learning model based on Koopman operator theory, which we call Koopman Invertible Autoencoders (KIA), that captures the inherent characteristic of the system by modeling both forward and backward dynamics in the infinite-dimensional Hilbert space. This enables us to efficiently learn low-dimensional representations, resulting in more accurate predictions of long-term system behavior. Moreover, our method’s invertibility design enforces reversibility and consistency in both forward and inverse operations. We illustrate the utility of KIA on pendulum and climate datasets, demonstrating 300% improvements in long-term prediction capability for pendulum while maintaining robustness against noise. Additionally, our method demonstrates the ability to better comprehend the intricate dynamics of the climate system when compared to existing Koopman-based methods.

Flood inundation mapping from Earth imagery plays a vital role in rapid disaster response and national water forecasting. However, the problem is non-trivial due to significant imagery noise and obstacles, complex spatial dependency on 3D terrains, spatial non-stationarity, and high computational cost. Existing machine learning approaches are mostly terrain-unaware and are prone to produce spurious results due to imagery noise and obstacles, requiring significant efforts in post-processing. Recently, several terrain- aware methods were proposed that incorporate complex spatial dependency (e.g., water flow directions on 3D terrains) but they assume that the inferred flood surface level is spatially stationary, making them insufficient for a large heterogeneous geographic area. To address these limitations, this paper proposes a novel spatial learning framework called hidden Markov forest, which decomposes a large heterogeneous area into local stationary zones, represents spatial dependency on 3D terrains via zonal trees (forest), and jointly infers the class map in different zonal trees with spatial regularization. We design efficient inference algorithms based on dynamic programming and multi-resolution filtering. Evaluations on real-world datasets show that our method outperforms baselines and our proposed computational refinement significantly reduces the time cost.

Physics-based groundwater flow equations are powerful tools for water resource assessment under different hydrological and climatic conditions. How these conditions affect the discharge of groundwater (i.e., base-flow) into rivers is one of the most important topics in the hydrology domain. However, due to the different environmental conditions in different basins, it is difficult to use a single physics-based equation to represent the discharge of groundwater in all river basins. Despite the promise of data-driven models in capturing complex relationships, they are also limited in learning heterogeneous baseflow patterns from multiple basins, especially with sparse training data. In this paper, we propose a new data-driven model Physics Guided MeTa Learning (PGMTL), which uses meta-learning to adapt the predictive model to multiple basins and also enhance the meta-learning process with knowledge embodied in different physics-based equations so as to improve the baseflow prediction over a large number of river basins. Experimental results show that our proposed PGMTL has a significant improvement over either physics-based equations or ML models. Moreover, our method has been shown to perform much better with sparse or localized training data. Finally, our method is able to interpret the contribution of each physics-based equation under different scenarios.

This paper introduces a new method for combining simulated data over different types of nodes in heterogeneous graphs to facilitate predictive learning. Simulation has been widely used in scientific domains to mitigate the need for a large number of observation samples. However, simulated data are often created separately for each type of physical systems while interactions amongst different types of systems remain unexplored. Our method is developed in the context of predicting water temperature in stream networks, which is critical for decision making in water management. In particular, we first develop a graph diffusion network (GDN) to model the interactions amongst stream segments and reservoirs in a heterogeneous graph. We use the GDN model to combine simulated data for both streams and reservoirs in the graph, and use the obtained composite simulations to train the GDN model in a semi-supervised manner. Then the GDN model is further fine-tuned using true observations. Since observation data are often sparse and localized, we further leverage the information from simulations to build a reweighting strategy so as to migitage the discrepancy between training and testing data. Our evaluations in the Delaware River Basin have shown the superiority of the proposed method over multiple baselines using either sparse or localized training data. The proposed GDN model also creates a better composite simulation dataset for heterogeneous graphs.

In many environmental applications, recurrent neural networks (RNNs) are often used to model physical variables with long temporal dependencies. However, due to minibatch training, temporal relationships between training segments within the batch (intra-batch) as well as between batches (inter-batch) are not considered, which can lead to limited performance. Stateful RNNs aim to address this issue by passing hidden states between batches. Since Stateful RNNs ignore intra-batch temporal dependency, there exists a trade-off between training stability and capturing temporal dependency. In this paper, we provide a quantitative comparison of different Stateful RNN modeling strategies, and propose two strategies to enforce both intra- and inter-batch temporal dependency. First, we extend Stateful RNNs by defining a batch as a temporally ordered set of training segments, which enables intra-batch sharing of temporal information. While this approach significantly improves the performance, it leads to much larger training times due to highly sequential training. To address this issue, we further propose a new strategy which augments a training segment with an initial value of the target variable from the timestep right before the starting of the training segment. In other words, we provide an initial value of the target variable as additional input so that the network can focus on learning changes relative to that initial value. By using this strategy, samples can be passed in any order (mini-batch training) which significantly reduces the training time while maintaining the performance. In demonstrating the utility of our approach in hydrological modeling, we observe that the most significant gains in predictive accuracy occur when these methods are applied to state variables whose values change more slowly, such as soil water and snowpack, rather than continuously moving flux variables such as streamflow.

Multi-modal time series data in real applications often contain data of different dimensionalities, e.g., high-dimensional modality such as image data series, and low-dimensional univariate time series. Multi-modal time series data with missing high-dimensional modal values are ubiquitous in real-world classification and regression applications. To accurately predict the target labels, it is important to appropriately impute the high-dimensional modal missing values. However, most existing imputation methods focus on multivariate time series, fail to simultaneously consider temporal dependencies within each series and the correlations across the series, and also lack a probabilistic interpretation. In this paper, we propose a novel method, which uses a new structured variational approximation technique for the imputation of missing values in multi-modal time series. Instead of directly imputing high-dimensional modal missing values, we use the variational approximation technique to impute intermediate lower-dimensional feature representations of high-dimensional modal missing values from simple modalities related to high-dimensional modality and then feed them into a dynamical model. The dynamical model captures the temporal dependencies of the feature representations and finally predicts the target labels. In order to address the optimization difficulties caused by the lack of ground truth values of lower-dimensional feature representations, we also propose a two-stage isolated optimization strategy for better convergence. We evaluate our method on a real-world stream monitoring dataset. Our extensive experiments demonstrate that the proposed method outperforms several state-of-the-art methods in both data imputation and prediction performance.

Airborne LiDAR point clouds record three-dimensional structures of ground surfaces with high precision, and have been widely used to identify geospatial objects, facilitating the understanding of the distribution and changing dynamics of the environment. Detection can be complicated by the complex structures of ground objects and noises in LiDAR point clouds. Related work has explored the use of deep learning techniques such as YOLO in detecting geospatial objects (e.g., building footprints) on both optical imagery and LiDAR point clouds. However, deep networks are data hungry and there are often limited labeled samples available for many geospatial object mapping tasks, making it difficult for the models to generalize to unseen test regions. This paper describes the framework used in the 11th SIGSPATIAL Cup Competition (GIS CUP 2022), which received the top-3 performance. Our framework incorporates domain knowledge to reduce the difficulty of learning and the model's reliance on large training sets. Specifically, we present knowledge-informed feature generation and filtering based on morphological characteristics to improve the generalizability of learned features. Then, we use a deep segmentation backbone (U-Net) with training- and test-time augmentation to generate preliminary candidates for building footprints. Finally, we utilize domain rules (e.g., geometric properties) to regularize and filter the detections to create the final map of building footprints. Experiment results show that the strategies can effectively improve detection results in different landscapes.

Collecting large annotated datasets in Remote Sensing is often expensive and thus can become a significant obstacle for training advanced machine learning models. Standard techniques for addressing this issue, based on the underlying idea of pre-training the Deep Neural Networks (DNN) on freely available large datasets, cannot be used for Remote Sensing due to the unavailability of such large-scale labeled datasets and the heterogeneity of data sources caused by the varying spatial and spectral resolution of different sensors. Self-supervised learning is an alternative approach that learns feature representation from unlabeled images without human annotations. In this paper, we introduce a new method for land cover mapping by using a clustering-based pretext task for self-supervised learning. We demonstrate the method's effectiveness in two societally relevant applications from the aspect of segmentation performance, discriminative feature representation learning, and the underlying cluster structure. We also show the effectiveness of the active sampling using the clusters obtained from our method in improving the mapping accuracy given a limited budget for annotating. Finally, a real-world application of the developed framework in identifying intra-class categories of well-managed and poorly-managed plantations demonstrates its utility in a problem of considerable societal importance.

As the adoption of machine learning continues to thrive, fairness of the algorithms has become a key factor determining their long-term success and sustainability. Among them, location-based fairness - or spatial fairness - is critical for a variety of essential societal applications that commonly rely on spatial data, including agriculture, disaster response, urban planning, etc. Spatial biases incurred by learning, if left unattended, may cause or exacerbate unfair distribution of resources, spatial disparity, social division, etc. However, very limited understanding has been developed on location-based fairness and bias in machine learning. Compared to traditional fairness-preserving techniques, the spatial consideration introduces two major layers of complication: (1) Space is continuous with no well-defined categories (e.g., categories by race or gender); and (2) Categorizations given by space-partitionings are known to be subject to high statistical sensitivity (e.g., gerrymandering). Under these challenges, we formally explore and demonstrate the fragility of learning methods in the spatial fairness-bias sphere. Specifically, we present a set of techniques that can maneuver the training process towards various targeted fairness-bias outcomes, while maintaining the same level of overall prediction performance (i.e., for "free"). Extensive experiments are carried out on two real-world problems: crop monitoring in the US and palm oil plantation mapping in Indonesia. The results demonstrate the effectiveness of the manipulation algorithms and the importance of explicitly regulating location-based fairness using a diverse set of criteria.

Human mobility estimation is crucial during the COVID-19 pandemic due to its significant guidance for policymakers to make non-pharmaceutical interventions. While deep learning approaches outperform conventional estimation techniques on tasks with abundant training data, the continuously evolving pandemic poses a significant challenge to solving this problem due to data non-stationarity, limited observations, and complex social contexts. Prior works on mobility estimation either focus on a single city or lack the ability to model the spatio-temporal dependencies across cities and time periods. To address these issues, we make the first attempt to tackle the cross-city human mobility estimation problem through a deep meta-generative framework. We propose a Spatio-Temporal Meta-Generative Adversarial Network (STORM-GAN) model that estimates dynamic human mobility responses under a set of social and policy conditions related to COVID-19. Facilitated by a novel spatio-temporal task-based graph (STTG) embedding, STORM-GAN is capable of learning shared knowledge from a spatio-temporal distribution of estimation tasks and quickly adapting to new cities and time periods with limited training samples. The STTG embedding component is designed to capture the similarities among cities to mitigate cross-task heterogeneity. Experimental results on real-world data show that the proposed approach can greatly improve estimation performance and outperform baselines.

Prediction of response to input drivers by unmonitored entities has been recognized as one of the most important problems in many scientific problems. This problem is challenging due to the non-stationary processes that underlie the dynamics of data observations over space and time. Hence, directly transferring models from well-observed data entities to unmonitored target entity often lead to sub-optimal performance due to the shift in data distribution. This paper proposes a new meta-transfer learning framework that automatically estimates the similarity amongst entities to transfer knowledge from well-observed entities to unmonitored entities. A sequence autoencoder embeds temporal behaviors of time series data and simulations generated by traditional physics-based models. This embedding model is trained in a meta-transfer learning framework under the guidance of source-to-source transferring experiences. We tested this method in streamflow prediction for multiple river segments in the Delaware River Basin, an ecologically diverse region along the eastern coast of the United States. The experimental results demonstrate the superiority of the proposed method in predicting streamflow for unmonitored stream segments compared to a diverse set of baselines. Our method also creates meaningful similarity estimates amongst segments to guide the transfer learning process.

This paper proposes a graph-based meta learning approach to separately predict water quantity and quality variables for river segments in stream networks. Given the heterogeneous water dynamic patterns in large-scale basins, we introduce an additional meta-learning condition based on physical characteristics of stream segments, which allows learning different sets of initial parameters for different stream segments. Specifically, we develop a representation learning method that leverages physical simulations to embed the physical characteristics of each segment. The obtained embeddings are then used to cluster river segments and add the condition for the meta-learning process. We have tested the performance of the proposed method for predicting daily water temperature and streamflow for the Delaware River Basin (DRB) over a 14 year period. The results confirm the effectiveness of our method in predicting target variables even using sparse training samples. We also show that our method can achieve robust performance with different numbers of clusterings.

Machine Learning is beginning to provide state-of-the-art performance in a range of environmental applications such as streamflow prediction in a hydrologic basin. However, building accurate broad-scale models for streamflow remains challenging in practice due to the variability in the dominant hydrologic processes, which are best captured by sets of process-related basin characteristics. Existing basin characteristics suffer from noise and uncertainty, among many other things, which adversely impact model performance. To tackle the above challenges, in this paper, we propose a novel Knowledge-guided Self-Supervised Learning (KGSSL) inverse framework to extract system characteristics from driver(input) and response(output) data. This first-of-its-kind framework achieves robust performance even when characteristics are corrupted or missing. We evaluate the KGSSL framework in the context of stream flow modeling using CAMELS (Catchment Attributes and MEteorology for Large-sample Studies) which is a widely used hydrology benchmark dataset. Specifically, KGSSL outperforms baseline by 16% in predicting missing characteristics. Furthermore, in the context of forward modelling, KGSSL inferred characteristics provide a 35% improvement in performance over a standard baseline when the static characteristic are unknown.

Given raster imagery features and imperfect vector training labels with registration uncertainty, this paper studies a deep learning framework that can quantify and reduce the registration uncertainty of training labels as well as train neural network parameters simultaneously. The problem is important in broad applications such as streamline classification on Earth imagery or tissue segmentation on medical imagery, whereby annotating precise vector labels is expensive and time-consuming. However, the problem is challenging due to the gap between the vector representation of class labels and the raster representation of image features and the need for training neural networks with uncertain label locations. Existing research on uncertain training labels often focuses on uncertainty in label class semantics or characterizes label registration uncertainty at the pixel level (not contiguous vectors). To fill the gap, this paper proposes a novel learning framework that explicitly quantifies vector labels' registration uncertainty. We propose a registration-uncertainty-aware loss function and design an iterative uncertainty reduction algorithm by re-estimating the posterior of true vector label locations distribution based on a Gaussian process. Evaluations on real-world datasets in National Hydrography Dataset refinement show that the proposed approach significantly outperforms several baselines in the registration uncertainty estimations performance and classification performance.

This paper proposes a new data-driven method for predicting water temperature in stream networks with reservoirs. The water flows released from reservoirs greatly affect the water temperature of downstream river segments. However, the information of released water flow is often not available for many reservoirs, which makes it difficult for data-driven models to capture the impact to downstream river segments. In this paper, we first build a state-aware graph model to represent the interactions amongst streams and reservoirs, and then propose a parallel learning structure to extract the reservoir release information and use it to improve the prediction. In particular, for reservoirs with no available release information, we mimic the water managers' release decision process through a pseudo-prospective learning method, which infers the release information from anticipated water temperature dynamics. For reservoirs with the release information, we leverage a physics-based model to simulate the water release temperature and transfer such information to guide the learning process for other reservoirs. The evaluation for the Delaware River Basin shows that the proposed method brings over 10% accuracy improvement over existing data-driven models for stream temperature prediction when the release data is not available for any reservoirs. The performance is further improved after we incorporate the release data and physical simulations for a subset of reservoirs.

Accurate predictions of water temperature are the foundation for many decisions and regulations, with direct impacts on water quality, fishery yields, and power production. Building accurate broad-scale models for lake temperature prediction remains challenging in practice due to the variability in the data distribution across different lake systems monitored by static and time-series data. In this paper, to tackle the above challenges, we propose a novel machine learning based approach for integrating static and time-series data in deep recurrent models, which we call Invertibility-Aware-Long Short-Term Memory(IA-LSTM), and demonstrate its effectiveness in predicting lake temperature. Our proposed method integrates components of the Invertible Network and LSTM to better predict temperature profiles (forward modeling) and infer the static features (i.e., inverse modeling) that can eventually enhance the prediction when static variables are missing. We evaluate our method on predicting the temperature profile of 450 lakes in the Midwestern U.S. and report relative improvement of 4% to capture data heterogeneity and simultaneously outperform baseline predictions by 12% when static features are unavailable.

Direct numerical simulation (DNS) of turbulent flows is computationally expensive and cannot be applied to flows with large Reynolds numbers. Large eddy simulation (LES) is an alternative that is computationally less demanding, but is unable to capture all of the scales of turbulent transport accurately. Our goal in this work is to build a new data-driven methodology based on super-resolution techniques to reconstruct DNS data from LES predictions. We leverage the underlying physical relationships to regularize the relationships amongst different physical variables. We also introduce a hierarchical generative process and a reverse degradation process to fully explore the correspondence between DNS and LES data. We demonstrate the effectiveness of our method through a single-snapshot experiment and a cross-time experiment. The results confirm that our method can better reconstruct high-resolution DNS data over space and over time in terms of pixel-wise reconstruction error and structural similarity. Visual comparisons show that our method performs much better in capturing fine-level flow dynamics.

Mapping and monitoring crops is a key step to-wards sustainable intensification of agriculture and addressing global food security. A dataset like ImageNet that revolutionized computer vision applications can accelerate development of novel crop mapping techniques. Currently, the United States Department of Agriculture (USDA) annually releases the Cropland Data Layer (CDL) which contains crop labels at 30m resolution for the entire United States of America. While CDL is state of the art and is widely used for a number of agricultural applications, it has a number of limitations (e.g., pixelated errors, labels carried over from previous years and errors in classification of minor crops). In this work, we create a new semantic segmentation benchmark dataset, which we call CalCROP21, for the diverse crops in the Central Valley region of California at 10m spatial resolution using a Google Earth Engine based robust image processing pipeline and a novel attention based spatio-temporal semantic segmentation algorithm STATT. STATT uses re-sampled (interpolated) CDL labels for training, but is able to generate a better prediction than CDL by leveraging spatial and temporal patterns in Sentinel2 multi-spectral image series to effectively capture phenologic differences amongst crops and uses attention to reduce the impact of clouds and other atmospheric disturbances. We also present a comprehensive evaluation to show that STATT has significantly better results when compared to the resampled CDL labels. We have released the dataset and the processing pipeline code for generating the benchmark dataset.

The availability of massive earth observing satellite data provides huge opportunities for land use and land cover mapping. However, such mapping effort is challenging due to the existence of various land cover classes, noisy data, and the lack of proper labels. Also, each land cover class typically has its own unique temporal pattern and can be identified only during certain periods. In this article, we introduce a novel architecture that incorporates the UNet structure with Bidirectional LSTM and Attention mechanism to jointly exploit the spatial and temporal nature of satellite data and to better identify the unique temporal patterns of each land cover class. We compare our method with other state-of-the-art methods both quantitatively and qualitatively on two real-world datasets which involve multiple land cover classes. We also visualise the attention weights to study its effectiveness in mitigating noise and in identifying discriminative time periods of different classes. The code and dataset used in this work are made publicly available for reproducibility.

Knowledge discovery from spatial data is essential for many important societal applications including crop monitoring, solar energy estimation, traffic prediction and public health. This paper aims to tackle a key challenge posed by spatial data - the intrinsic spatial heterogeneity commonly embedded in their generation processes - in the context of deep learning. In related work, the early rise of convolutional neural networks showed the promising value of explicit spatial-awareness in deep architectures (i.e., preservation of spatial structure among input cells and the use of local connection). However, the issue of spatial heterogeneity has not been sufficiently explored. While recent developments have tried to incorporate awareness of spatial variability (e.g., SVANN), these methods either rely on manually-defined space partitioning or only support very limited partitions (e.g., two) due to reduction of training data. To address these limitations, we propose a Spatial-Net to simultaneously learn a space-partitioning scheme and a deep network architecture with a Significance-based Grow-and-Collapse (SIG-GAC) framework. SIG-GAC allows collaborative training between partitions and uses an exponential reduction tree to control the network size. Experiments using real-world datasets show that Spatial-Net can automatically learn the pattern underlying heterogeneous spatial process and greatly improve model performance.

This paper presents a physics-guided machine learning approach that incorporates partial differential equations (PDEs) in a graph neural network model to improve the prediction of water temperature in river networks. The standard graph neural network model often uses pre-defined edge weights based on distance or similarity measures. Such static graph structure can be limited in capturing multiple processes in a physical system that interact and evolve over time. The limitation to represent underlying physical processes can severely affect the performance of the predictive model, especially when we have access to limited training data. To better capture the dynamic interactions among multiple segments in a river network, we built a dynamic graph model, where the graph structure is driven by the PDE that describes underlying physical processes. We further combine the dynamic graph structure and the recurrent layers to model temporal dependencies and improve the prediction. We demonstrate the effectiveness of the proposed method in a subnetwork of the Delaware River Basin. In particular, we show that the proposed method outperforms existing physics-based and machine learning models in temperature prediction using sparse observation data for training. The proposed method has also been shown to produce better performance when generalized to different seasons.

Accurate prediction of water temperature in streams is critical for monitoring and understanding biogeochemical and ecological processes in streams. Stream temperature is affected by weather patterns (such as solar radiation) and water flowing through the stream network. Additionally, stream temperature can be substantially affected by water releases from man-made reservoirs to downstream segments. In this paper, we propose a heterogeneous recurrent graph model to represent these interacting processes that underlie stream-reservoir networks and improve the prediction of water temperature in all river segments within a network. Because reservoir release data may be unavailable for certain reservoirs, we further develop a data assimilation mechanism to adjust the deep learning model states to correct for the prediction bias caused by reservoir releases. Our evaluation for the Delaware River Basin has demonstrated the superiority of our proposed method over multiple existing methods. We have extensively studied the effect of the data assimilation mechanism under different scenarios.

This paper proposes a new physics-guided machine learning approach that incorporates the scientific knowledge in physics-based models into machine learning models. Physics-based models are widely used to study dynamical systems in a variety of scientific and engineering problems. Although they are built based on general physical laws that govern the relations from input to output variables, these models often produce biased simulations due to inaccurate parameterizations or approximations used to represent the true physics. In this paper, we aim to build a new data-driven framework to monitor dynamical systems by extracting general scientific knowledge embodied in simulation data generated by the physics-based model. To handle the bias in simulation data caused by imperfect parameterization, we propose to extract general physical relations jointly from multiple sets of simulations generated by a physics-based model under different physical parameters. In particular, we develop a spatio-temporal network architecture that uses its gating variables to capture the variation of physical parameters. We initialize this model using a pre-training strategy that helps discover common physical patterns shared by different sets of simulation data. Then we fine-tune it using limited observation data via a contrastive learning process. By leveraging the complementary strength of machine learning and domain knowledge, our method has been shown to produce accurate predictions, use less training samples and generalize to out-of-sample scenarios. We further show that the method can provide insights about the variation of physical parameters over space and time in two domain applications: predicting temperature in streams and predicting temperature in lakes.

Spatial data are ubiquitous, massively collected, and widely used to support critical decision-making in many societal domains, including public health (e.g., COVID-19 pandemic control), agricultural crop monitoring, transportation, etc. While recent advances in machine learning and deep learning offer new promising ways to mine such rich datasets (e.g., satellite imagery, COVID statistics), spatial heterogeneity – an intrinsic characteristic embedded in spatial data - poses a major challenge as data distributions or generative processes often vary across space at different scales, with their spatial extents unknown. Recent studies (e.g., SVANN, spatial ensemble) targeting this difficult problem either require a known space-partitioning as the input, or can only support very limited number of partitions or classes (e.g., two) due to the decrease in training data size and the complexity of analysis. To address these limitations, we propose a model-agnostic framework to automatically transform a deep learning model into a spatial-heterogeneity-aware architecture, where the learning of arbitrary space partitionings is guided by a learning-engaged generalization of multivariate scan statistic and parameters are shared based on spatial relationships. We also propose a spatial moderator to generalize learned space partitionings to new test regions. Experiment results on real-world datasets show that the spatial transformation and moderation framework can effectively capture flexibly-shaped heterogeneous footprints and substantially improve prediction performances.

Effective training of advanced ML models requires large amounts of labeled data, which is often scarce in scientific problems given the substantial human labor and material cost to collect labeled data. This poses a challenge on determining when and where we should deploy measuring instruments (e.g., in-situ sensors) to collect labeled data efficiently. This problem differs from traditional pool-based active learning settings in that the labeling decisions have to be made immediately after we observe the input data that come in a time series. In this paper, we develop a real-time active learning method that uses the spatial and temporal contextual information to select representative query samples in a reinforcement learning framework. To reduce the need for large training data, we further propose to transfer the policy learned from simulation data which is generated by existing physics-based models. We demonstrate the effectiveness of the proposed method by predicting streamflow and water temperature in the Delaware River Basin given a limited budget for collecting labeled data. We further study the spatial and temporal distribution of selected samples to verify the ability of this method in selecting informative samples over space and time.

This paper proposes a physics-guided machine learning approach that combines machine learning models and physicsbased models to improve the prediction of water flow and temperature in river networks.We first build a recurrent graph network model to capture the interactions among multiple segments in the river network.Then we transfer knowledge from physics-based models to guide the learning of the machine learning model.We also propose a new loss function that balances the performance over different river segments.We demonstrate the effectiveness of the proposed method in predicting temperature and streamflow in a subset of the Delaware River Basin.In particular, the proposed method has brought a 33%/14% accuracy improvement over the state-of-the-art physics-based model and 24%/14% over traditional machine learning models (e.g., LSTM) in temperature/streamflow prediction using very sparse (0.1%) training data.The proposed method has also been shown to produce better performance when generalized to different seasons or river segments with different streamflow ranges.

Cluster detection is important and widely used in a variety of applications, including public health, public safety, transportation, and so on. Given a collection of data points, we aim to detect density-connected spatial clusters with varying geometric shapes and densities, under the constraint that the clusters are statistically significant. The problem is challenging, because many societal applications and domain science studies have low tolerance for spurious results, and clusters may have arbitrary shapes and varying densities. As a classical topic in data mining and learning, a myriad of techniques have been developed to detect clusters with both varying shapes and densities (e.g., density-based, hierarchical, spectral, or deep clustering methods). However, the vast majority of these techniques do not consider statistical rigor and are susceptible to detecting spurious clusters formed as a result of natural randomness. On the other hand, scan statistic approaches explicitly control the rate of spurious results, but they typically assume a single “hotspot” of over-density and many rely on further assumptions such as a tessellated input space. To unite the strengths of both lines of work, we propose a statistically robust formulation of a multi-scale DBSCAN, namely Significant DBSCAN+, to identify significant clusters that are density connected. As we will show, incorporation of statistical rigor is a powerful mechanism that allows the new Significant DBSCAN+ to outperform state-of-the-art clustering techniques in various scenarios. We also propose computational enhancements to speed-up the proposed approach. Experiment results show that Significant DBSCAN+ can simultaneously improve the success rate of true cluster detection (e.g., 10–20% increases in absolute F1 scores) and substantially reduce the rate of spurious results (e.g., from thousands/hundreds of spurious detections to none or just a few across 100 datasets), and the acceleration methods can improve the efficiency for both clustered and non-clustered data.

The dynamic network data have become ubiquitous with the rapid development of Internet and smart devices. To effectively manage the involved vertices in networks, it is crucial to track the special community patterns and analyze the relationships among vertices. In this paper, we propose a new method to measure the coherence strength, also referred to as community consistency, of a community over a specific observation period. The measurement of community consistency is especially challenging given the dynamic community structure over time, i.e., vertices can leave their original communities and join new communities. In order to interpret the causes of evolving community structure and model the influence of evolving community structure on community consistency, we introduce an influence propagation process having a causal relation with the community consistency. Specifically, a generative model is proposed to combine the influence propagation and the network topological structure at each time step. The proposed influence-based approach for modeling evolution can be instantiated in a variety of real-world network data. The comprehensive experiments on both synthetic and real-world datasets demonstrate the superiority of the proposed framework in estimating the community consistency. Besides, we conduct a case study to show the effectiveness of the proposed method in real-world applications.

Personalization is essential for enhancing the customer experience in retrieval tasks. In this paper, we develop a novel method CA-GCN for personalized image retrieval in the Adobe Stock image system. The proposed method CA-GCN leverages user behavior data in a Graph Convolutional Neural Network (GCN) model to learn user and image embeddings simultaneously. Standard GCN performs poorly on sparse user-image interaction graphs due to the limited knowledge gain from less representative neighbors. To address this challenge, we propose to augment the sparse user-image interaction data by considering the similarities among images. Specifically, we detect clusters of similar images and introduce a set of hidden super-nodes in the graph to represent clusters. We show that such an augmented graph structure can significantly improve the retrieval performance on real-world data collected from Adobe Stock service. In particular, when testing the proposed method on real users' stock image retrieval sessions, we get promoted average click position from 70 to 51.

In the era of big data, data-driven methods have become increasingly popular in various applications, such as image recognition, traffic signal control, fake news detection. The superior performance of these data-driven approaches relies on large-scale labeled training data, which are probably inaccessible in real-world applications, i.e., "small (labeled) data" challenge. Examples include predicting emergent events in a city, detecting emerging fake news, and forecasting the progression of conditions for rare diseases. In most scenarios, people care about these small data cases most and thus improving the learning effectiveness of machine learning algorithms with small labeled data has been a popular research topic.

Many real-world phenomena are observed at multiple resolutions. Predictive models designed to predict these phenomena typically consider different resolutions separately. This approach might be limiting in applications where predictions are desired at fine resolutions but available training data is scarce. In this paper, we propose classification algorithms that leverage supervision from coarser resolutions to help train models on finer resolutions. The different resolutions are modeled as different views of the data in a multi-view framework that exploits the complementarity of features across different views to improve models on both views. Unlike traditional multi-view learning problems, the key challenge in our case is that there is no one-to-one correspondence between instances across different views in our case, which requires explicit modeling of the correspondence of instances across resolutions. We propose to use the features of instances at different resolutions to learn the correspondence between instances across resolutions using attention mechanism. Experiments on the real-world application of mapping urban areas using satellite observations and sentiment classification on text data show the effectiveness of the proposed methods.

In this work, we present a multi-view framework to classify spatio-temporal phenomena at multiple resolutions. This approach utilizes the complementarity of features across different resolutions and improves the corresponding models by enforcing consistency of their predictions on unlabeled data. Unlike traditional multi-view learning problems, the key challenge in our case is that there is a many-to-one correspondence between instances across different resolutions, which needs to be explicitly modeled. Experiments on the real-world application of mapping urban areas using spatial raster data-sets from satellite observations show the benefits of the proposed multi-view framework.

The last decade has witnessed a surge of interest in applying deep learning models for discovering sequential patterns from a large volume of data. Recent works show that deep learning models can be further improved by enforcing models to learn a smooth output distribution around each data point. This can be achieved by augmenting training data with slight perturbations that are designed to alter model outputs. Such adversarial training approaches have shown much success in improving the generalization performance of deep learning models on static data, e.g., transaction data or image data captured on a single snapshot. However, when applied to sequential data, the standard adversarial training approaches cannot fully capture the discriminative structure of a sequence. This is because real-world sequential data are often collected over a long period of time and may include much irrelevant information to the classification task. To this end, we develop a novel adversarial training approach for sequential data classification by investigating when and how to perturb a sequence for an effective data augmentation. Finally, we demonstrate the superiority of the proposed method over baselines in a diversity of real-world sequential datasets.

Detecting land use and land cover changes is critical to monitor natural resources and analyze global environmental changes. In this paper, we investigate the land cover detection using the remote sensing data from earth-observing satellites. Due to the natural disturbances, e.g., clouds and aerosoles, and the data acquisition errors by devices, remote sensing data frequently contain much noise. Also, many land covers cannot be easily identified in most dates of a year. Instead, they show distinctive temporal patterns only during certain period of a year, which is also referred to as the discriminative period. To address these challenges, we propose a novel framework which combines the spatial context knowledge with the LSTM-based temporal modeling for land cover detection. Specifically, the framework learns the spatial context knowledge selectively from its neighboring locations. Then we propose two approaches for discriminative period detection based on multi-instance learning and local attention mechanism, respectively. Our evaluations in two real-world applications demonstrate the effectiveness of the proposed method in identifying land covers and detecting discriminative periods.

Recent advances in processing remote sensing data have provided unprecedented potential for monitoring land covers. However, it is extremely challenging to deploy an automated monitoring system for different regions and across different years given the involved data heterogeneity over space and over time. The heterogeneity exists on two aspects. First, for many land covers, the distinguishing temporal patterns are only visible in certain discriminative period. Due to the change of weather conditions, the discriminative period can shift across space and time, which causes heterogeneity to the sequential data. Second, the collected remote sensing data are affected by acquisition devices and natural variables, e.g., precipitation and sunlight. In this paper, we introduce a novel framework to effectively detect land covers using the sequential remote sensing data. At the same time, we propose new learning strategies based on attention networks and domain adaptation to addresses the aforementioned challenges. The evaluation on two real-world applications - cropland mapping and burned area detection, demonstrate that the proposed method can effectively detect land covers under different weather conditions.

We propose a nonlinear manifold learning technique based on deep convolutional autoencoders that is appropriate for model order reduction of physical systems in complex geometries. Convolutional neural networks have proven to be highly advantageous for compressing data arising from systems demonstrating a slow-decaying Kolmogorov $n$-width. However, these networks are restricted to data on structured meshes. Unstructured meshes are often required for performing analyses of real systems with complex geometry. Our custom graph convolution operators based on the available differential operators for a given spatial discretization effectively extend the application space of deep convolutional autoencoders to systems with arbitrarily complex geometry that are typically discretized using unstructured meshes. We propose sets of convolution operators based on the spatial derivative operators for the underlying spatial discretization, making the method particularly well suited to data arising from the solution of partial differential equations. We demonstrate the method using examples from heat transfer and fluid mechanics and show better than an order of magnitude improvement in accuracy over linear methods.

Time series classification algorithms designed to use local context do not work on landcover classification problems where the instances of the two classes may often exhibit similar feature values due to the large natural variations in other land covers across the year and unrelated phenomena that they undergo. In this paper, we propose to learn discriminative patterns from the entire length of the time series, and use them as predictive features to identify the class of interest. We propose a novel neural network algorithm to learn the key signature of the class of interest as a function of the feature values together with the discriminative pattern made from that signature through the entire time series in a joint framework. We demonstrate the utility of this technique on the landcover classification application of burned area mapping that is of considerable societal importance.

Automated cropland monitoring can offer timely and reliable agricultural information, which is critical to meet the increasing demand for food supply and food security. In most cropland mapping tasks, domain researchers provide manually labeled training samples for several major crop types and request for identifying these major crops in a target region. However, it is very expensive to hire experts to label all the other land covers that exist in the target region. In this paper, we propose a novel learning framework to identify major crops without using labeled training samples for other land covers. For each major crop type, we train a one-class classification model based on sparse-autoencoder (SAE). Specifically, we utilize the high-resolution (~10m) remote sensing data as input features to classify each location either as one of major crop types or as other land covers. Many crop types are similar to each other in most dates of a year, but are distinguishable only during a short period in growing season. To better model the seasonal patterns of different crop types and to capture the their discriminative periods, we introduce a sliding window to cover different growing periods in a year and learn separate SAEs from these periods. Moreover, since remote sensing data are commonly disturbed by natural noise factors, we explore the spatial contiguity of unlabeled data in test region and incorporate it as a constraint in training process to further improve the performance. In this way, we utilize both labeled data and unlabeled data in a semi-supervised method to jointly train SAE. Finally, we design a mechanism to combine the SAEs trained for different crop types to make final classification decisions. We extensively evaluate the proposed method in mapping several major crops in Minnesota, US. The experimental results demonstrate that the proposed method can accurately map the extent of major crops, and capture the temporal growing patterns of different crops. Besides, the results confirm the effectiveness of spatial constraint in mitigating noise factors and making spatially contiguous classification. In addition, we give illustrative examples to show that the proposed method can help detect errors in existing cropland mapping product.

Land cover prediction is essential for monitoring global environmental change. Unfortunately, traditional classification models are plagued by temporal variation and emergence of novel/unseen land cover classes in the prediction process. In this paper, we propose an LSTM-based spatio-temporal learning framework with a dual-memory structure. The dual-memory structure captures both long-term and short-term temporal variation patterns, and is updated incrementally to adapt the model to the ever-changing environment. Moreover, we integrate zero-shot learning to identify unseen classes even without labelled samples. Experiments on both synthetic and real-world datasets demonstrate the superiority of the proposed framework over multiple baselines in land cover prediction.

Successful land cover prediction can provide promising insights in the applications where manual labeling is extremely difficult. However, traditional machine learning models are plagued by temporal variation and noisy features when directly applied to land cover prediction. Moreover, these models cannot take fully advantage of the spatio-temporal relationship involved in land cover transitions. In this paper, we propose a novel spatio-temporal framework to discover the transitions among land covers and at the same time conduct classification at each time step. Based on the proposed model, we incrementally update the model parameters in the prediction process, thus to mitigate the impact of the temporal variation. Our experiments in two challenging land cover applications demonstrate the superiority of the proposed method over multiple baselines. In addition, we show the efficacy of spatio-temporal transition modeling and incremental learning through extensive analysis.

Learning semantics based on context information has been researched in many research areas for decades. Context information can not only be directly used as the input data, but also sometimes used as auxiliary knowledge to improve existing models. This survey aims at providing a structured and comprehensive overview of the research on context learning. We summarize and group the existing literature into four categories, Explicit Analysis, Implicit Analysis, Neural Network Models, and Composite Models, based on the underlying techniques adopted by them. For each category, we talk about the basic idea and techniques, and also introduce how context information is utilized as the model input or incorporated into the model to enhance the performance or extend the domain of application as auxiliary knowledge. In addition, we discuss the advantages and disadvantages of each model from both the technical and practical point of view.

The development of Internet and social networks has provided more emerging network data which facilitates the dynamic network analysis. In this paper, we propose a new method to measure coherence strength, also referred to as community consistency, of a community under dynamic settings. In order to better interpret the influence of evolving community structure on community consistency, we model the problem as one of influence propagation processes having a causal relation with the community consistency. To this effect a generative model is proposed to combine the influence propagation and the network topological structure at each time stamp. Our comprehensive experiments on both synthetic and real-world datasets demonstrate the superiority of the proposed framework in estimating the community consistency.

The development of social networks has not only improved the online experience, but also stimulated the advances in knowledge mining so as to assist people in planning their offline social events. Users can explore their favorite events, such as celebrations and symposiums, through the pictures and the posts from their friends on social networks. An effective event recommendation can offer great convenience for both event organizers and participants, which yet remains extremely challenging due to a wide range of practical concerns. In this paper we propose a novel recommendation framework, which combines the information from multiple sources and establishes a connection between the online knowledge and the event participation.

The need for effective change detection is ever growing with more emerging large-scale spatial-temporal datasets that contain gridded time series data. To detect meaningful changing events with respect to our desired characteristics, in this paper we focus on the post-classification change detection problem which aims to apply change detection techniques on the time series of classification outputs. To study the challenges and to evaluate the performance, we apply the change detection techniques to an application of water monitoring using remote sensing data. Since the learning model can be affected by special properties of remote sensing data, the obtained classification outputs usually contain much noise. Therefore the successful change detection requires an elaborate mechanism to handle the time series of noisy classification outputs. To this end we propose to integrate spatial and temporal constraints into an optimization based change detection framework. The proposed framework mitigates the noise in the time series and can be efficiently solved by an EM-style algorithm. The extensive experimental results on both synthetic and real-world datasets very well demonstrate the effectiveness of the proposed method in detecting the water dynamics.

Plantation mapping is important for understanding deforestation and climate change. Most existing plantation products rely heavily on visual interpretation of satellite imagery, which results in both false positives and false negatives. In this paper we aim to design an automatic framework that map plantations in large regions. Conventional classification methods cannot be directly applied due to the lack of ground-truth data. To this end, we propose a novel method that learns from multiple imperfect annotators. Since each annotator's labeling accuracy varies across different land covers due to his expertise and reference imagery, we model the annotator's reliability level to be associated with different types of locations. On the other hand, the temporal variation of land covers also greatly impacts the performance of conventional learning model. Therefore we utilize the remote sensing data which are available at multiple periods of a year and extend our proposed method by incorporating multi-instance learning. Finally, we show the superiority of the proposed method over multiple baselines in both synthetic dataset and real-world dataset. In addition, through several case studies we demonstrate that our method can achieve a better balance of precision and recall than the existing plantation products.

Despite the ability to model many real world settings as a network, one major challenge in analyzing network data is that important and reliable links between objects are usually obscured by noisy information and hence not readily discernible. In this paper, we propose to detect these important and reliable links - significant edges, from a target network by using multiple auxiliary networks and a limited amount of labelled information. In this process, we first abstract the community knowledge learnt across target and auxiliary networks to detect significant patterns. The mined community knowledge captures the key profile of network relationships and thus can be used to determine whether an existing edge indicates a true or false relationship. Experiments on real world network data show that our two staged solution -- a joint matrix factorisation procedure followed by edge significance score ranking, accurately predicts significant edges in target network by jointly exploring the underlying knowledge embedded in both target and auxiliary networks.

The increasing popularity of mobile devices has brought severe challenges to device usability and big data analysis. In this paper we investigate the intellectual recommender system on cell phones by incorporating mobile data analysis. Nowadays with the development of smart phones, more and more applications have emerged on various areas, such as entertainment, education and health care. While these applications have brought great convenience to people's daily life, they also provide tremendous opportunities for analyzing users' interests. In this work we develop an Android background service to collect the user behaviors and analyze their preferences based on their Android application usage. As one of the most intuitive media for visual representation, videos with various types of contents are recommended to users based on a proposed graphical model. The proposed model jointly utilizes the textual descriptions of Android applications and videos, as well as the extracted video content based features. Besides, by analyzing the user's habit of application usage we seamlessly integrate the user's personal interests during the recommendation. The extensive comparisons to multiple baselines reveal the superiority of the proposed model on the recommendation quality. Furthermore, we conduct experiments on personalized recommendation to demonstrate the capacity of the proposed model in effectively analyzing the user's personal interests.

Morden physiological analysis begins to involve more and more types of information. Electroencephalogram (EEG) signals as a typical example is starting to be analyzed with facial expressions videos to detect emotions. Emotions play an important role in the daily life of human beings, the need and importance of automatic emotion recognition has grown with increasing role of human computer interface applications. In this paper, we concentrate on recognition of the emotions jointly from "inner" and "outer" reactions, which are electroencephalogram (EEG) signals and facial expression video. Due to the streaming nature of this problem, the data volume and velocity is very challenging. We address these challenges from the theoretic perspective and propose a real time algorithm based on EEG signals and synchronized facial video to learn feature vector jointly. Our algorithm consists of an unsupervisedly EEG dictionary component based on deep learning theorem, and a probability pooling component transforms a continuous sequential signal into an EEG "sentence" which consists of a sequence of EEG words. The EEG sentence is then jointly learned with video features into a new fixed length feature representation for emotion classification. We overcome several computational challenges on the data based on the idea of convolution and pooling, and we conduct extensive evaluation for each component of our model. We also demonstrate the state-of-the-art classification result on real-world dataset. The superior performances on the emotion recognition task indicates that 1) the natural language scenario can be applied in EEG sequences and 2) borrowing video modality can increase the overall performance.

Sequential data modeling has received growing interests due to its impact on real world problems. Sequential data is ubiquitous - financial transactions, advertise conversions and disease evolution are examples of sequential data. A long-standing challenge in sequential data modeling is how to capture the strong hidden correlations among complex features in high volumes. The sparsity and skewness in the features extracted from sequential data also add to the complexity of the problem. In this paper, we address these challenges from both discriminative and generative perspectives, and propose novel stochastic learning algorithms to model nonlinear variances from static time frames and their transitions. The proposed model, Deep Recurrent Network (DRN), can be trained in an unsupervised fashion to capture transitions, or in a discriminative fashion to conduct sequential labeling. We analyze the conditional independence of each functional module and tackle the diminishing gradient problem by developing a two-pass training algorithm. Extensive experiments on both simulated and real-world dynamic networks show that the trained DRN outperforms all baselines in the sequential classification task and obtains excellent performance in the regression task.

Community formation analysis of dynamic networks has been a hot topic in data mining which has attracted much attention. Recently, there are many studies which focus on discovering communities successively from consecutive snapshots by considering both the current and historical information. However, these methods cannot provide us with much historical or successive information related to the detected communities. Different from previous studies which focus on community detection in dynamic networks, we define a new problem of tracking the progression of the community strength-a novel measure that reflects the community robustness and coherence throughout the entire observation period. To achieve this goal, we propose a novel framework which formulates the problem as an optimization task. The proposed community strength analysis also provides foundation for a wide variety of related applications such as discovering how the strength of each detected community changes over the entire observation period. To demonstrate that the proposed method provides precise and meaningful evolutionary patterns of communities which are not directly obtainable from traditional methods, we perform extensive experimental studies on one synthetic and five real datasets: Social evolution, tweeting interaction, actor relationships, bibliography, and biological datasets. Experimental results show that the proposed approach is highly effective in discovering the progression of community strengths and detecting interesting communities.

Temporal analysis on dynamic networks has become a popularly discussed topic today, with more and more emerging data over time. In this paper we investigate the problem of detecting and tracking the variational communities within a given time period. We first define a metric to measure the strength of a community, called the normalized temporal community strength. And then, we propose our analysis framework. The community may evolve over time, either split to multiple communities or merge with others. We address the problem of evolutionary clustering with requirement on temporal smoothness and propose a revised soft clustering method based on non-negative matrix factorization. Then we use a clustering matching method to find the soft correspondence between different community distribution structures. This matching establishes the connection between consecutive snapshots. To estimate the variational rate and meanwhile address the smoothness during continuous evolution, we propose an objective function that combines the conformity of current variation and historical variational trend. In addition, we integrate the weights to the objective function to identify the temporal outliers. An iterative coordinate descent method is proposed to solve the optimization framework. We extensively evaluate our method with a synthetic dataset and several real datasets. The experimental results demonstrate the effectiveness of our method, which is greatly superior to the baselines on detection of the communities with significant variation over time.