This paper presents an evolvable conditional diffusion method such that black-box, non-differentiable multi-physics models, as are common in domains like computational fluid dynamics and electromagnetics, can be effectively used for guiding the generative process to facilitate autonomous scientific discovery. We formulate the guidance as an optimization problem where one optimizes for a desired fitness function through updates to the descriptive statistic for the denoising distribution, and derive an evolution-guided approach from first principles through the lens of probabilistic evolution. Interestingly, the final derived update algorithm is analogous to the update as per common gradient-based guided diffusion models, but without ever having to compute any derivatives. We validate our proposed evolvable diffusion algorithm in two AI for Science scenarios: the automated design of fluidic topology and meta-surface. Results demonstrate that this method effectively generates designs that better satisfy specific optimization objectives without reliance on differentiable proxies, providing an effective means of guidance-based diffusion that can capitalize on the wealth of black-box, non-differentiable multi-physics numerical models common across Science.

Transfer multiobjective optimization promises sample-efficient discovery of near Pareto-optimal solutions to a target task by utilizing experiential priors from related source tasks. In this paper, we show that in domains where evaluation data is at a premium, e.g., in scientific and engineering disciplines involving time-consuming computer simulations or complex real-world experimentation, knowledge transfer through surrogate models can be pivotal in saving sample evaluation costs. While state-of-the-art algorithms (without transfer) typically assume budgets in the order of only a few hundred evaluations, we seek to explore how far we can get on even tighter budgets. The uniqueness of our proposed Expensive Transfer Evolutionary Multiobjective Optimizer (ExTrEMO) is that it can maximally utilize external information from hundreds of source datasets, including those that may be negatively correlated with the target task. This is achieved by melding evolutionary search with factorized transfer Gaussian process surrogates, capturing varied source-target correlations in potentially decentralized computation environments. We provide a regret bound analysis for ExTrEMO that translates to a theoretical proof of increasingly faster convergence as a result of multi-source transfers. The theory is experimentally verified on benchmark functions and toward accelerated design of biomedical microdevices. We release our code at https://github.com/LiuJ-2023/ExTrEMO.

Recent rapid advances in artificial intelligence methods and computational resources have opened up the possibility of solving complex problems in physics with machine learning. While promising, the training of performant deep learning models remains challenging, especially as the choice of various neural architectures or model hyper-parameters can all greatly impact model performance. Hence, it is now fairly common for researchers to systematically conduct either a neural architecture search (NAS) or hyper-parameter tuning as part of the model training process. However, these usually involve repeated model training and evaluation (typically via a cross-validation framework) and can be very computationally expensive. While there have been training-free heuristics proposed for NAS in recent years, this remains an active area of work, and much remains unresolved. Concurrently, physics-informed neural networks (PINNs) have been proposed in recent years as an attractive means of incorporating physics-derived knowledge into a deep learning framework for the modelling of real-world physical systems. However, recent literature has begun to emerge on the additional complexity of training PINNs due to the addition of these physics-derived equations (typically in the form of partial differential equations, PDEs). The extended optimization trajectories observed further exacerbate the aforementioned computational cost of NAS or hyper-parameter search. Hence, in this work, we propose a first demonstration of the utility of training-free heuristics for accelerating the selection of model parameters. In particular, we assess the correlation between basic concepts espoused in Nyquist-Shannon sampling, and the appropriate selection of mini-batch size for PINN training with stochastic gradient descent (specifically, the state-of-the-art ADAM algorithm). As a proof-of-concept, this paper focuses on training PINNs to predict the solution to a canonical 2D Helmholtz equation with varying complexity (as measured by the frequency of the signal present), and we evaluate and report the influence of different mini-batch size parameters on the effective training of PINNs. The results show that PINN model performance exhibits a step-like phase transition with increased mini-batch size during training. Critically, we note that the minimum mini-batch size used increases proportionally with the presence of higher frequencies in the physical system being modelled, as is intuitively described by the Nyquist-Shannon sampling theorem. This both shows the utility of similar potential heuristics for the training-free selection of hyper-parameters when training PINNs and suggests the potential for insights towards accelerated design and training of PINNs for physical systems from prior theoretical concepts in signal processing and reconstruction.

We present a novel loss formulation for efficient learning of complex dynamics from governing physics, typically described by partial differential equations (PDEs), using physics-informed neural networks (PINNs). In our experiments, existing versions of PINNs are seen to learn poorly in many problems, especially for complex geometries, as it becomes increasingly difficult to establish appropriate sampling strategy at the near boundary region. Overly dense sampling can adversely impede training convergence if the local gradient behaviors are too complex to be adequately modelled by PINNs. On the other hand, if the samples are too sparse, existing PINNs tend to overfit the near boundary region, leading to incorrect solution. To prevent such issues, we propose a new Boundary Connectivity (BCXN) loss function which provides linear local structure approximation (LSA) to the gradient behaviors at the boundary for PINN. Our BCXN-loss implicitly imposes local structure during training, thus facilitating fast physics-informed learning across entire prob-lem domains with order of magnitude sparser training samples. This LSA-PINN method shows a few orders of magnitude smaller errors than existing methods in terms of the standard L2-norm metric, while using dramatically fewer training samples and iterations. Our proposed LSA-PINN does not pose any requirement on the differentiable property of the networks, and we demonstrate its benefits and ease of implementation on both multi-layer perceptron and convolutional neural network versions as commonly used in current PINN literature.

Traffic emissions are known to contribute significantly to air pollution around the world, especially in heavily urbanized cities such as Singapore. It has been previously shown that the particulate pollution along major roadways exhibit strong correlation with increased traffic during peak hours, and that reductions in traffic emissions can lead to better health outcomes. However, in many instances, obtaining proper counts of vehicular traffic remains manual and extremely laborious. This then restricts one's ability to carry out longitudinal monitoring for extended periods, for example, when trying to understand the efficacy of intervention measures such as new traffic regulations (e.g. car-pooling) or for computational modelling. Hence, in this study, we propose and implement an integrated machine learning pipeline that utilizes traffic images to obtain vehicular counts that can be easily integrated with other measurements to facilitate various studies. We verify the utility and accuracy of this pipeline on an open-source dataset of traffic images obtained for a location in Singapore and compare the obtained vehicular counts with collocated particulate measurement data obtained over a 2-week period in 2022. The roadside particulate emission is observed to correlate well with obtained vehicular counts with a correlation coefficient of 0.93, indicating that this method can indeed serve as a quick and effective correlate of particulate emissions.

Numerical weather prediction (NWP) and machine learning (ML) methods are popular for weather forecasting. However, NWP models have multiple possible physical parameterizations, which requires site-specific NWP optimization. This is further complicated when regional NWP models are used with global climate models, each with multiple possible parameterizations. In this study, a hybrid numerical-statistical approach is proposed and evaluated for four radiation models. Weather Research and Forecasting (WRF) model is run in both global and regional mode to provide an estimate for solar irradiance. This estimate is then post-processed using ML to provide a final prediction. Normalized root-mean-square error from WRF is reduced by up to 40-50% with this ML error correction model. Results obtained using CAM, GFDL, New Goddard and RRTMG radiation models were comparable after this correction, negating the need for WRF parameterization tuning. Other models incorporating nearby locations and an ensemble set-up are also evaluated, although they produced much smaller improvements.