Solving partial differential equations (PDEs) using neural networks has become a central focus in scientific machine learning. Training neural networks for singularly perturbed problems is particularly challenging due to certain parameters in the PDEs that introduce near-singularities in the loss function. In this study, we overcome this challenge by introducing a novel method based on homotopy dynamics to effectively manipulate these parameters. From a theoretical perspective, we analyze the effects of these parameters on training difficulty in these singularly perturbed problems and establish the convergence of the proposed homotopy dynamics method. Experimentally, we demonstrate that our approach significantly accelerates convergence and improves the accuracy of these singularly perturbed problems. These findings present an efficient optimization strategy leveraging homotopy dynamics, offering a robust framework to extend the applicability of neural networks for solving singularly perturbed differential equations.

Constructing the architecture of a neural network is a challenging pursuit for the machine learning community, and the dilemma of whether to go deeper or wider remains a persistent question. This paper explores a comparison between deeper neural networks (DeNNs) with a flexible number of layers and wider neural networks (WeNNs) with limited hidden layers, focusing on their optimal generalization error in Sobolev losses. Analytical investigations reveal that the architecture of a neural network can be significantly influenced by various factors, including the number of sample points, parameters within the neural networks, and the regularity of the loss function. Specifically, a higher number of parameters tends to favor WeNNs, while an increased number of sample points and greater regularity in the loss function lean towards the adoption of DeNNs. We ultimately apply this theory to address partial differential equations using deep Ritz and physics-informed neural network (PINN) methods, guiding the design of neural networks.

Solving nonlinear partial differential equations (PDEs) with multiple solutions is essential in various fields, including physics, biology, and engineering. However, traditional numerical methods, such as finite element and finite difference methods, often face challenges when dealing with nonlinear solvers, particularly in the presence of multiple solutions. These methods can become computationally expensive, especially when relying on solvers like Newton's method, which may struggle with ill-posedness near bifurcation points. In this paper, we propose a novel approach, the Newton Informed Neural Operator, which learns the Newton solver for nonlinear PDEs. Our method integrates traditional numerical techniques with the Newton nonlinear solver, efficiently learning the nonlinear mapping at each iteration. This approach allows us to compute multiple solutions in a single learning process while requiring fewer supervised data points than existing neural network methods.

This paper addresses the problem of nearly optimal Vapnik--Chervonenkis dimension (VC-dimension) and pseudo-dimension estimations of the derivative functions of deep neural networks (DNNs). Two important applications of these estimations include: 1) Establishing a nearly tight approximation result of DNNs in the Sobolev space; 2) Characterizing the generalization error of machine learning methods with loss functions involving function derivatives. This theoretical investigation fills the gap of learning error estimations for a wide range of physics-informed machine learning models and applications including generative models, solving partial differential equations, operator learning, network compression, distillation, regularization, etc.

Holonic manufacturing systems (HMS) provide a flexible and decentralized manufacturing environment to accommodate changes dynamically. This paper presents a framework to model and control HMS based on Petri net and MAS theory. A time Petri net (TPN) model was proposed to achieve this goal. A TPN represents a set of established contracts among the agents in HMS to fulfil an order. A scheduling architecture which integrates TPN models and AI techniques is proposed. By introducing dynamic individuals into the reproducing pool randomly according to their fitness, a variable population-size genetic algorithm is presented to enhance the convergence speed of GA. Based on the novel GA and the particle swarm optimization (PSO) algorithms, a hybrid PSO-GA algorithm (HPGA) is also proposed in this paper. Simulation results show that the proposed method is effective for the optimization problems

The dynamic re-organization of holons is a key element of current research on HMS. Dynamic intelligent reconfiguration is important for holonic control. This paper extends the mechanism of virtual clustering to the reorganization of holons and uses contract net-based task allocation protocol to efficiently deal with the communication and coordination problems during task-oriented clustering. The PSO-based virtual clustering optimization algorithm described in this paper can solve the optimization problem of task allocation on the basis of global optimization. The hybrid algorithm combines the high speed of PSO with the powerful ability to avoid being trapped in local minimum of SA. We compare the PSO algorithm to both GA and SA models, the simulation results show that the proposed model and algorithm are effective