The insufficient generalization of adaptive moment estimation (Adam) has hindered its broader application. Recent studies have shown that flat minima in loss landscapes are highly associated with improved generalization. Inspired by the filtering effect of integration operations on high-frequency signals, we propose multiple integral Adam (MIAdam), a novel optimizer that integrates a multiple integral term into Adam. This multiple integral term effectively filters out sharp minima encountered during optimization, guiding the optimizer towards flatter regions and thereby enhancing generalization capability. We provide a theoretical explanation for the improvement in generalization through the diffusion theory framework and analyze the impact of the multiple integral term on the optimizer's convergence. Experimental results demonstrate that MIAdam not only enhances generalization and robustness against label noise but also maintains the rapid convergence characteristic of Adam, outperforming Adam and its variants in state-of-the-art benchmarks.

Pedestrian localization has broad applications. However, achieving seamless positioning in complex environments remains a significant challenge due to factors like environmental occlusion and indoor-outdoor transitions. This article proposes a combined localization framework using global navigation satellite system (GNSS), inertial measurement unit (IMU), ultrawideband (UWB), and MAP data to address this issue. In the framework, a three-step GNSS localization method is proposed to enhance accuracy and stability by eliminating nonline-of-sight (NLOS) effects and filtering anomalies. Inertial navigation system (INS) and pedestrian dead reckoning (PDR) models are integrated to improve local precision and stability. UWB, known for its superior localization capabilities, has been integrated into some smartphones and is promising for large-scale indoor applications. This article presents a gradient descent-based optimal anchor deployment algorithm and localization methods for varying anchor quantities. Additionally, a nonstationary Markov model fuses map data for trajectory prediction and correction. Finally, anomaly detection filtering and low-pass filtering are applied to the corrected trajectories, yielding high accuracy and stability. Experiments on the Honor V20 and iQOO U3 smartphones showed significant improvements compared to the baseline algorithm. Outdoor root mean square error (RMSE) was reduced by 44.4% (10.83 m) for the Honor V20 and 40.0% (10.77 m) for the iQOO U3. Additionally, outdoor standard deviation (STD) improved by 43.9% (4.80 m) and 43.5% (5.50 m), respectively. In transitional areas, RMSE improved by 4.61% (10.77 m) and 19.6% (4.76 m), respectively. Indoor RMSE also showed improvements, with the Honor V20 reducing it by 8.00% (3.79 m) and the iQOO U3 by 5.78% (3.91 m).

Theoretical and empirical evidence highlights a positive correlation between the flatness of loss landscapes around minima and generalization. However, most current approaches that seek to find flat minima either incur high computational costs or struggle to balance generalization, training stability, and convergence. This work proposes reshaping the loss landscape to induce the optimizer toward flat regions, an approach that has negligible computational costs and does not compromise training stability, convergence, or efficiency. We focus on nonlinear, loss-dependent reshaping functions underpinned by theoretical insights to reshape the loss landscape. To design these functions, we first identify where and how these functions should be applied. With the aid of recently developed tools in stochastic optimization, theoretical analysis shows that steepening the low-loss landscape improves the rate of sharp minimum escape while flattening the high- and ultralow-loss landscapes enhances training stability and optimization performance, respectively. Simulations and experiments reveal that the subtly designed reshaping functions not only induce optimizers to find flat minima and improve generalization performance but also stabilize training, promote optimization, and keep efficiency. Our approach is evaluated on image classification, adversarial robustness, and natural language processing (NLP) tasks and achieves significant improvement in generalization performance with negligible computational cost. We believe that the new perspective introduced in this work will broadly impact the field of deep neural network training. The code is available at https://github.com/LongJin-lab/LLR.

Redundant manipulators have been widely used in various industries whose applications not only improve production efficiency and reduce manual labor but also promote innovation in robotics and artificial intelligence. Kinematic control plays a fundamental and crucial role in robot control. Over the past few decades, numerous motion control schemes have been proposed and applied to trajectory tracking tasks. However, most of these schemes do not consider the introduction of sparsity into the motion control of redundant manipulators, resulting in excessive joint movements, which not only consume extra energy but also increase the risk of unexpected collisions in complex environments. To solve this problem, we transform the issue of increasing the sparsity into a nonconvex optimization problem. Furthermore, a collective neural dynamics for sparse motion planning (CNDSMP) scheme for motion planning of redundant manipulators is proposed. By incorporating sparsity into the control scheme, the excessive joint movements are minimized, leading to improved efficiency and reduced collision risks. Through simulations, comparisons, and physical experiments, the effectiveness and superiority of the proposed scheme are demonstrated.

Inter-robot relative positions are crucial for executing various multirobot missions, such as formation maneuvering and collaborative inspection. However, the current sensing technology usually provides part of relative position information, such as inter-robot distances, bearings and angles. This prompts the study of determining inter-robot relative positions, i.e., relative localization, from these partial measurements. Based on the existing results of static networks' localizability and mobile robots' relative localization, we propose a novel concept,relative localizabilityto describe whether a multirobot system isrelatively localizable. Given each robot's self-displacement measurements and inter-robot partial measurements in$d$($d\leq 4$) sampling instants, we show that a multirobot system's relative localization can be achieved in a purelyalgebraicanddistributedmanner, in which the multirobot system is said to be$d$-step relatively localizable. To make the results more general, we consider that the multirobot system consists of landmarks, leaders, and followers, and that the inter-robot measurements can be distances, bearings or angles. When robots' coordinate frames have different orientations, we show that the given local measurements can be used to determine robots' relative positions and their coordinate frames' relative orientations simultaneously. Simulations and experiments of relative localization for ground robots are conducted to validate the obtained results.

The formation control of multi-agent systems has increasingly drawn attention for fulfilling numerous emerging applications and services. To achieve high-accuracy formation, the location awareness of all agents becomes an essential requirement. In this paper, we address the problem of network localization and formation control in a cooperative system with asynchronous agents. In particular, we formulate the joint localization and synchronization of agents as a statistical inference problem. The underlying probabilistic model is represented by a factor graph from which a message-passing algorithm is designed that computes approximations of the marginals of unknown variables, i.e. agents’ locations and clock offsets. Due to the Euclidean-norm operator involved in their computation no parametric closed-form expressions of the messages exist. As a compromise, implemented message-passing methods therefore resort to approximations of these messages. Conventional methods rely either on a first-order Taylor expansion of the norm operation or on non-parametric representations, e.g. by means particle filters (PFs), to compute such approximations. However, the former approach suffers from poor performance while the latter one experiences high complexity. The proposed message-passing algorithm in this paper is parametric. Specifically, it passes Gaussian messages that can be essentially obtained by suitably augmenting the factor graph and applying on it a hybrid method for combining belief propagation and variational message passing. Subsequently, the agents can exploit the estimated locations for determining the control policy. Two types of control policy are designed based on the optimization of a generalized cost function. We show that the proposed scheme enjoys a reduced complexity for multi-agent localization while achieving the desired formation with excellent accuracy.

Various quadrotor drones have been developed in recent years, mainly focusing on either improving maximum thrust per platform area or flight maneuverability. Evidently, achieving both advantages simultaneously is a challenging task, since they call for opposing rotor requirements. Specifically, improving the drone's maximum thrust per platform area mainly requires reducing the number of rotors to make way for larger and more powerful rotors. While this can be an effective method to increase overall thrust, improving flight maneuverability requires a greater number of rotors to generate larger rotating torques or to increase the thrust vectoring capability. To address this challenge, we design a novel coaxial drone with two contra-rotating rotors for high thrust efficiency while enabling independent dual-axis rotor rotation to maintain maneuverability along the roll and pitch axes. The thrust vectoring capability is provided by two dedicated servo motors connected vertically in series with the coaxial propellers to produce a compact and elongated fuselage frame. A nonlinear flight model in six degrees of freedom is developed for the underactuated system, incorporating four control inputs from the two propellers and servos respectively. Consequently, a nonlinear control allocation approach is proposed such that the drone can produce a desired control force and yaw torque to stabilize the drone's position and yaw angle. For the uncontrolled roll and pitch dynamics, a damping component is added such that the roll and pitch angular velocities can also be stabilized. Both numerical simulations and real experiments are conducted to validate the design of the drone and the effectiveness of the proposed control strategy.

In industrial applications where computational resources are finite and data noises are prevalent, the representation power of deep neural networks (DNNs) is crucial. Traditional network structures often require a significant increase in the parameter amount to enhance the representation power, making it difficult to achieve effective representation under parameter amount constraints. In order to alleviate this problem, this work leverages the ordinary differential equation (ODE) interpretation of deep residual networks, elucidating the relationship between the fine-grained connectivity modes of blocks in DNNs and the representation power. We build a bridge from the order of numerical methods and the order of ODEs to the representation power of DNNs. Besides, we show that higher-order ODEs can be approximated by k-step methods incorporating trainable coefficients. Empirically, we validate our theoretical insights by demonstrating the superior representation power of our proposed network structures through enhanced performance on industrial tasks, such as surface defect detection, critical temperature prediction of superconductors, and image classification under noises. The proposed method provides a new approach to the design of network structures for robust and accurate DNNs, enhancing the representation power with a negligible number of additional parameters. The code is publicly available athttps://github.com/LongJin-lab/Order-and-Representation-Power.

Deep neural networks often suffer from poor performance or even training failure due to the ill-conditioned problem, the vanishing/exploding gradient problem, and the saddle point problem. In this article, a novel method by acting the gradient activation function (GAF) on the gradient is proposed to handle these challenges. Intuitively, the GAF enlarges the tiny gradients and restricts the large gradient. Theoretically, this article gives conditions that the GAF needs to meet and, on this basis, proves that the GAF alleviates the problems mentioned above. In addition, this article proves that the convergence rate of SGD with the GAF is faster than that without the GAF under some assumptions. Furthermore, experiments on CIFAR, ImageNet, and PASCAL visual object classes confirm the GAF's effectiveness. The experimental results also demonstrate that the proposed method is able to be adopted in various deep neural networks to improve their performance. The source code is publicly available at https://github.com/LongJin-lab/Activated-Gradients-for-Deep-Neural-Networks.

The question of what kind of convolutional neural network (CNN) structure performs well is fascinating. In this work, we move toward the answer with one more step by connecting zero stability and model performance. Specifically, we found that if a discrete solver of an ordinary differential equation is zero stable, the CNN corresponding to that solver performs well. We first give the interpretation of zero stability in the context of deep learning and then investigate the performance of existing first- and second-order CNNs under different zero-stable circumstances. Based on the preliminary observation, we provide a higher-order discretization to construct CNNs and then propose a zero-stable network (ZeroSNet). To guarantee zero stability of the ZeroSNet, we first deduce a structure that meets consistency conditions and then give a zero stable region of a training-free parameter. By analyzing the roots of a characteristic equation, we theoretically obtain the optimal coefficients of feature maps. Empirically, we present our results from three aspects: We provide extensive empirical evidence of different depth on different datasets to show that the moduli of the characteristic equation's roots are the keys for the performance of CNNs that require historical features; Our experiments show that ZeroSNet outperforms existing CNNs which is based on high-order discretization; ZeroSNets show better robustness against noises on the input. The source code is available at https://github.com/logichen/ZeroSNet.

This paper focuses on the simulation and comparison of different types of first-order decentralized sliding mode estimators (FDSMEs). From the previous works, three types of FDSMEs were presented and applied to solve the cooperative control problems. Utilizing the FDSME, a finite-time leader-follower tracking control algorithm is proposed for a networked single-integrator vehicle system. Then based on the existing structure of FDSME, a new compound FDSME is developed to improve the estimation performance. Simulation and comparison of all the presented FDSMEs are given in detail to evaluate the theoretical results. Finally, regulation rules of the parameters in the compound FDSME are summarized according to the simulation results.

This paper is concerned with the stability analysis and L1-gain characterization for positive Takagi-Sugeno fuzzy systems subject to impulse effects. The main tool utilized is the construction of an impulse-time-dependent discretized copositive Lyapunov function where the impulse interval partitioning idea is exploited to derive less conservative results. For the disturbance-free case, stability criteria are developed to guarantee global exponential stability of the concerned systems under ranged and constant dwell-time constraints. The analysis of the influence of the discretizing parameter on the reduction of conservativeness is performed. We further employ the proposed impulse-time-dependent discretized copositive Lyapunov function to analyze the L1-gain performance for the case where external disturbances are considered. By performing a grid search for a scalar in an interval, all the proposed criteria can be checked via linear programming. Finally, numerical simulations are discussed to illustrate the effectiveness of the theoretical results.