Stochastic differential equation networks (SDENets), a subset of continuous-time neural networks, offer the natural ability to model continuous time-series data with greater expressivity than discrete-time neural networks. However, SDENets face challenges related to stability and high computational overhead. In this paper, we introduce SE-SDENet, a stable and efficient variant of SDENet. Leveraging the inherent capability of SDENets to model randomness in time-series data, we establish a theoretical framework that ensures SE-SDENet's stability during training by regulating the dynamics of each neuron. Additionally, we propose an efficient training and inference framework that enables SE-SDENet to achieve low forward-pass complexity and to dynamically adjust its complexity at run-time. Evaluation with four datasets and four edge devices demonstrates that SE-SDENet achieves a 6.6x higher throughput than the solver-based SDENet and exhibits improved stability in handling noisy data and long-term predictions. A validation on a robot vehicle shows that SE-SDENet can dynamically adjust its complexity at run-time to meet varying resource constraints.

Inspired by the Kolmogorov-Arnold representation theorem and Ku̇rková's principle of using approximate representations, we propose the Ku̇rková-Kolmogorov-Arnold Network (KKAN), a new two-block architecture that combines robust multi-layer perceptron (MLP) based inner functions with flexible linear combinations of basis functions as outer functions. We first prove that KKAN is a universal approximator, and then we demonstrate its versatility across scientific machine-learning applications, including function regression, physics-informed machine learning (PIML), and operator-learning frameworks. The benchmark results show that KKANs outperform MLPs and the original Kolmogorov-Arnold Networks (KANs) in function approximation and operator learning tasks and achieve performance comparable to fully optimized MLPs for PIML. To better understand the behavior of the new representation models, we analyze their geometric complexity and learning dynamics using information bottleneck theory, identifying three universal learning stages, fitting, transition, and diffusion, across all types of architectures. We find a strong correlation between geometric complexity and signal-to-noise ratio (SNR), with optimal generalization achieved during the diffusion stage. Additionally, we propose self-scaled residual-based attention weights to maintain high SNR dynamically, ensuring uniform convergence and prolonged learning.

Artificial Intelligence (AI) based feature extractors provide new possibility in the localization problem because of trainable characteristic. In this paper, the confidence information from AI learning process is used to further improve the accuracy. By resampling interest points based on different confidence thresholds, we are able to pixel-stack highlyconfident interest points to increase their bias for pose optimization. Then, the complementary descriptors are used to describe the pixel stacked interest points. As the result, the proposed Keypoint Resampling and Fusion (KRF) method improves the absolute trajectory error by 40% over state-of the-art vision SLAM algorithm on TUM Freiburg dataset. It is also more robust against tracking lost, and is compatible with existing optimizers.

This paper concerns multiphase piecewise smooth image segmentation with intensity inhomogeneities. Traditional methods based on the Mumford-Shah (MS) model require solving complicated diffusion equations evolving in irregular subdomains, leading to significant difficulties in efficient and accurate segmentation, especially in multiphase scenarios. In this paper, we propose a general framework to modify the MS model by using smoothing operators that can avoid the complicated implementation and inaccurate segmentation of traditional approaches. A detailed analysis connecting the smoothing operators and the diffusion equations is given to justify the modification. In addition, we present an efficient algorithm based on the direct augmented Lagrangian method, which requires fewer parameters than the commonly used augmented Lagrangian method. Typically, the smoothing operator in the general model is chosen to be Gaussian kernel, the bilateral kernel, and the directional diffusion kernel, respectively. Ample numerical results are provided to demonstrate the efficiency and accuracy of the modified model and the proposed minimization algorithm through various comparisons with existing approaches.

Variational image segmentation based on the Mumford-Shah model requires to solve the heat diffusion equations in evolving irregular subdomains. It brings about significant difficulties in efficient and accurate segmentation, especially, in multi-phase scenarios. In this paper, we propose a new Mumford-Shah type model involving a smoothing operator, acting a similar role as the diffusion process and avoiding complicated computation. We introduce two simplified numerical models based on the Gaussian operator and the bilateral operator respectively, and provide efficient numerical algorithms to solve both segmentation problems. Numerical validation experiments on various images show better accuracy and robustness over existing methods.

This paper is concerned with overlapping domain decomposition methods (DDMs), based on successive subspace correction (SSC) and parallel subspace correction (PSC), for the Rudin--Osher--Fatemi (ROF) model in image restoration. In contrast to recent attempts, we work with a dual formulation of the ROF model, where one significant difficulty resides in the decomposition of the global constraint of the dual variable. We introduce a stable “unity decomposition” using a set of “partition of unity functions,” which naturally leads to overlapping DDMs based on the dual formulation. The main objective of this paper is to rigorously analyze the convergence of the SSC and PSC algorithms and derive the rate of convergence $O(n^{-1/2})$, where $n$ is the number of iterations. Moreover, we characterize the explicit dependence of the convergence rate on the subdomain overlapping size and other important parameters. To the best of our knowledge, such a convergence rate has not yet been claimed for domain decomposition related algorithms for the ROF model.

Image segmentation is to separate the image domain into sub-domains according to some optimization rule. Variational level-set image segmentation based on the Mumford-Shah model, together with level set formulation, leads to complicate implementation and expensive computational time. To overcome the difficulty in solving the complex coupled level set functions and handle any number of phases in the image, we improve the region-base active contour model in this paper. We formulate the image segmentation as a new multiphase minimization problem to handle images containing any integer phases of intensities. We also propose efficient numerical algorithms to solve such optimization problem without extra steps of convex relaxation. Numerical validation experiments on various images show better accuracy and robustness over existing methods.

Image segmentation is to separate the image domain into local regions. Implicit active contour models via the level-set method can provide smooth and closed contours. The major category is based on the Mumford-Shah image component decomposition model. Approximating the image domain by a set of homogeneous regions, segmentation based on the piecewise constant (PC) models fail in intensity inhomogeneous images. Requiring intensity smoothness in local regions, segmentation based on the piecewise smooth (PS) models can perform better for intensity inhomogeneity than those PC based. Existing PS-based techniques are inefficient and not robust to complicate intensity scenarios with noise. Here we introduce a new functional model by decomposing an image into three parts: PS components, PC components and noise components. We convexify the decomposition model by incorporating with relaxation techniques and optimize the PS components over the whole image domain. New numerical algorithms are also proposed to implement the above approaches efficiently. Numerical validation experiments show that the proposed approaches can achieve much faster, more robust and more accurate image segmentation than existing arts.

This paper intends to extend the minimization algorithm developed by Bae, Yuan and Tai [IJCV, 2011] in several directions. First, we propose a new primal-dual approach for global minimization of the continuous Potts model with applications to the piecewise constant Mumford-Shah model for multiphase image segmentation. Different from the existing methods, we work directly with the binary setting without using convex relaxation, which is thereby termed as a direct approach. Second, we provide the sufficient and necessary conditions to guarantee a global optimum. Moreover, we provide efficient algorithms based on a reduction in the intermediate unknowns from the augmented Lagrangian formulation. As a result, the underlying algorithms involve significantly fewer parameters and unknowns than the naive use of augmented Lagrangian-based methods; hence, they are fast and easy to implement. Furthermore, they can produce global optimums under mild conditions.