We investigate the universal approximation property (UAP) of transformer-type architectures, providing a unified theoretical framework that extends prior results on residual networks to models incorporating attention mechanisms. Our work identifies token distinguishability as a fundamental requirement for UAP and introduces a general sufficient condition that applies to a broad class of architectures. Leveraging an analyticity assumption on the attention layer, we can significantly simplify the verification of this condition, providing a non-constructive approach in establishing UAP for such architectures. We demonstrate the applicability of our framework by proving UAP for transformers with various attention mechanisms, including kernel-based and sparse ones. The corollaries of our results either generalize prior works or establish UAP for architectures not previously covered. Furthermore, our framework offers a principled foundation for designing novel transformer architectures with inherent UAP guarantees, including those with specific functional symmetries. We propose examples to illustrate these insights.

Abstract. Image restoration is critical across many fields, as it addresses the challenge of recovering clear images from degraded data. Two prominent approaches to this problem are wavelet-based methods and partial differential equation (PDE) models. Wavelet methods can be viewed as discrete analogs of PDE models, and through asymptotic analysis, wavelet models often converge to PDE-based approaches such as the total variation model. Wavelet methods are known for their simple implementation and multiscale time-frequency analysis, while PDE models provide a geometric interpretation of image structures, particularly edges. The relationship between these two approaches offers a comprehensive framework for image restoration. This paper designs a wavelet frame-based image restoration model, focusing on enhancing edge preservation and regularity. More importantly, we establish a connection to the [Formula: see text] version of the Mumford–Shah model, showing that the wavelet model converges to this variational model. This connection is significant, as it combines the geometric explanation of edges in the Mumford–Shah model with the simplicity of wavelet-based implementation. The primary contribution of this paper lies in the asymptotic analysis and proof of convergence of the two-scale wavelet model to the [Formula: see text] Mumford–Shah model, providing both theoretical insights and practical wavelet models for image restoration with enhanced edge detection.

We study the approximation of functions which are invariant with respect to certain permutations of the input indices using flow maps of dynamical systems. Such invariant functions include the much studied translation-invariant ones involving image tasks, but also encompasses many permutation-invariant functions that find emerging applications in science and engineering. We prove sufficient conditions for universal approximation of these functions by a controlled dynamical system, which can be viewed as a general abstraction of deep residual networks with symmetry constraints. These results not only imply the universal approximation for a variety of commonly employed neural network architectures for symmetric function approximation, but also guide the design of architectures with approximation guarantees for applications involving new symmetry requirements.

Correcting the optical aberrations and the manufacturing deviations of cameras is a challenging task. Due to the limitation on volume and the demand for mass production, existing mobile terminals cannot rectify optical degradation. In this work, we systematically construct the perturbed lens system model to illustrate the relationship between the deviated system parameters and the spatial frequency response (SFR) measured from photographs. To further address this issue, an optimization framework is proposed based on this model to build proxy cameras from the machining samples' SFRs. Engaging with the proxy cameras, we synthetic data pairs, which encode the optical aberrations and the random manufacturing biases, for training the learning-based algorithms. In correcting aberration, although promising results have been shown recently with convolutional neural networks, they are hard to generalize to stochastic machining biases. Therefore, we propose a dilated Omni-dimensional dynamic convolution (DOConv) and implement it in post-processing to account for the manufacturing degradation. Extensive experiments which evaluate multiple samples of two representative devices demonstrate that the proposed optimization framework accurately constructs the proxy camera. And the dynamic processing model is well-adapted to manufacturing deviations of different cameras, realizing perfect computational photography. The evaluation shows that the proposed method bridges the gap between optical design, system machining, and post-processing pipeline, shedding light on the joint of image signal reception (lens and sensor) and image signal processing (ISP).

Distributed acoustic sensing (DAS) is a new downhole vertical seismic profile (VSP) acquisition technology, which has many advantages of low cost, sensitive signal capture capability and high spatial-temporal resolution. It can provide dense wavefield information for subsequent processing. Although DAS has obvious advantages over geophones, some weakness may limit its application. The main challenge is that DAS data are polluted by various types of noise, including optical abnormal noise, random background noise, fading noise, and so on. The noise brings great difficulties to the interpretation of seismic data. In order to suppress the noise and recover the buried weak effective signals, we design a new sparse parallel-subnet network (SPSNet) in this paper. It includes a parallel-subnet feature extraction module with sparse mechanism, simultaneously extracting global and local dual features. In this way, we can extract as much detailed information as possible from DAS seismic data. Then the following enhancement module fuses these features for signals complement. Another outstanding advantage is its high efficiency owing to the parallel structure. Compared with the complex networks with the same noise suppression effect, SPSNet has higher work efficiency. We generate a large number of geologic structure models with different parameters to optimize SPSNet. The denoising results show that the proposed method can effectively suppress a variety of noise in DAS seismic data. And the deep layer signals with weak energy are also well recovered.

Diverse application scenarios in 5G and beyond wireless communication systems have introduced various requirements in code lengths and rates of channel codes. For the decoding of polar codes, especially the belief-propagation (BP) decoding, flexible configuration of codeword length is still not involved in current decoders. In this paper, a semi-folded decoding structure is proposed which can be reconfigured to support multiple codeword lengths. Up to 16 codes can be decoded in parallel and the utilization of processing units is no less than 87.5% for various codeword lengths. The peak throughput of 19.29 Gbps can be achieved by the proposed decoder in SMIC 55 nm CMOS technology.

The detection of driving events could be useful for reducing accidents, fleet management and insurance premiums etc. Currently, top of the range vehicles and large fleets employ expensive driver monitoring systems. However, most drivers do not have access to such systems. The required monitoring platform would have to deliver the required performance while also being affordable and accessible. A candidate with considerable promise is the smartphone with sensors built-in that could be exploited for the detection of driving events. However, to date it has not been possible to achieve the required correct, missed and false detection rates in addition to the computational efficiency for real-time operations. This paper proposes a novel bagging tree and dynamic time warping (DTW) integrated algorithm for the detection of driving events employing acceleration and orientation data from a smartphone's low cost three-axis accelerometers and gyroscopes. The bagging tree-based machine learning algorithm provides the initial maneuver detection results, as well as the location of the event start and end points. Event detection is then achieved by calculating the similarity of the results predicted through the bagging tree algorithm with the corresponding templates extracted from the experience datasets, while also applying a number of constraints to verify the calculated results. Field test results show that the proposed integrated algorithm is superior to the state-of-the-art, achieving a high correct detection accuracy of 97.5%, a low missed detection of 2.5% and a false detection rate of 2.9%. The corresponding results for the best alternative candidate method are 90.2%, 9.8% and 11.7%. Furthermore, the improvement in computational efficiency offered by our proposed approach is three to more than ten times greater than that of the other state-of-the-art algorithms.

Summary In this paper, we consider the statistical analysis of parallel system with inverse Weibull distributed components. Due to cost and time constraints, the causes of system failures are masked and the type‐II censored observations might occur in the collected data. Under the symmetric and asymmetric loss functions, the expected Bayesian (E‐Bayesian) method and the hierarchical Bayesian method are proposed to estimate the parameters, as well as the reliability function. Numerical simulations using the Monte Carlo (MC) method are given to demonstrate the performances of the estimations under different masking levels and effective sample sizes. Finally, one data set is analyzed for illustrative purpose.

In this paper, we study scheduling problems that have batching considerations, and with sequence-dependent setup times for single machine and job shop flow planning. An important focus is the explicit treatment of setup times (costs), missed due dates (tardiness) and wastage as important cost components that impacts the direct and indirect cost. Indirect cost reflects the true cost of an urgent order that caused the machine setup to be re-arranged that can influence business decision on how to priced such order and how on a recurring basis might affect the bottom line profit and lost. [Allahverdi et al., 2006] found that the majority of the earlier papers assumed that the setup time (cost) is negligible or part of the job processing time (cost). This assumption simplifies the analysis however it adversely affects the solution quality. However, there is an increased interest in scheduling problems involving setup times as many recognised that there are tremendous savings when setup times/costs are explicitly incorporated in scheduling decision in various real world industrial environments.