We model the evolution of tokens within a deep stack of Transformer layers as a continuous-time flow on the unit sphere, governed by a mean-field interacting particle system, building on the framework introduced in Geshkovski et al. (2023). Studying the corresponding mean-field Partial Differential Equation (PDE), which can be interpreted as a Wasserstein gradient flow, in this paper we provide a mathematical investigation of the long-term behavior of this system, with a particular focus on the emergence and persistence of meta-stable phases and clustering phenomena, key elements in applications like next-token prediction. More specifically, we perform a perturbative analysis of the mean-field PDE around the iid uniform initialization and prove that, in the limit of large number of tokens, the model remains close to a meta-stable manifold of solutions with a given structure (e.g., periodicity). Further, the structure characterizing the meta-stable manifold is explicitly identified, as a function of the inverse temperature parameter of the model, by the index maximizing a certain rescaling of Gegenbauer polynomials.

This paper introduces a dynamic current comparator employing a parasitic charge recycling technique to achieve an ultra-low energy per comparison of 175 fJ. Furthermore, incorporating a switched capacitor supply in the static portion of the circuit, the comparison energy is further reduced to 55 fJ, more than three times lower. The proposed circuit, designed and extensively simulated in a 130-nm CMOS process with thick gate MOS transistors, demonstrates a comparison time of 3.8 ns, with a 100-nA input difference and a 32-MHz clock, while offering an input-referred noise of 2.9 nArms. These features make the proposed comparator highly suitable for medium-to high-resolution data conversion systems, particularly in biomedical and sensor-readout applications. Additionally, it is well-suited for high-speed and cryogenic CMOS systems, where current-mode architectures are becoming increasingly relevant.

In this paper, we study the evolution of tokens through the depth of encoder-only transformer models at inference time by modeling them as a system of particles interacting in a mean-field way and studying the corresponding dynamics. More specifically, we consider this problem in the moderate interaction regime, where the number $N$ of tokens is large and the inverse temperature parameter $\beta$ of the model scales together with $N$. In this regime, the dynamics of the system displays a multiscale behavior: a fast phase, where the token empirical measure collapses on a low-dimensional space, an intermediate phase, where the measure further collapses into clusters, and a slow one, where such clusters sequentially merge into a single one. We provide a rigorous characterization of the limiting dynamics in each of these phases and prove convergence in the above mentioned limit, exemplifying our results with some simulations.

The Neural Tangent Kernel (NTK) viewpoint is widely employed to analyze the training dynamics of overparameterized Physics-Informed Neural Networks (PINNs). However, unlike the case of linear Partial Differential Equations (PDEs), we show how the NTK perspective falls short in the nonlinear scenario. Specifically, we establish that the NTK yields a random matrix at initialization that is not constant during training, contrary to conventional belief. Another significant difference from the linear regime is that, even in the idealistic infinite-width limit, the Hessian does not vanish and hence it cannot be disregarded during training. This motivates the adoption of second-order optimization methods. We explore the convergence guarantees of such methods in both linear and nonlinear cases, addressing challenges such as spectral bias and slow convergence. Every theoretical result is supported by numerical examples with both linear and nonlinear PDEs, and we highlight the benefits of second-order methods in benchmark test cases.

European banking groups have been addressing the problem of restructuring their branch networks. Indeed, in the light of the recent digital transition of services, they aim to leverage digital channels to deliver basic services and reduce the number of physical facilities to dedicate to more complex and added-value operations. In order to implement the new business model, banking groups are shrinking their internal branch networks and outsourcing basic services to maintain physical proximity to customers unwilling to adopt digital channels. This work aims to formulate a mathematical programming model to support the decision-making process concerning the branch network restructuring. The model is formulated as a hierarchical covering location problem. Three types of branches providing different categories of services are considered. They are organised according to a three-level hierarchical structure, and each level is associated with a covering radius, representing the related accessibility condition to be guaranteed to users. A further category of facilities is considered, namely external facilities, that may support internal ones in providing basic banking services. The objective is to identify the network structure able to serve all the demand for banking services and minimize the total costs. A specific parameter is introduced to regulate the maximum outsourcing level that the bank is willing to achieve. The model is tested by considering a real case study concerning one of the main banking groups in Italy. The obtained results show the capability of the model to provide interesting scenarios and fruitful managerial implications.

This paper presents a non-imaging differential digital passive Infra-Red (PIR) remote sensing system using CMOS-SOI-MEMS transistors as the thermal sensor. A large $660\mu {\mathrm m}\, {\mathrm x}\, 660\mu{\mathrm m}$ pixel area is developed by the 8x8 mosaic matrix of $60\mu {\mathrm m} \,{\mathrm x}\, 60\mu{\mathrm m}$ sub-pixels connected in-parallel. The mosaic sensors, which are manufactured by nano-fabrication methods in CMOS FABs, exhibit enhanced performance and robust manufacturing in wafer level processing and vacuum packaging. Since the sub-pixels are thermally isolated, the thermal time constant of the large pixel is determined by that of the sub-pixel, which is $\sim {\mathrm 80}$ msec when packaged in vacuum of $\sim 1{\mathrm Pa}$. For outdoor operation, two identical large pixels are differentially measured. The pixels view the detected scene with an optics that may be based either on mirror optics or Fresnel plastic lenses. The optics defines a narrow field of view (± 3 degrees) as required for curtain sensors. Furthermore, the optics forms “cockeyed” vision which enables differential measurement that cancels the environmental” noise” and allows outdoor operation. The overall measured performance for detecting human targets at extended ranges and hot spots detection are reported. This sensor outperforms thermopiles and pyroelectric sensors at outdoor operation.

This paper presents a novel micromachined high-responsivity thermopile sensor, fully compatible with standard CMOS-SOI (Silicon-On-Insulator) processes. The proposed thermopile features a versatile mosaic structure, based on 128 60 μ m × 60 μ m pixels connected in series and/or in parallel. Two versions of the proposed thermopile sensor, featuring a different number of equivalent pixels, are presented and fully characterized. The most performing of the two features $2.9\cdot 10^{4}\frac{V}{W}$-responsivity, outperforming other state-of-the-art thermopile sensors. The application of the most performing thermopile as proximity and motion detector was also verified through measurements.

This paper presents a fully integrated CMOS contactless temperature sensor. The sensing element is a CMOS compatible thermopyle sensor, which consists of polysilicon resistors with different doping in a suspended membrane to increase their thermal resistance and sensitivity. The output signal of the thermopyle is fed into a high impedance programmable gain amplifier (PGA) and digitized by an energy-efficient voltage-to-digital converter based on a second-order delta-sigma modulator ($\Delta\Sigma$). This $\Delta\Sigma$ employs a first stage low noise integrator, a feedforward to reduce input swing on the second stage integrator, and scrambling of the input capacitors to reduce mismatch errors. The $\Delta\Sigma$ also converts the die temperature sensor used to read the absolute temperature of the radiating object. The proposed contactless temperature sensor has been realized with 130-nm CMOS process with nominal supply voltage at 1. 8V. A power consumption of approximately 3.6 $\mu$W has been recorded from measurements, while an equivalent Object temperature rms noise of 30 mdegC can be achieved, rendering the circuit suitable for portable and wearable applications.

This paper presents a microsystem suitable for contact-less human body temperature measurements, as well as for presence, motion and proximity detection. It consists of a 130-nm CMOS-SOI MEMS (Micro-Electro Mechanical System) thermal sensor, referred to as “TMOS”, and its 130-nm CMOS interface circuit. The TMOS, based on a micromachined transistor, being an active device, features advantages in terms of internal gain: with optimal biasing, indeed, the TMOS achieves 274-$\mu \text{V}/^\circ \text{C}$input-referred sensitivity at 3-cm distance and 50.33° field-of-view (FOV), outperforming thermopile detectors. The sensor and the interface circuit, featuring a chopper-stabilized-based analog readout with a 12-bit SAR ADC (Successive Approximation Register Analog-to-Digital Converter), were mounted in the same package and extensively measured: the microsystem achieves repeatability and ±0.17°C precision, thus satisfying the requirements for contact-less human body temperature measurements; furthermore, its performance as presence, motion and proximity detector was also verified.

This paper presents a readout circuit for thermopile-based thermal sensors, suitable for contactless temperature measurements. The circuit, designed and extensively simulated in a standard 130-nm CMOS process, employs chopper stabilization in order to minimize offset and noise contributions at low frequency, while providing proper amplification to the input signal, which behaves substantially as a DC. The circuit nominal supply voltage is 1.2 V and its power consumption is approximately 260 μW. The proposed single-ended architecture solves the drawbacks of the most straightforward fully differential approach, while achieving a simulated input-referred residual offset mean value equal to 84.4 nV, with 487 nV of standard deviation. An accuracy of ±0.3°C is provided at room temperature, making the circuit suitable for medical devices, such as contactless fever thermometers, and security systems, such as motion and human presence detection sensors.

The emerging wave of Big Data applications is flooding all branches of scientific knowledge. Economic and statistical applied research carried out in central banks and policy advising institutions is no exception. In this paper we present one of the most promising platform providing a unifying framework for different researchers willing to harness their knowledge of popular and simple computing environment such as R and Python. Along with their Integrated Development Environment (IDE), these are two of the most used numerical computing framework which are open source, provide built-in capabilities for statistical analysis and include a wide array of user contributed packages for an ample set of analytical tools suitable for different scientific applications. In the Big Data framework, we show how to provide researchers with a suitable programming environment allowing them to tame the intrinsic complexity of a High Performance Computing Cluster. Here we provide few empirical applications based on classical econometric and machine learning modeling.

The emerging wave of Big Data applications is flooding all branches of scientific knowledge. Economic and statistical applied research carried out in central banks and policy advising institutions is no exception. In this paper we present one of the most promising platform providing a unifying framework for different researchers willing to harness their knowledge of popular and simple computing environment such as R and Python. Along with their Integrated Development Environment (IDE), these are two of the most used numerical computing framework which are open source, provide built-in capabilities for statistical analysis and include a wide array of user contributed packages for an ample set of analytical tools suitable for different scientific applications. In the Big Data framework, we show how to provide researchers with a suitable programming environment allowing them to tame the intrinsic complexity of a High Performance Computing Cluster. Here we provide few empirical applications based on classical econometric and machine learning modeling.

The deep transformation induced by the World Wide Web (WWW) revolution has thoroughly impacted a relevant part of the social interactions in our present global society. The huge amount of unstructured information available on blogs, forum and public institution web sites puts forward different challenges and opportunities. Starting from these considerations, in this paper we pursue a two-fold goal. Firstly we review some of the main methodologies employed in text mining and for the extraction of sentiment and emotions from textual sources. Secondly we provide an empirical application by considering the latest 20 issues of the Bank of Italy Governor's concluding remarks from 1996 to 2015. By taking advantage of the open source software package R, we show the following: 1) checking the word frequency distribution features of the documents; 2) extracting the evolution of the sentiment and the polarity orientation in the texts; 3) evaluating the evolution of an index for the readability and the formality level of the texts; 4) attempting to measure the popularity gained from the documents in the web. The results of the empirical analysis show the feasibility in extracting the main topics from the considered corpus. Moreover it is shown how to check for positive and negative terms in order to gauge the polarity of statements and whole documents. The evaluation of these synthetic indexes is quite relevant for increasing the transparency of the central banks' communications.