Mixture-of-experts (MoE) architectures enable scalable Large Language Models (LLMs) with reduced computational overhead, yet their deployment on memory-constrained edge devices is hindered by substantial memory demands. Traditional expert-offloading techniques mitigate memory constraints but often significantly increase inference latency. We introduce MoE-APEX, an Adaptive Precision EXpert offloading system that optimizes MoE inference for edge architectures by dynamically managing expert precision. Our core innovation is to replace less critical cache-miss experts with low-precision variants, reducing loading latency while maintaining accuracy. MoE-APEX introduces three innovative techniques that map the natural hierarchy of MoE computation: (1) a token-level dynamic expert loading mechanism, (2) a layer-level adaptive expert prefetching technique, and (3) a sequence-level cost-aware expert caching policy. These innovations enable MoE-APEX to leverage the benefits of mixed-precision expert inference fully. Implemented atop Llama.cpp, MoE-APEX achieves decoding speedups ranging from 1.34x to 9.75x compared to state-of-the-art MoE offloading systems across diverse edge devices, offering a robust solution for efficient MoE deployment in resource-constrained environments.

Multi-modal neural networks have demonstrated exceptional performance by merging information across modalities, surpassing the state-of-the-art uni-modal DNNs. However, this accuracy improvement comes at the cost of increased computation, leading to higher inference latency. This defect significantly limits the practical value of multi-modal DNNs, especially for latency-aware applications. Therefore, we propose an adaptive and efficient multi-modal shortcut architecture called M2SN to reduce the execution latency with accuracy guarantees. It skips ineffective network layers to reduce computational costs as well as alleviate the overfitting problem adaptive to specific models and scenarios. The key contributions of M2SN are twofold: 1) We design and insert shortcuts into each uni-modal network to perform adaptive computing. 2) We design a navigator to dynamically choose the optimal shortcuts. Unlike previous approaches, M2SN features high generality as it does not rely on any prior knowledge. The experimental results show that M2SN can reduce 28.3% average latency while obtaining the same or higher accuracy compared with SOTA baselines.

Moving object detection is critical for automated video analysis in many vision-related tasks, such as surveillance tracking, video compression coding, etc. Robust Principal Component Analysis (RPCA), as one of the most popular moving object modelling methods, aims to separate the temporally-varying (i.e., moving) foreground objects from the static background in video, assuming the background frames to be low-rank while the foreground to be spatially sparse. Classic RPCA imposes sparsity of the foreground component using$\ell _1$-norm, and minimizes the modeling error via$\ell _2$-norm. We show that such assumptions can be too restrictive in practice, which limits the effectiveness of the classic RPCA, especially when processing videos with dynamic background, camera jitter, camouflaged moving object, etc. In this paper, we propose a novel RPCA-based model, called Hyper RPCA, to detect moving objects on the fly. Different from classic RPCA, the proposed Hyper RPCA jointly applies the maximum correntropy criterion (MCC) for the modeling error, and Laplacian scale mixture (LSM) model for foreground objects. Extensive experiments have been conducted, and the results demonstrate that the proposed Hyper RPCA has competitive performance for foreground detection to the state-of-the-art algorithms on several well-known benchmark datasets.

Considering the$ \alpha$-stable input signals and noises, several robust fractional-order adaptive filtering algorithms have been proposed in recent years. However, the mean-square performance analysis of the FoAF algorithm has not been well studied in the literature. Toward that end, this letter provide a theoretical mean-square performance analysis of the FoAF algorithm, involving transient and steady-state behavior. Furthermore, an improved fractional-order adaptive filtering algorithm is derived by utilizing the M-estimate technique. Simulation experiments verify the results of the theoretical analysis and demonstrate the advantages of the proposed algorithm in heavy-tailed non-Gaussian environments.

In this paper, the Fractional-Order Memristive Chaotic Circuit (FMCC) is proposed to be achieved by the fracmemristor, which is a portmanteau of “fractional-order” and “memristor”. Considering the unique fingerprints and nonlinearities of fracmemristors, it is natural to ponder a challenging theoretical problem to generalize the Integer-Order Memristive Chaotic Circuit (IMCC) to the FMCC. Motivated by this inspiration, the paper proposes an FMCC by replacing the diode in Chua’s chaotic circuit with a fracmemristor and a negative resistor in parallel. To simplify analysis, a new Cubic Nonlinear Voltage-Controlled Capacitive Ladder Scaling Fracmemristor (CVCLF) is proposed to implement the FMCC. New fingerprints are found in the CVCLF. Compared with the IMCC, dynamical behaviors of the FMCC are not only related to circuit parameters and initial conditions, but also related to the circuit stage and the operational order. The FMCC provides two extra degrees of freedom. Numerical simulations and hardware experiments demonstrate that the FMCC has multistability, transient chaos, state transition phenomena, etc. A significant advantage of the FMCC is that it possesses the fractional-order-sensitivity characteristic, which represents its dynamical behaviors change with the operational order. The proposed FMCC is the first application of fracmemristors in chaos.

Complex-valued neural network is a kind of learning model which can deal with problems in complex domain. Fully complex extreme learning machine (CELM) is a much faster training algorithm than the complex backpropagation (CBP) scheme. However, it is at the cost of using more hidden nodes to obtain the comparable performance. An upper-layer-solution-aware algorithm has been proposed for training single-hidden layer feedforward neural networks, which performs much better than its counterparts, pseudo-inverse learning (PIL)/extreme learning machine and gradient decent-based backpropagation neural networks. Consequently, there exist two challenges that need to be dealt with: 1) How to combine the advantages of CBP and CELM to develop a novel complex learning algorithm? and 2) What is the convergent behavior of the presented algorithm? In this article, an input weights dependent complex-valued (IWDCV) learning algorithm based on Wirtinger calculus has been proposed, which effectively solves the nonanalytic problem of the common activation functions during training neural networks. In addition, the monotonicity of the error function and the deterministic convergence of the proposed model have been strictly proved, which theoretically guarantee the efficiency and effectiveness of the given model, IWDCV. Finally, for real and complex-valued problems, a variety of simulations have been done to demonstrate the comparable performance of the proposed algorithm which support the theoretical observations as well.

We propose a novel circuit for the fractional-order memristive neural synaptic weighting (FMNSW). The introduced circuit is different from the majority of the previous integer-order approaches and offers important advantages. Since the concept of memristor has been generalized from the classic integer-order memristor to the fractional-order memristor (fracmemristor), a challenging theoretical problem would be whether the fracmemristor can be employed to implement the fractional-order memristive synapses or not. In this research, characteristics of the FMNSW, realized by a pulse-based fracmemristor bridge circuit, are investigated. First, the circuit configuration of the FMNSW is explained using a pulse-based fracmemristor bridge circuit. Second, the mathematical proof of the fractional-order learning capability of the FMNSW is analyzed. Finally, experimental work and analyses of the electrical characteristics of the FMNSW are presented. Strong ability of the FMNSW in explaining the cellular mechanisms that underlie learning and memory, which is superior to the traditional integer-order memristive neural synaptic weighting, is considered a major advantage for the proposed circuit.

Active noise control (ANC) is gaining attention for attenuating noise from a remote location. Considering the problem of nonlinear active noise control (NLANC) at a virtual location, a robust filtered-s subband adaptive filtering algorithm based on the q-gradient maximum correntropy criterion (RFsSAF-qMCC) is proposed in this paper. The proposed RFsSAF-qMCC algorithm develops the functional link artificial neural network (FLANN)-SAF structure as the controller, and embeds the MCC with the concept of q-gradient, thereby improving the convergence speed in the impulsive environment. To solve the trade-off between fast convergence and low noise residue caused by the fixed q-gradient, a variable q-gradient algorithm, termed as RFsSAF-vqMCC, is further developed. As an additional contribution, the convergence behavior of the proposed RFsSAF-qMCC and RFsSAF-vqMCC algorithms is analyzed. Simulation results corroborate the effectiveness of the proposed algorithms as compared to state-of-the-art algorithms.

Single-image super-resolution (SR) has been widely used in computer vision applications. The reconstruction-based SR methods are mainly based on certain prior terms to regularize the SR problem. However, it is very challenging to further improve the SR performance by the conventional design of explicit prior terms. Because of the powerful learning ability, deep convolutional neural networks (CNNs) have been widely used in single-image SR task. However, it is difficult to achieve further improvement by only designing the network architecture. In addition, most existing deep CNN-based SR methods learn a nonlinear mapping function to directly map low-resolution (LR) images to desirable high-resolution (HR) images, ignoring the observation models of input images. Inspired by the split Bregman iteration (SBI) algorithm, which is a powerful technique for solving the constrained optimization problems, the original SR problem is divided into two subproblems: 1) inversion subproblem and 2) denoising subproblem. Since the inversion subproblem can be regarded as an inversion step to reconstruct an intermediate HR image with sharper edges and finer structures, we propose to use deep CNN to capture low-level explicit image profile enhancement prior (PEP). Since the denoising subproblem aims to remove the noise in the intermediate image, we adopt a simple and effective denoising network to learn implicit image denoising statistics prior (DSP). Furthermore, the penalty parameter in SBI is adaptively tuned during the iterations for better performance. Finally, we also prove the convergence of our method. Thus, the deep CNNs are exploited to capture both implicit and explicit image statistics priors. Due to SBI, the SR observation model is also leveraged. Consequently, it bridges between two popular SR approaches: 1) learning-based method and 2) reconstruction-based method. Experimental results show that the proposed method achieves the state-of-the-art SR results.

-norm of weight matrix between the input and hidden layers. These penalty terms are nonsmooth at the origin, and hence, one simple but efficient smoothing technique is employed to overcome this issue. The monotonicity and convergence of the proposed algorithm are specified and proved under suitable assumptions. Then, extensive experiments are conducted on both artificial and real data sets. Empirical results explicitly demonstrate the ability of the proposed FS scheme and its effectiveness in controlling redundancy. The empirical simulations are observed to be consistent with the theoretical results.

The Internet of Vehicles (IoV) plays a central role in intelligent transportation systems. Components, such as motor and transmission in the vehicle may produce noise, which seriously affects comfort. Therefore, vehicle manufacturers attach great importance to active noise control (ANC) technology. However, such an ANC system may have some nonlinear distortions in practical, thereby the nonlinear ANC (NANC) system is warranted. Moreover, we consider using IoV for rational resource allocation and record historical data for fault diagnosis, early warning, etc. So far, no work on NANC in the IoV environment is reported. In this article, based on the Hermite polynomial, a class of functional link artificial neural network (FLANN) algorithms is developed for NANC. The first proposed algorithm, called filtered-h least mean ${\mathcal {L}}_{p}$ -norm (FhLMP), incorporates the ${\mathcal {L}}_{p}$ -norm to obtain reliable performance. To further enhance the performance, the recursive FhLMP (RFhLMP) and hyperbolic recursive FhLMP (HRFhLMP) algorithms are designed by formulating two recursive structures. The proposed RFhLMP algorithm takes the filter output as part of the input and is expanded by the Hermite FLANN. The HRFhLMP algorithm activates the output by a hyperbolic tangent function and then recursively returns the activated output to the filter input. Simulations verify the improvement of the proposed algorithms for the NANC system.

We introduce the fractional-order global optimal backpropagation machine, which is trained by an improved fractional-order steepest descent method (FSDM). This is a fractional-order backpropagation neural network (FBPNN), a state-of-the-art fractional-order branch of the family of backpropagation neural networks (BPNNs), different from the majority of the previous classic first-order BPNNs which are trained by the traditional first-order steepest descent method. The reverse incremental search of the proposed FBPNN is in the negative directions of the approximate fractional-order partial derivatives of the square error. First, the theoretical concept of an FBPNN trained by an improved FSDM is described mathematically. Then, the mathematical proof of fractional-order global optimal convergence, an assumption of the structure, and fractional-order multi-scale global optimization of the FBPNN are analyzed in detail. Finally, we perform three (types of) experiments to compare the performances of an FBPNN and a classic first-order BPNN, i.e., example function approximation, fractional-order multi-scale global optimization, and comparison of global search and error fitting abilities with real data. The higher optimal search ability of an FBPNN to determine the global optimal solution is the major advantage that makes the FBPNN superior to a classic first-order BPNN.

Given that the existing image denoising methods damage the texture details of an image, a new method based on fractional integration is proposed. First, the fractional-order integral formula is deduced by generalizing the Cauchy integral, and then the approximate value of the fractional-order integral operator is estimated by a numerical method. Finally, a fractional-order integral mask operator of any order is constructed in eight pixel directions of the image. Simulation results show that the proposed image denoising method can protect the edge texture information of the image while removing the noise. Moreover, this method can obtain higher image feature values and better image vision after denoising than the existing denoising methods, because a texture protection mechanism is adopted during the iterative processing.

This paper advocates a novel conceptual formulation of the fractional-order spatial steganography (FSS) and blind steganalysis for printed matter, which can be efficiently employed in the anti-counterfeiting for product external packing in Internet-of-Things (IoT). Traditional digital steganography is not printable. Within the limits of our knowledge, until now, there are not a well-established steganography and a corresponding steganalysis for printed matter in IoT, which should receive desired attention. Fractional calculus has potentially received prominence in applications in the domain of image processing mainly because of its strengths like long-term memory, nonlocality, and weak singularity. Therefore, in an attempt to overcome the aforementioned technical limitation of traditional digital steganography, this paper has studied here, as an interesting theoretical problem, would it be possible to apply the capability of preserving the edges and textural details of fractional calculus to the achievement of the steganography and steganalysis for printed matter in IoT. Motivated by this inspiration, in this work, this paper introduces a novel conceptual formulation of an FSS and a fractional-order blind steganalysis (FBS) for printed matter. At first, according to the opponent process theory of color vision, to better achieve the imperceptibility of the hidden secret information, this paper uses both the self-similar complex textures in a neighborhood and the opponent channel of blue versus yellow to implement FSS for printed matter. Second, without requiring a priori knowledge regarding the characteristics of the original carrier image, hidden secret image, and steganography, an FBS, a fractional-order multimodal function optimization algorithm, is proposed. Finally, the efficient capability of hiding secret information of FSS and that of detecting secret information of FBS are analyzed in detail experimentally, respectively. These two important advantages lead to the superiority of the proposed approach for defending against statistics attack, rotation and distortion attack, cropping attack, scaling attack, noise attack, and color copy attack. The main contribution of this paper is the first preliminary attempt of a feasible achievement of a spatial steganography and a blind steganalysis for printed matter.

It is widely acknowledged that single image super-resolution (SISR) methods play a critical role in recovering the missing high-frequencies in an input low-resolution image. As SISR is severely ill-conditioned, image priors are necessary to regularize the solution spaces and generate the corresponding high-resolution image. In this paper, we propose an effective SISR framework based on the enhanced non-local similarity modeling and learning-based multi-directional feature prediction (ENLTV-MDFP). Since both the modeled and learned priors are exploited, the proposed ENLTV-MDFP method benefits from the complementary properties of the reconstruction-based and learning-based SISR approaches. Specifically, for the non-local similarity-based modeled prior [enhanced non-local total variation, (ENLTV)], it is characterized via the decaying kernel and stable group similarity reliability schemes. For the learned prior [multi-directional feature prediction prior, (MDFP)], it is learned via the deep convolutional neural network. The modeled prior performs well in enhancing edges and suppressing visual artifacts, while the learned prior is effective in hallucinating details from external images. Combining these two complementary priors in the MAP framework, a combined SR cost function is proposed. Finally, the combined SR problem is solved via the split Bregman iteration algorithm. Based on the extensive experiments, the proposed ENLTV-MDFP method outperforms many state-of-the-art algorithms visually and quantitatively.

This letter presents a novel super-resolution (SR) method via nonlocal similarity modeling and deep convolutional neural network (CNN) gradient prior (GP). Specifically, on the one hand, the group similarity reliability (GSR) strategy is proposed for improving the adaptive high-dimensional nonlocal total variation (AHNLTV) model [statistical prior, GSR-based AHNLTV (GA)], which captures the structures of the underlying high-resolution (HR) image via the image itself. On the other hand, the GP is learned by using the deep CNN (learned prior), which predicts the gradients from external images. Finally, the GA-GP approach is proposed by incorporating the two complementary priors. The results show that GA-GP achieves better performance than other state-of-the-art SR methods.

This paper discusses a novel conceptual formulation of the fractional-order variational framework for retinex, which is a fractional-order partial differential equation (FPDE) formulation of retinex for the multi-scale nonlocal contrast enhancement with texture preserving. The well-known shortcomings of traditional integer-order computation-based contrast-enhancement algorithms, such as ringing artefacts and staircase effects, are still in great need of special research attention. Fractional calculus has potentially received prominence in applications in the domain of signal processing and image processing mainly because of its strengths like long-term memory, nonlocality, and weak singularity, and because of the ability of a fractional differential to enhance the complex textural details of an image in a nonlinear manner. Therefore, in an attempt to address the aforementioned problems associated with traditional integer-order computation-based contrast-enhancement algorithms, we have studied here, as an interesting theoretical problem, whether it will be possible to hybridize the capabilities of preserving the edges and the textural details of fractional calculus with texture image multi-scale nonlocal contrast enhancement. Motivated by this need, in this paper, we introduce a novel conceptual formulation of the fractional-order variational framework for retinex. First, we implement the FPDE by means of the fractional-order steepest descent method. Second, we discuss the implementation of the restrictive fractional-order optimization algorithm and the fractional-order Courant-Friedrichs-Lewy condition. Third, we perform experiments to analyze the capability of the FPDE to preserve edges and textural details, while enhancing the contrast. The capability of the FPDE to preserve edges and textural details is a fundamental important advantage, which makes our proposed algorithm superior to the traditional integer-order computation-based contrast enhancement algorithms, especially for images rich in textural details.

This paper presents a state-of-the-art application of fractional hopfield neural networks (FHNNs) to defend against chip cloning attacks, and provides insight into the reason that the proposed method is superior to physically unclonable functions (PUFs). In the past decade, PUFs have been evolving as one of the best types of hardware security. However, the development of the PUFs has been somewhat limited by its implementation cost, its temperature variation effect, its electromagnetic interference effect, the amount of entropy in it, etc. Therefore, it is imperative to discover, through promising mathematical methods and physical modules, some novel mechanisms to overcome the aforementioned weaknesses of the PUFs. Motivated by this need, in this paper, we propose applying the FHNNs to defend against chip cloning attacks. At first, we implement the arbitrary-order fractor of a FHNN. Secondly, we describe the implementation cost of the FHNNs. Thirdly, we propose the achievement of the constant-order performance of a FHNN when ambient temperature varies. Fourthly, we analyze the electrical performance stability of the FHNNs under electromagnetic disturbance conditions. Fifthly, we study the amount of entropy of the FHNNs. Lastly, we perform experiments to analyze the pass-band width of the fractor of an arbitrary-order FHNN and the defense against chip cloning attacks capability of the FHNNs. In particular, the capabilities of defense against chip cloning attacks, anti-electromagnetic interference, and anti-temperature variation of a FHNN are illustrated experimentally in detail. Some significant advantages of the FHNNs are that their implementation cost is considerably lower than that of the PUFs, their electrical performance is much more stable than that of the PUFs under different temperature conditions, their electrical performance stability of the FHNNs under electromagnetic disturbance conditions is much more robust than that of the PUFs, and their amount of entropy is significantly higher than that of the PUFs with the same rank circuit scale.

This paper mainly discusses a novel conceptual framework: fractional Hopfield neural networks (FHNN). As is commonly known, fractional calculus has been incorporated into artificial neural networks, mainly because of its long-term memory and nonlocality. Some researchers have made interesting attempts at fractional neural networks and gained competitive advantages over integer-order neural networks. Therefore, it is naturally makes one ponder how to generalize the first-order Hopfield neural networks to the fractional-order ones, and how to implement FHNN by means of fractional calculus. We propose to introduce a novel mathematical method: fractional calculus to implement FHNN. First, we implement fractor in the form of an analog circuit. Second, we implement FHNN by utilizing fractor and the fractional steepest descent approach, construct its Lyapunov function, and further analyze its attractors. Third, we perform experiments to analyze the stability and convergence of FHNN, and further discuss its applications to the defense against chip cloning attacks for anticounterfeiting. The main contribution of our work is to propose FHNN in the form of an analog circuit by utilizing a fractor and the fractional steepest descent approach, construct its Lyapunov function, prove its Lyapunov stability, analyze its attractors, and apply FHNN to the defense against chip cloning attacks for anticounterfeiting. A significant advantage of FHNN is that its attractors essentially relate to the neuron's fractional order. FHNN possesses the fractional-order-stability and fractional-order-sensitivity characteristics.

A random submatrix method (RSM) is proposed to calculate the low-rank decomposition U(m×r)V(n×r)(T) (r < m, n) of the matrix Y∈R(m×n) (assuming m > n generally) with known entry percentage 0 < ρ ≤ 1. RSM is very fast as only O(mr(2)ρ(r)) or O(n(3)ρ(3r)) floating-point operations (flops) are required, compared favorably with O(mnr+r(2)(m+n)) flops required by the state-of-the-art algorithms. Meanwhile, RSM has the advantage of a small memory requirement as only max(n(2),mr+nr) real values need to be saved. With the assumption that known entries are uniformly distributed in Y, submatrices formed by known entries are randomly selected from Y with statistical size k×nρ(k) or mρ(l)×l , where k or l takes r+1 usually. We propose and prove a theorem, under random noises the probability that the subspace associated with a smaller singular value will turn into the space associated to anyone of the r largest singular values is smaller. Based on the theorem, the nρ(k)-k null vectors or the l-r right singular vectors associated with the minor singular values are calculated for each submatrix. The vectors ought to be the null vectors of the submatrix formed by the chosen nρ(k) or l columns of the ground truth of V(T). If enough submatrices are randomly chosen, V and U can be estimated accordingly. The experimental results on random synthetic matrices with sizes such as 13 1072 ×10(24) and on real data sets such as dinosaur indicate that RSM is 4.30 ∼ 197.95 times faster than the state-of-the-art algorithms. It, meanwhile, has considerable high precision achieving or approximating to the best.

The application of fractional calculus to signal processing and adaptive learning is an emerging area of research. A novel fractional adaptive learning approach that utilizes fractional calculus is presented in this paper. In particular, a fractional steepest descent approach is proposed. A fractional quadratic energy norm is studied, and the stability and convergence of our proposed method are analyzed in detail. The fractional steepest descent approach is implemented numerically and its stability is analyzed experimentally.

In this paper, we intend to implement a class of fractional differential masks with high-precision. Thanks to two commonly used definitions of fractional differential for what are known as GrUmwald-Letnikov and Riemann-Liouville, we propose six fractional differential masks and present the structures and parameters of each mask respectively on the direction of negative x-coordinate, positive x-coordinate, negative y-coordinate, positive y-coordinate, left downward diagonal, left upward diagonal, right downward diagonal, and right upward diagonal. Moreover, by theoretical and experimental analyzing, we demonstrate the second is the best performance fractional differential mask of the proposed six ones. Finally, we discuss further the capability of multiscale fractional differential masks for texture enhancement. Experiments show that, for rich-grained digital image, the capability of nonlinearly enhancing complex texture details in smooth area by fractional differential-based approach appears obvious better than by traditional intergral-based algorithms.