Transformer models have been gaining substantial interest in the field of computer vision tasks nowadays. Although a vision transformer contains two important components which are self-attention module and feedforward network (FFN) module, the majority of research tends to concentrate on modifying the former while leaving the latter in its original form. In this paper, we focus on improving the FFN module within the vision transformer. Through theoretical analysis, we demonstrate that the effect of the FFN module primarily lies in providing non-linearity, whose degree corresponds to the hidden dimensions. Thus, the computational cost of the FFN module can be reduced by enhancing the degree of non-linearity in the nonlinear function. Leveraging this insight, we propose an improved FFN (IFFN) module for vision transformers which involves the usage of the arbitrary GeLU (AGeLU) function and integrating multiple instances of it to augment non-linearity so that the number of hidden dimensions can be effectively reduced. Besides, a spatial enhancement part is involved to further enrich the non-linearity in the proposed IFFN module. Experimental results show that we can apply our method to a wide range of state-of-the-art vision transformer models irrespective of how they modify their self-attention part and the overall architecture, and reduce FLOPs and parameters without compromising classification accuracy on the ImageNet dataset.

Vision transformer model (ViT) is widely used and performs well in vision tasks due to its ability to capture long-range dependencies. However, the time complexity and memory consumption increase quadratically with the number of input patches which limits the usage of ViT in real-world applications. Previous methods have employed linear attention to mitigate the complexity of the original self-attention mechanism at the expense of effectiveness. In this paper, we propose QT-ViT models that improve the previous linear self-attention using quadratic Taylor expansion. Specifically, we substitute the softmax-based attention with second-order Taylor expansion, and then accelerate the quadratic expansion by reducing the time complexity with a fast approximation algorithm. The proposed method capitalizes on the property of quadratic expansion to achieve superior performance while employing linear approximation for fast inference. Compared to previous studies of linear attention, our approach does not necessitate knowledge distillation or high-order attention residuals to facilitate the training process. Extensive experiments demonstrate the efficiency and effectiveness of the proposed QT-ViTs, showcasing the state-of-the-art results. Particularly, the proposed QT-ViTs consistently surpass the previous SOTA EfficientViTs under different model sizes, and achieve a new Pareto-front in terms of accuracy and speed.

Despite the fact that transformer-based models have yielded great success in computer vision tasks, they suffer from the challenge of high computational costs that limits their use on resource-constrained devices. One major reason is that vision transformers have redundant calculations since the self-attention operation generates patches with high similarity at a later stage in the network. Hierarchical architectures have been proposed for vision transformers to alleviate this challenge. However, by shrinking the spatial dimensions to half of the originals with downsampling layers, the challenge is actually overcompensated, as too much information is lost. In this paper, we propose FDViT to improve the hierarchical architecture of the vision transformer by using a flexible downsampling layer that is not limited to integer stride to smoothly reduce the sizes of the middle feature maps. Furthermore, a masked auto-encoder architecture is used to facilitate the training of the proposed flexible downsampling layer and produces informative outputs. Experimental results on benchmark datasets demonstrate that the proposed method can reduce computational costs while increasing classification performance and achieving state-of-the-art results. For example, the proposed FDViT-S model achieves a top-1 accuracy of 81.5%, which is 1.7 percent points higher than the ViT-S model and reduces 39% FLOPs.

In line with the rapid development of the Internet of Things (IoT), the maintenance of on-board batteries for a trillion sensor nodes has become prohibitive both in time and costs. Energy harvesting is a promising solution to this problem. However, conventional energy-harvesting systems with storage suffer from low efficiency because of conversion loss and storage leakage. Direct supply systems without energy buffer provide higher efficiency, but fail to satisfy quality of service (QoS) due to mismatches between input power and workloads. Recently, a novel dual-channel photovoltaic power system has paved the way to achieve both high energy efficiency and QoS guarantee. This article focuses on the design-time and run-time co-optimization of the dual-channel solar power system. At the design stage, we develop a task failure rate estimation framework to balance design costs and failure rate. At run-time, we propose a task failure rate aware QoS tuning algorithm to further enhance energy efficiency. Through the experiments on both a simulation platform and a prototype board, this study demonstrates a 27% task failure rate reduction compared with conventional architectures with identical design costs. And the proposed online QoS tuning algorithm brings up to 30% improvement in energy efficiency with nearly zero failure rate penalty.

Self-powered systems running on scavenged energy will be a key enabler for pervasive computing across the Internet of Things. The variability of input power in energy-harvesting systems limits the effectiveness of static optimizations aimed at maximizing the input-energy-to-computation ratio. We show that the resultant gap between available and exploitable energy is significant, and that energy storage optimizations alone do not significantly close the gap. We characterize these effects on a real, fabricated energy-harvesting system based on a nonvolatile processor. We introduce a unified energy-oriented approach to first optimize the number of backups, by more aggressively using the stored energy available when power failure occurs, and then optimize forward progress via improving the rate of input energy to computation via dynamic voltage and frequency scaling and self-learning techniques. We evaluate combining these schemes and show capture of up to 75.5% of all input energy toward processor computation, an average of 1.54 × increase over the best static “Forward Progress” baseline system. Notably, our energy-optimizing policy combinations simultaneously improve both the rate of forward progress and the rate of backup events (by up to 60.7% and 79.2% for RF power, respectively, and up to 231.2% and reduced to zero, respectively, for solar power). This contrasts with static frequency optimization approaches in which these two metrics are antagonistic.

Energy harvesting from natural environment gives range of benefits for the Internet of things. Scavenging energy from photovoltaic (PV) cells is one of the most practical solutions in terms of power density among existing energy harvesting sources. PV power systems mandate the maximum power point tracking (MPPT) to scavenge the maximum possible solar energy. In general, a switching-mode power converter, an MPPT charger, controls the charging current to the energy storage element (a battery or equivalent), and the energy storage element provides power to the load device. The mismatch between the maximum power point (MPP) current and the load current is managed by the energy storage element. However, such architecture causes significant energy loss (typically over 20%) and a significant weight/volume and a high cost due to the cascaded power converters and the energy storage element. This paper pioneers a converter-less PV power system with the MPPT that directly supplies power to the load without the power converters or the energy storage element. The proposed system uses a nonvolatile microprocessor to enable an extremely fine-grain dynamic power management in a few hundred microseconds. This makes it possible to match the load current with the MPP current. We present detailed modeling, simulation, and optimization of the proposed energy harvesting system including the radio frequency transceiver. Experiments show that the proposed setup achieves an 87.1% of overall system efficiency during a day, 30.6% higher than the conventional MPPT methods in actual measurements, and thus a significantly higher duty cycle under a weak solar irradiance.

Solar-powered sensor nodes with energy storages are widely used today and promising in the coming trillion sensor era, as they do not require manual battery charging or replacement. The changeable and limited solar power supply seriously affects the deadline miss rates (DMRs) of tasks on these nodes and therefore energy-driven task scheduling is necessary. However, current algorithms focus on the single period (or the current task queue) for high energy utilization and suffer from bad long term DMR. To get better long term DMR, we propose a long term deadline-aware scheduling algorithm with energy migration strategies for distributed super capacitors. Experimental results show that the proposed algorithm reduces the DMR by 27.8% and brings less than 3% of the total energy consumption.

Nonvolatile (NV) processors have attracted much attention in recent years due to their zero standby power, resilience to power failures, and instant-on feature. One design challenge of NV processors is the excess area needed by NV registers. This paper introduces a parallel compare and compress (PaCC) architecture to reduce such excess area. A key component of the PaCC architecture is a new codec which effectively balances area and performance. In addition, the PaCC architecture includes a configurable state table to support reference vector selection for different applications. With the proposed vector selection algorithm, the PaCC architecture can outperform other vector selection approaches by over 59% in terms of reduction in the number of NV registers. The proposed architecture has been fully realized at the circuit level and synthesized for the Rohm's 0.13-μm ferroelectric-CMOS hybrid process. Results demonstrate that the design can reduce the number of NV registers by 70%-80% with less than 1% overflow possibility, which leads to up to 30% processor area saving. The overall approach is applicable to any NV processor design regardless of the NV material used.

Nonvolatile processor (NVP) has become an emerging topic in recent years. The conventional NV processor equips each flip-flop with a nonvolatile storage for data backup, which results in much faster backup speed with significant area overheads. A compression based architecture (PRLC) solved the area problem but with a nontrivial increasing on backup time. This paper provides a segment-based parallel compression (SPaC) architecture to achieve tradeoffs between area and backup speed. Furthermore, we use an off-line and online hybrid method to balance the workloads of different compression modules in SPaC. Experimental results show that SPaC can achieve 76% speed up against PRLC and meanwhile reduces the area by 16% against conventional NV processors.