In the natural world, energy and information are deeply entwined, mutually constraining and complementing each other. To exploit this natural merit, this paper proposes a FEI strategy: Fusion processing of sensing Energy and Information for infrared smart vision system. The proposed Information-Power-Coupler (IPCp) takes the ability of simultaneous energy harvesting and low power inpixel computing, which utilizes in-situ coupled energy to process the containing information on the same focal plane. Furthermore, a self-adaptive Intelligent-Power- Controller (IPCtrl) capable of scheduling the harvested energy to complete low power neural network inference is introduced. The implementation of IPC2 system utilizes a software-hardware co-design strategy to exploit the layer-wise characteristic of the computation process and circuit topology, achieving energy-efficient self-sustainable fusion processing of sensing energy and information. Simulation results show that the IPCtrl could supply 594.68nW with the power conversion efficiency of 93.38%, when the harvested energy from the IPCp is 636.84nW. The performance validates the self-sustainability of the system with the self-powered image recognition of a complete network running at 4fps with an accuracy of 99.4%.

This paper proposes an in-situ self-powered BNN-based intelligent visual perception system that harvests light energy utilizing the indispensable image sensor itself. The harvested energy is allocated to the low-power BNN computation modules layer by layer, adopting a light-weighted duty-cycling-based energy scheduler. A software-hardware co-design method, which exploits the layer-wise error tolerance of BNN as well as the computing-error and energy consumption characteristics of the computation circuit, is proposed to determine the parameters of the energy scheduler, achieving high energy efficiency for self-powered BNN inference. Simulation results show that with the proposed inference-adaptive energy scheduling method, self-powered MNIST classification task can be performed at a frame rate of 4 fps if the harvesting power is 1μW, while guaranteeing at least 90% inference accuracy using binary LeNet-5 network.

Deploying intelligent visual algorithms in terminal devices for always-on sensing is an attractive trend in the IoT era. In-sensor-processing architecture is proposed to reduce power consumption on A/D conversion and data transmission, which performs pre-processing and only converting low-throughput features. However, current designs still require high energy consumption on photoelectric conversion and analog data movement. In this paper, two methods are proposed to improve the energy efficiency of in-sensor-processing architecture, including direct photocurrent computation and 2D kernel scheduling. Photocurrents are directly involved in computation to avoid data conversion; thus the indispensable imaging power is also utilized for computing. Since the location of the pixel data is fixed, data scheduling is conducted on digital weights to eliminate analog data storage and movement. We implement a prototype chip with an array of 32 × 32 units to calculate the first layer of binarized LeNet-5. The post-simulation shows that the proposed architecture reaches the energy efficiency of 11.49TOPs/W, about 14.8x higher than previous works.