Event-based Vision Sensors (EVS) gain popularity in en-hancing CMOS Image Sensor (CIS) video capture. Nonide-alities of EVS such as pixel or readout latency can significantly influence the quality of the enhanced images and warrant dedicated consideration in the design of fusion algorithms. A novel approach for jointly computing de-blurred, rolling-shutter artifact corrected high-speed videos with frame rates up to 10000 FPS using inherently blurry rolling shutter CIS frames of 120 FPS to 150 FPS in conjunction with EVS data from a hybrid CIS-EVS sensor is presented. EVS pixel latency, readout latency and the sensor's refractory period are explicitly incorporated into the measurement model. This inverse function problem is solved on a per-pixel manner using an optimization-based framework. The interpolated images are subsequently processed by a novel refinement network. The proposed method is evaluated using simulated and measured datasets, under natural and controlled environments. Extensive experiments show reduced shadowing effect, a 4 dB increment in PSNR, and a 12 % improvement in LPIPS score compared to state-of-the-art methods.

Unlike conventional ICs, approximate computing chips are less sensitive to hardware errors. This fascinating feature can be utilized to improve the performance of chip design and even change the timing closure procedure of digital circuit design flow. In this paper, we study the potential of resilience-aware circuit clocking scheme, and demonstrate the methodology with advanced neural network (NN)-based accelerator. We propose a novel timing analysis and frequency setting method for NN-based approximate computing circuits based on in-field NN retraining. With the proposed iterative retiming-and-retraining framework, NN-based accelerator can be retrained to operate safely at aggressive operating frequencies compared with the frequency decided purely by statistical timing analysis or Monto Carlo analysis. For nanometer process technology with increasing threats of timing errors induced by process variation, noises, and so on, our retiming-and-retraining method enables higher circuit operating frequency and enables dynamic precision/frequency adjustment for approximate computing circuits. We evaluate the methodology with both the neural and deep learning accelerators in experiments. The experimental results show that timing errors in neural circuits can be effectively tamed for different applications, so that the circuits can operate at higher clocking rates under the specified quality constraint or be dynamically scaled to work at a wide range of frequency states with only minor accuracy losses.

Because of tight power and energy constraints, industry is progressively shifting toward heterogeneous system-on-chip (SoC) architectures composed of a mix of general-purpose cores along with a number of accelerators. However, such SoC architectures can be very challenging to efficiently program for the vast majority of programmers, due to numerous programming approaches and languages. Libraries, on the other hand, provide a simple way to let programmers take advantage of complex architectures, which does not require programmers to acquire new accelerator-specific or domain-specific languages. Increasingly, library-based, also called algorithm-centric, programming approaches propose to generalize the usage of libraries and to compose programs around these libraries, instead of using libraries as mere complements. In this article, we present a software framework for achieving performance portability by leveraging a generalized library-based approach. Inspired by the notion of a component, as employed in software engineering and HW/SW codesign, we advocate nonexpert programmers to write simple wrapper code around existing libraries to provide simple but necessary semantic information to the runtime. To achieve performance portability, the runtime employs machine learning (simulated annealing) to select the most appropriate accelerator and its parameters for a given algorithm. This selection factors in the possibly complex composition of algorithms used in the application, the communication among the various accelerators, and the tradeoff between different objectives (i.e., accuracy, performance, and energy). Using a set of benchmarks run on a real heterogeneous SoC composed of a multicore processor and a GPU, we show that the runtime overhead is fairly small at 5.1% for the GPU and 6.4% for the multi-core. We then apply our accelerator selection approach to a simulated SoC platform containing multiple inexact accelerators. We show that accelerator selection together with hardware parameter tuning achieves an average 46.2% energy reduction and a speedup of 2.1× while meeting the desired application error target.

Emerging nonvolatile memories such as phase change memory (PCM) have the potential to replace internal memories in embedded devices. In this brief, we propose to use PCM as image buffer in application-specific multimedia systems. To improve the lifetime of PCM-based image buffer, we first eliminate redundant writes using data comparison. After redundant write elimination, PCM cells with respect to lower order bits of pixels are written more frequently than those corresponding to higher order. Based on this observation, we show that the lifetime can be further improved either by wear leveling using periodical data reversion to make write traffic even across PCM cells or by application-level error tolerance evaluation without leveling. Experimental results demonstrate that with the proposed techniques, the lifetime of PCM-based image buffer can be improved significantly.

As an emerging non-volatile memory technology, phase change memory (PCM) is promising as an alternative for traditional memories such as DRAM. In spite of its non-volatility, high density, low standby power, and resilience to soft errors, PCM has a limited write endurance or lifetime, which means that each PCM cell can only be overwritten finite times. More importantly, limited lifetime potentially provides malicious attackers an opportunity to intentionally aggravate write traffic into PCM. In this paper, from the standpoint of attackers, we propose random stream attacks (RSAK) methods for phase change memory used in video applications. Experimental results show that compared to natural video sequences, RSAK incurs higher total write traffic or worsened lifetime. RSAK also gives hints on how to build a more secure PCM in video applications to counter malicious write streams.

Modern video applications such as video codecs are memory-intensive. As an emerging non-volatile memory technology, phase change memory (PCM) will benefit video applications due to its high density, low leakage power and superior scalability. However, PCM consumes high write energy and can only sustain a limited write number. Hence it is necessary to reduce the write number of PCM for video applications. In this paper, we propose SoftPCM to enhance both energy efficiency and lifetime of PCM. SoftPCM utilizes the error tolerance characteristic of video applications to relax the accuracy of write operations. Experimental results show that SoftPCM can reduce 22% writes and thus improve energy efficiency and lifetime of PCM with slight video quality degradation.

Error tolerance is evolving into a new computing paradigm with further technology scaling, cost constraint, system scalability and emerging applications. Distinguished from defect tolerance and fault tolerance, error tolerance is based on application characteristics and relaxes the constraint of 100 percent functional correctness. From the viewpoint of error tolerance, this paper proposes a framework across multiple layers for fault criticality evaluation. Furthermore, taking an H.264/AVC decoder as an example, fault injection experiments demonstrate that for different functional modules, the faults in them bear different fault criticalities because of their unbalanced effects on applications, the faults in the same module also have diverse fault criticalities. The information that which faults are most critical can aid in test for yield and design for cost-effective fault tolerance. Error control techniques can be used to suppress error propagation and make more faults acceptable.