From the perspective of real-time processing, such as autonomous driving, sudden failures could potentially lead to severe and even life-threatening accidents. In particular, hardware faults in AI models used for real-time decision-making can compromise safety. To prevent these accidents before they happen, it is crucial to detect failures promptly. Moreover, for small and power-constrained devices such as drones, it is essential to develop a fault-tolerant AI that is computationally efficient and minimizes memory usage and power consumption. Approximate Triple Modular Redundancy (TMR) with quantization has been proposed for convolutional layers, which enables fault-tolerant inference with reduced computational cost. However, sufficiently efficient and reliable fault-tolerant methods for quantized Vision Transformers based on Transformer blocks have not yet been developed. In this paper, we propose Approximate Dual Modular Redundancy (DMR) for fault detection and Approximate TMR for fault recovery, specifically designed for Vision Transformers. In our evaluation, the proposed Approximate TMR was applied to an 8-bit quantized Swin Transformer. The results show that it reduces the computational cost significantly compared to conventional TMR. Furthermore, it successfully detects faults when single-bit flips occur with a probability exceeding 1%. We also demonstrate that the proposed Approximate TMR maintains higher accuracy than existing methods, such as Ranger and Clipper, even under single-bit flip faults.

In this paper, we propose a battery-powered wild animal tracking device using a Pan-Tilt-Zoom (PTZ) camera and deep learning. The proposed tracking device detects wild animals using YOLOv5 and tracks the detected wild animals using DeepSort. In addition, the proposed tracking device realizes tracking for a wide range by controlling a PTZ camera according to the movement direction of the detected wild animals. In the experiment, we developed a prototype for the proposed tracking device and conducted the field test of the developed prototype. We confirmed and discussed cases where the developed prototype could and could not track wild animals. The energy consumption of the developed prototype during tracking was 526.03J at the daytime and 730.99J at the nighttime.

In mission-critical systems such as self-driving, medical, and infrastructure systems, hardware faults can lead to serious accidents. Therefore, we need a method to detect hardware faults of artificial intelligence (AI) with high accuracy. A low-cost fault detection method with less computation is required to reduce AI’s chip area and/or energy consumption. In this paper, we propose an approximate Dual Modular Redundancy (DMR) unit using a Random Forest approximation method, which can significantly reduce the computation for inference in the convolutional neural networks (CNNs). We assume various scenarios of faults and evaluate the fault effects. In our experiments, we demonstrate that the proposed approximate DMR unit achieves high fault detection for three types of fault models. In addition, we report a 42.8% to 48.3% reduction in the computation for inference compared to the conventional method.

The bear attack to human beings is one of the fatal accidents, and it is becoming more critical to avoid such accidents because human 's encountering a bear happens every year, even in a city area. It is required to discover bears quickly and warn people to avoid bear accidents. To realize sensor nodes that detect bears automatically using image recognition technology, we aim to realize an accurate and computationally-efficient bear-presence detection. In this paper, we propose a bear-presence detection method combining region proposal of a low-resolution and image classification. In the experiments, we show that the proposed method achieves 4.9% higher recall and 2.3% higher F-score than image classification with-out region-proposal. Moreover, the proposed method achieved 0.6% higher recall and 18.5% higher F-score than YOLOv3, which is one of state-of-the-art object detection methods while the execution time was reduced to 72.4% for bear images and 55.5% for non-bear images.

Camera model identification has been studied extensively within digital image forensics as a deterrence to secret photography and image forgery. The feasibility of convolutional neural networks (CNNs) has been proven for an image classification algorithm. CNN-based algorithms have been proposed for camera model classification, and they focus on training with image/noise in a spatial domain. However, because the periodic characteristics of image noise are one of the essential types of information for model classification, it is more efficient to train CNN models with image noise in a frequency domain. In this paper, we propose a CNN-based approach for camera model classification that trains a CNN model with high-frequency components of images in the frequency domain. In the experiments, we evaluated the accuracy of camera model/brand classification using a Dresden image dataset. We achieved 97.35% and 99.32% accuracy, respectively, for 14-model classification of 256×256 image patches and full images. Using this approach, our results indicated a 1.84% and 1.35% improvement, respectively, compared with a state-of-the-art method. We also achieved 100% accuracy for 10-brand classification.

In advanced semiconductor-process technology, the ability to detect and repair lithography hotspots, which can affect printability, is essential. In this paper, we propose a two-stage cascade classifier for accurate hotspot detection. Our classifier uses a novel layout feature based on the propagation of light passing through a photomask. We performed experiments to evaluate our cascade classifier by applying it to the ICCAD-2012 CAD contest problem. The hotspot detection performance was evaluated according to two indices: (I1) the number of correctly detected hotspots over the number of actual hotspots and (I2) the number of correctly detected hotspots over the number of false hotspots. The results showed that the proposed method gained a 1.15% improvement in I1 and 24.4 times improvement in I2 on average compared to existing state-of-the-art methods, even the one with the best I1.

In recent years, camera identification methods have attracted attention in the field of digital forensics. The existing camera identification methods use features, such as the Exif header data and image noise, that indicate the characteristics of the camera. Of them, photo-response non-uniformity (PRNU) noise contains the unique features of an image sensor and is different for each individual camera. A camera identification method using the PRNU noise should have high identification ability, and a camera identification method using the pairwise magnitude relations of the clustered PRNU noise was previously proposed. In general, identification accuracy is estimated from test data sets, such as the Dresden image database. However, identification accuracy can be evaluated only with respect to the range of images within a database in the conventional evaluation method. A more detailed accuracy evaluation method is required for practical use. Furthermore, studies have not yet reported a false acceptance rate (FAR) evaluation method for the clustered PRNU pair-based camera identification capable of guaranteeing a low FAR (e.g., FAR = 10-9). In this paper, we proposed a new pixel clustering method that guarantees An FAR for camera identification using pairs of clustered PRNU noise, and evaluate its FAR based on a probability calculation of a mathematical model. In addition, we investigate the appropriate cluster size by using the Shapiro-Wilk test for an FAR evaluation. We show that it is possible to reliably calculate the FAR of a clustered PRNU noise pair-based camera identification method by using the proposed evaluation method. To demonstrate the validity of our calculations, we compare the actual identification result with the result of the proposed calculation. In this case, we used 16958 query images from the Dresden image database, which is a benchmark data set. The results of our evaluation indicate that this identification method maintains a false rejection rate of less than 5% (10%) for 5 (8) of the 10 tested cameras even for FAR = 10-9.

Field-programmable gate arrays (FPGAs) are extremely advanced with regard to high performance; they are becoming one of the primary device choices to realize high-performance computing (HPC). In this work, we propose an FPGA-based accelerator for the two-dimensional (2D) finite difference method (FDM) with the implicit scheme and implement a 2D unsteady-state heat conduction simulation using red/black successive over-relaxation (SOR). The accelerator consists of a 2D single-instruction multiple-data (SIMD) array processor, which has pipelined processing elements (PEs) including 32-bit floating point calculation units. This processor can avoid the memory-access bottleneck and perform with high operating efficiency and low waiting time for data transfer by applying the proposed control method with synchronous shift data transfer. We demonstrate that the experimental hardware implemented on an Altera Stratix V FPGA (5SGSMD5K2F40C2N) reaches a 99.83% operating rate of the calculation units for the computation of red/black SOR. In addition, it is approximately six times faster than GPU computing on an NVIDIA GeForce GTX 770 for a 32-bit floating-point calculation of a printed circuit board (PCB) heat conduction simulation, and it is about eight times faster than an NVIDIA Tesla C2075 for the same calculation.

Worldwide, many surveillance systems are in operation for crime deterrence purposes. An effective system should be characterized by requiring low-power consumption, a small storage capacity, and little human effort. Multi-stream tracking on field programmable gate array (FPGA) is important for such surveillance systems. In this paper, we propose multi-stream tracking hardware that can extract moving objects and their motion vectors from a multi-stream received from 64 cameras in real time. The key technology for multi-stream processing is as follows. (1) In order to avoid maintaining the background, we apply a frame difference method. Moreover, the flows of object are calculated by block matching. The flows are effective for analyzing human motion. (2) In order to avoid a bus bottleneck and memory contention in the communication between processing elements (PEs), synchronous shift data transfer (SSDT), which transfers data in the same direction for all PEs, is applied. In this paper, an extended SSDT is proposed for communication between PEs when multi-blocks are processed in one PE. (3) C++ based integrated control code development tool is shown. Control code written in C++ language can easily be assembled and verified by the tool. We implemented the proposed hardware on a Stratix V 5SGXEA7K2F40C2N device. The operating frequency is 50 MHz and the average number of clocks for processing a set of four frames of QVGA images is 394k clocks. The proposed hardware achieved 520 fps, and can process multi-stream video from 64 cameras. The execution time on 3.4 GHz Core i7-3770 CPU was 8.4 fps. Therefore, the proposed hardware was about 62 times faster than that CPU.

Three-dimensional (3D) numerical simulation is an indispensable technique for various analyses of physical phenomena, but it generally requires numerous computation. In this paper, we propose an FPGA-based accelerator for 3D numerical simulations and focus on acceleration of the 3D finite-difference time-domain (FDTD) method. This accelerator consists of a 2D single instruction multiple data (SIMD) array processor, and it can execute 3D parallel computing with little data transfer overhead by applying virtual processing-elements cuboid (VPEC) with synchronous shift data transfer. We demonstrate that the experimental hardware implemented on an Altera Stratix V FPGA (5SGSMD5K2F40C2N) is 3.1 times faster than parallel computing on the NVIDIA Tesla C2075, and it reaches a 94.57% operating rate of the calculation units for the computation of the 3D FDTD method. The proposed accelerator is suitable for multi-chip composition.

Mobile surveillance robots are being used for practical applications, and such mobile robots have gathered attention as an attractive option for video surveillance systems. Several methods for patrol course planning have been proposed, and these methods can generate optimum/approximate patrol courses with certain objectives and conditions such as minimization of observation intervals and the number of cameras. However, these patrol plans do not consider energy consumption. In actual operation, we must consider the power consumption and battery station placement. It is possible to reduce the costs of surveillance systems by reducing the number of battery stations. In this paper, we propose a method for obtaining a patrol course that satisfies the observation constraint, and an optimum location of battery stations, simultaneously. The effectiveness of the proposed method is demonstrated by applying it to some practical examples of airports and towns.

Patrolling by mobile robots is an attractive option for enhancing the reliability of video surveillance systems; mobile robots can observe a wide area effectively during particular intervals, which helps in the early detection of fires and other unusual situations. Moreover, if mobile robots patrol so that anymoving objects cannot exist without being observed, such patrol can be helpful in detecting suspicious individuals. In this paper, we propose a method for determining the minimum number of mobile surveillance cameras and their patrol plans that can realize periodic observation of all regions and moving objects. We demonstrate that we can obtain short patrol courses to achieve reliablemonitoring in our experiments.

Owing to the rapid progress in digital camera technologies, a large amount of image content is distributed on the World Wide Web. Digital camera identification, which is the identification of the source camera of an input image, is becoming increasingly important for presenting evidence in a court and helping police investigations. In recent years, a digital camera identification method using the image sensor's pattern noise has received considerable attention. Photo-response non-uniformity (PRNU) noise is mainly generated by the existence of differences between the sensitivities of pixels, and it is useful as a fingerprint of a camera. However, the PRNU noise of an image is usually contaminated by random noise and scene content and affected by the image processing engine, which inhibits stable identification. In this paper, we propose a novel digital camera identification method using the pairwise magnitude relations of image sensor noise, which are robust to noise contamination. By performing experiments, we demonstrate that the proposed method can identify the source cameras of query images with high accuracy.

Video surveillance systems are becoming increasingly important for crime investigation and deterrence, and the number of cameras installed in public space is increasing. However, many cameras installed at fixed positions are required to observe a wide and complex area. In order to efficiently observe such a wide area at lower cost, mobile robots are an attractive option. In this paper, we propose a method for determining the traveling route of a mobile surveillance camera. Our method is based on mixed integer linear programming and obtains an optimum traveling route such that a camera with a certain visual angle and visual distance can observe the entire region at the shortest intervals. Through our experiments, we apply this method to several artificially generated data and data for a real university campus and demonstrate that effective patrol courses for specified mobile surveillance cameras can be generated.

Along with the popularization of digital cameras, the reliable identification of digital image source is becoming increasingly important as an evidence in a court and some help of investigations. In this paper, we propose an enhanced digital camera identification method using the pixel non-uniformity (PNU) noise of image sensors. By clustering the PNU noises, the proposed method extracts the robust features of image sensors to the random noise, scene content, and image processing engine such as the noise reduction. In the experiments, the proposed method shows the high identification accuracy even for latest digital cameras.

Ball Grid Array packages in which I/O pins are arranged in a grid array pattern realize a number of connections between chips and a printed circuit board, but it takes much time in manual routing. We propose a fast routing method for 2-layer Ball Grid Array packages to support designers. Our method distributes wires evenly on top layer and increases completion ratio of nets by improving via assignment iteratively.

In this paper, we give the necessary and sufficient condition that all nets can be connected by monotonic routes when a net consists of a finger and a ball and fingers are on the two parallel boundaries of the ball grid array package, and propose a monotonic routing method based on this condition. Moreover, we give a necessary condition and a sufficient condition when fingers are on the two orthogonal boundaries, and propose a monotonic routing method based on the necessary condition