Sparse convolutional neural networks (SCNNs) which can prune trivial parameters in the network while maintaining the model accuracy has been proved to be an attractive approach to alleviate the heavy computation of convolutional neural networks (CNNs). However, the invalid data resulting from sparse patterns leads to unnecessary and irregular computation workload, which challenges the efficiency of the underlying hardware accelerators. Therefore, this article proposes an SCNN inference accelerator, which can deal with the imbalanced workload both intra- and interprocessing element (PE). A valid weight encoding (VWE) scheme is proposed to compress sparse weights into dense ones to alleviate the load imbalance intra-PE. Leveraging the VWE scheme, a randomized load rearrangement (RLR) method is proposed to dynamically schedule convolution kernels with similar sparsity into the same computation batch to alleviate the load imbalance inter-PEs. In addition, to reduce off-chip memory accesses, a recurrent weight stationary (RWS) dataflow is proposed, which adopts a small-batch and multichannel strategy to stack data from multiple channels within one off-chip access and let them compute simultaneously thereby enabling efficient reuse of on-chip data. Based on the proposed scheme, an efficient SCNN inference accelerator has been designed and verified on the field-programmable gate array (FPGA). Compared with state-of-the-art works, our design achieves$1.16\times $to$2.77\times $higher digital signal processors (DSPs) efficiency and$1.75\times $to$15\times $higher logic efficiency.

This brief proposes a scalable multistage and multipath architecture for variable number-theoretic transform (NTT). The proposed architecture adopts multiple parallel paths, each of which uses cascaded radix-2 butterfly units (BFUs). The radix-2 scheme simplifies the control logic and the cascaded BFU structure reduces the amount of RAM banks and the frequency of memory accesses. Moreover, a conflict-free and hardware-friendly in-place memory mapping scheme is proposed to ease the adaption to multiple paths, letting it be scalable for various throughputs. Compared with state-of-the-art works, the proposed architecture uses fewer resources and has better area-time product performance without penalty in throughput.

A smart-phone based hand-held indoor positioning system is presented in this paper. The system collects data using the accelerometers, gyroscopes, barometers and gravity sensors embedded in the smart-phone. The accelerometer and gravity data are used for zero-velocity detection and calculating the vertical displacement of each walking step, and then the inverted pendulum model is applied to calculate the step length of every step. The angle of direction is estimated by processing gyroscope data with the quaternion method. The step length and the direction angle of each step are combined to determine the coordinates of each step. The barometer is used for measuring the height information. A Kalman filter is used in zero-velocity-update (ZUPT) to reduce the vertical speed offset caused by accelerometer drift errors. Wifi is also fused in our system. In order to guarantee the accuracy, map information and magnetic field information are used in the navigation systems. The experiment results show that we obtained high precision results with common hand-held smart-phone often seen on market.

Despite the success of many advanced tracking methods in this area, tracking targets with drastic variation of appearance such as deformation, view change and partial occlusion in video sequences is still a challenge in practical applications. In this paper, we take these serious tracking problems into account simultaneously, proposing a dynamic graph based model to track object and its deformable parts at multiple resolutions. The method introduces well learned structural object detection models into object tracking applications as prior knowledge to deal with deformation and view change. Meanwhile, it explicitly formulates partial occlusion by integrating spatial potentials and temporal potentials with an unparameterized occlusion handling mechanism in the dynamic conditional random field framework. Empirical results demonstrate that the method outperforms state-of-the-art trackers on different challenging video sequences.

This paper proposes an innovative and effective hybrid way to recognize human interactions, which incorporates the advantages of both global feature (Motion Context, MC) and Spatio-Temporal (S-T) correlation of local Spatio-Temporal Interest Points (STIPs). The MC feature, which also derives from STIPs, is used to train a random forest where Genetic Algorithm (GA) is applied to the training phase to achieve a good compromise between reliability and efficiency. Besides, we design an effective and efficient S-T correlation based match to assist the MC feature, where MC's structure and a biological sequence matching algorithm are employed to calculate the spatial and temporal correlation score, respectively. Experiments on the UT-Interaction dataset show that our GA search based random forest and S-T correlation based match achieve better performance than some other prevalent machine leaning methods, and that a combination of those two methods outperforms most of the state-of-the-art works.

This paper explores the performance of Neural Network (NN) for human action recognition and proposes a novel hierarchical and boosting-based action recognition system. Specifically, the main contributions of our work are three-fold: (1) A boosted NN based scheme is applied to the human action recognition task for the first time, during which we extend the standard binary AdaBoost algorithm to a multiclass version; (2) A novel hierarchical recognition framework with pre-decision and post-decision modules is proposed, which can significantly enhance the training efficiency as well as the frame-based recognition accuracy; (3) Numerous modified features (both motion and shape features) are utilized and combined in this paper. Experiments on the Weizmann dataset show promising results of our approach in comparison with other state-of-the-art methods.

This paper addresses the problem of small scale target tracking. The divided-by-zero problem in the weight computation of mean shift algorithm and its associated tracking interrupt problem are presented. To tackle these problems, the Parzen window density estimation method is modified to interpolate the histogram of the target candidate. Then the Kullback-Leibler distance is employed as a new similarity measure between the target model and the target candidate. Its corresponding weight computation and new location expressions are derived. On the basis of these works, we propose a new small target tracking algorithm using mean shift framework. The tracking experiments for real world video sequences show that the proposed algorithm can track the target successively and accurately. It can successfully track very small targets with only 6×12 pixels.

Super-resolution reconstruction (SRR) produces a high-resolution image from multiple low-resolution images. Many image SRR algorithms assume that the blurring process, i.e., point spread function (PSF) of the imaging system is known in advance. However, the blurring process is not known or is known only to within a set of parameters in many practical applications. In this letter, we propose an approach for solving the joint blur identification and image SRR based on the principle similar to the variable projection method. The approach can avoid some shortcomings of cyclic coordinate descent optimization procedure. We also propose an efficient implementation based on Lanczos algorithm and Gauss quadrature theory. Experimental results are presented to demonstrate the effectiveness of our method.

In the research of spatial hearing and realization of virtual auditory space, it is important to accurately model the acoustical characteristics of HRTFs (head-related transfer functions) or HRIRs (head-related impulse responses). In our study, for the sake of modeling HRIRs more truthfully, aiming at HRTFs' characteristics in time domain, we managed to carry through adaptive non-linear approximation in the field of wavelet transformation and introduced a new spatial hearing approximation model based on translation-invariant a' trous algorithm and reconstruction from modulus maxima. And the simulation results show that, this is a more effective HRIRs' model, averagely 7.7 dB better than the traditional PCA (principal component analysis) approximation based on relative MSE criterion

This paper addresses the problem of accurately realizing the interpolation of spatially discrete head-related transfer function (HRTF) for synthesis of virtual auditory space. By analyzing the advantages and disadvantages of general bilinear interpolation method in 3D-sound, associating with human auditory system's mechanism of band-pass filtering and consulting critical bands of psychoacoustics, the paper presents a piecewise interpolation method based on log-least square error criterion for HRTF's magnitude to compensate the deficiency of bilinear method in intermediate frequency. As seen from the following simulations, this new method accomplishes preferable results

During the study of spatial hearing, it is requisite to consider how to properly model the acoustical characteristics of HRTFs (head-related transfer functions: HRTFs) or HRIRs (head-related impulse responses: HRIRs) corresponding to certain positions. In our work, we managed to carry through adaptive non-linear approximation in the field of wavelet transformation. The results show that, the HRIRs' adaptive nonlinear approximation model is a more effective data reduction model, faster and averagely 5 dB better than the traditional PCA (Karhunen-Loeve transform) model based on relative error

We propose a novel color image enhancement method, the human visual system controlled color image enhancement and evaluation (HCCIEE) algorithm, and apply it to color images by considering natural image quality metrics. This HCCIEE algorithm is based on a multiscale representation of pattern, luminance and color processing in the human visual system. Experiments illustrate that the HCCIEE algorithm can produce distinguishing details while avoiding artifacts, which often occur in conventional multiscale enhancement methods, as well as producing images that appear as similar as possible to the viewer's perception of actual scenes.

The performance of speech recognition system will be significantly deteriorated because of the mismatches between training and testing conditions. This paper addresses the problem and proposes an algorithm to adapt the mean and covariance of HMM simultaneously within the minimum classification error linear regression (MCELR) framework. Rather than estimating the transformation parameters using maximum likelihood estimation (MLE) or maximum a posteriori, we proposed to use minimum classification error (MCE) as the estimation criterion. The proposed algorithm, called IMCELR (Improved MCELR), has been evaluated on a Chinese digit recognition tasks based on continuous density HMM. The experiments show that the proposed algorithm is more efficient than maximum likelihood linear regression with the same amount of adaptation data.

This paper addresses the problem of accurately realizing the interpolation of spatially discrete head-related transfer function (HRTF) for synthesis of virtual auditory space. By analyzing the advantages and disadvantages of general bilinear interpolation method in 3D-sound, associating with human auditory system's mechanism of band-pass filtering and consulting critical bands of psychoacoustics, the paper presents a piecewise interpolation method based on log-least square error criterion for HRTF's magnitude to compensate the deficiency of bilinear method in intermediate frequency. As seen from the following simulations, this new method accomplishes preferable results