The growing complexity of CNNs demands both hardware acceleration design and dataflow mapping solutions. The large co-design solution space presents a huge challenge. We introduce an analytical model for assessing CNN hardware design and dataflow solutions, using a matrix-based approach. Our co-optimization method, combining nonlinear programming and parallel local search, excels in addressing the power-performance-area tradeoff. The average relative error of our analytical model compared with Timeloop is as small as 1%. Compared to state-of-the-art methods, our co-optimization achieves solutions with average$3.14\times $shorter inference latency,$\mathbf {68.2\%}$less power consumption, and$\mathbf {74\%}$less area on all testcases. It also provides a$200\times $speedup of optimization runtime.

Many real-world CNNs exhibit sparsity, a characteristic that has primarily been utilized in manual design processes and has received little attention in existing automatic optimization techniques. To the best of our knowledge, this paper presents the first systematic investigation of automatic dataflow and hardware optimization for sparse CNN computation. A differentiable PPA (Power Performance Area) model incorporating stochastic modeling of sparse CNN workloads is developed to enable fast nonlinear optimization solving and massively parallel local search-based discretization. Experimental results on public domain testcases demonstrate the efficacy of the proposed approach, achieving an average of 5× and 10× better PPA than the previous work for two different sparsity patterns.

Automatic segmentation of liver tumors is crucial to assist radiologists in clinical diagnosis. While various deep learningbased algorithms have been proposed, such as U-Net and its variants, the inability to explicitly model long-range dependencies in CNN limits the extraction of complex tumor features. Some researchers have applied Transformer-based 3D networks to analyze medical images. However, the previous methods focus on modeling the local information (eg. edge) or global information (eg. morphology) with fixed network weights. To learn and extract complex tumor features of varied tumor size, location, and morphology for more accurate segmentation, we propose a Dynamic Hierarchical Transformer Network, named DHT-Net. The DHT-Net mainly contains a Dynamic Hierarchical Transformer (DHTrans) structure and an Edge Aggregation Block (EAB). The DHTrans first automatically senses the tumor location by Dynamic Adaptive Convolution, which employs hierarchical operations with the different receptive field sizes to learn the features of various tumors, thus enhancing the semantic representation ability of tumor features. Then, to adequately capture the irregular morphological features in the tumor region, DHTrans aggregates global and local texture information in a complementary manner. In addition, we introduce the EAB to extract detailed edge features in the shallow fine-grained details of the network, which provides sharp boundaries of liver and tumor regions. We evaluate DHT-Net on two challenging public datasets, LiTS and 3DIRCADb. The proposed method has shown superior liver and tumor segmentation performance compared to several state-of-the-art 2D, 3D, and 2.5D hybrid models.

The continuous growth of CNN complexity not only intensifies the need for hardware acceleration but also presents a huge challenge. That is, the solution space for CNN hardware design and dataflow mapping becomes enormously large besides the fact that it is discrete and lacks a well behaved structure. Most previous works either are stochastic metaheuristics, such as genetic algorithm, which are typically very slow for solving large problems, or rely on expensive sampling, e.g., Gumbel Softmax-based differentiable optimization and Bayesian optimization. We propose an analytical model for evaluating power and performance of CNN hardware design and dataflow solutions. Based on this model, we introduce a co-optimization method consisting of nonlinear programming and parallel local search. A key innovation in this model is its matrix form, which enables the use of deep learning toolkit for highly efficient computations of power/performance values and gradients in the optimization. In handling power-performance tradeoff, our method can lead to better solutions than minimizing a weighted sum of power and latency. The average relative error of our model compared with Timeloop is as small as 1%. Compared to state-of-the-art methods, our approach achieves solutions with up to 1.7 × shorter inference latency, 37.5% less power consumption, and 3 × less area on ResNet 18. Moreover, it provides a 6.2 × speedup of optimization runtime.

Gesture recognition has raised wide attention in computer vision owing to its many applications. However, the task of video-based large-scale gesture recognition yet faces many challenges, since many gesture-irrelevant factors like the background may disturb the recognition accuracy. To better recognize gestures with large-scale videos, we propose a method based on RGB-D data in this paper, where the “RGB-D” means RGB and depth data captured simultaneously by specific devices like Kinect. To learn gesture details better, we first use an adaptive frame unification strategy to unify the frame number of inputs, and then the RGB and depth data are sent to the C3D model to extract spatiotemporal features, respectively. In order to alleviate the interference of gesture-irrelevant factors, the saliency theory is also employed to generate auxiliary data. Next the features of these data are combined to boost the performance, which can also avoid unreasonable synthetic data, since the dimension of C3D features is uniform. Finally the performances of several classifiers are tested and the best one of SVM classifier is selected to output the ultimate accuracy. Our approach achieves 52.04% and 59.43% accuracy on the validation and testing subset of the Chalearn LAP IsoGD, respectively, both of which outperform our results in the chalearn LAP Large-scale Gesture Recognition Challenge as reported in ICPR 2016.

It is difficult to separate the point symbols from the scanned topographic maps accurately, which brings challenges for the recognition of the point symbols. In this paper, based on the framework of generalized Hough transform (GHT), we propose a new algorithm, which is named shear line segment GHT (SLS-GHT), to recognize the point symbols directly in the scanned topographic maps. SLS-GHT combines the line segment GHT (LS-GHT) and the shear transformation. On the one hand, LS-GHT is proposed to represent the features of the point symbols more completely. Its R-table has double level indices, the first one is the color information of the point symbols, and the other is the slope of the line segment connected a pair of the skeleton points. On the other hand, the shear transformation is introduced to increase the directional features of the point symbols; it can make up for the directional limitation of LS-GHT indirectly. In this way, the point symbols are detected in a series of the sheared maps by LS-GHT, and the final optimal coordinates of the setpoints are gotten from a series of the recognition results. SLS-GHT detects the point symbols directly in the scanned topographic maps, totally different from the traditional pattern of extraction before recognition. Moreover, several experiments demonstrate that the proposed method allows improved recognition in complex scenes than the existing methods.

The gesture recognition has raised attention in computer vision owing to its many applications. However, video-based large-scale gesture recognition still faces many challenges, since many factors like background may disturb the accuracy. To achieve gesture recognition with large-scale videos, we propose a method based on RGB-D data. To learn gesture details better, the inputs are expanded into 32-frame videos first, and then the RGB and depth videos are sent to the C3D model to extract spatiotemporal features respectively. Next these features are combined to boost the performance, which can also avoid unreasonable synthetic data due to the uniform dimension of C3D features. Our approach achieves 49.2% accuracy on the validation subset of the Chalearn LAP IsoGD Database just with a linear SVM classifier. It also outperforms the baseline and other methods in the challenge and wins the first place at 56.9% on testing set.

Superpixels have been widely used in lots of computer vision and image processing tasks but rarely used in topographic map processing due to the complex distribution of geographic elements in this kind of images. We propose a novel superpixel-generating method based on guided watershed transform (GWT). Before GWT, the cues of geographic element distribution and boundaries between different elements need to be obtained. A linear feature extraction method based on a compound opposite Gaussian filter and a shear transform is presented to acquire the distribution information. Meanwhile, a boundary detection method, which based on the color-opponent mechanisms of the visual system, is employed to get the boundary information. Then, both linear features and boundaries are input to the final partition procedure to obtain superpixels. The experiments show that our method has the best performance in shape control, size control, and boundary adherence, among all the comparison methods, which are classic and state of the art. Furthermore, we verify the low complexity and low cost of memory in our method through experiments, which makes it possible to deal with large-scale topographic maps.

AdaBoost has attracted much attention in the machine learning community because of its excellent performance in combining weak classifiers into strong classifiers. However, AdaBoost tends to overfit to the noisy data in many applications. Accordingly, improving the antinoise ability of AdaBoost plays an important role in many applications. The sensitiveness to the noisy data of AdaBoost stems from the exponential loss function, which puts unrestricted penalties to the misclassified samples with very large margins. In this paper, we propose two boosting algorithms, referred to as RBoost1 and RBoost2, which are more robust to the noisy data compared with AdaBoost. RBoost1 and RBoost2 optimize a nonconvex loss function of the classification margin. Because the penalties to the misclassified samples are restricted to an amount less than one, RBoost1 and RBoost2 do not overfocus on the samples that are always misclassified by the previous base learners. Besides the loss function, at each boosting iteration, RBoost1 and RBoost2 use numerically stable ways to compute the base learners. These two improvements contribute to the robustness of the proposed algorithms to the noisy training and testing samples. Experimental results on the synthetic Gaussian data set, the UCI data sets, and a real malware behavior data set illustrate that the proposed RBoost1 and RBoost2 algorithms perform better when the training data sets contain noisy data.

Machine learning is among the most popular methods in designing unknown and variant malware detection algorithms. However, most of the existing methods take a single type of features to build binary classifiers. In practice, these methods have limited ability in depicting malware characteristics and the binary classification suffers from inadequate sampling of benign samples and extremely imbalanced training samples when detecting malware. In this paper, we present a malware detection Framework based on ENsemble One-Class Learning, namely FENOC. It uses hybrid features at different semantic layers to ensure a comprehensive insight of the program to be analyzed. We construct the malware detector by a novel learning algorithm called Cost-sensitive Twin One-class Classifier (CosTOC), which uses a pair of one-class classifiers to describe malware and benign programs respectively. CosTOC is more flexible and robust in comparison to conventional binary classifiers when training samples are extremely imbalanced or the benign programs are inadequately sampled. Finally, random subspace method and clustering-based ensemble method are developed to enhance the generalization ability of CosTOC. Experimental results show that FENOC gives a comparative detection rate and a lower false positive rate than many other binary classification algorithms, especially when the detector are trained with imbalanced data, or evaluated in terms of false positive rate.

In a learning process, features play a fundamental role. In this paper, we propose a Boosting-based feature selection algorithm called BoostFS. It extends AdaBoost which is designed for classification problems to feature selection. BoostFS maintains a distribution over training samples which is initialized from the uniform distribution. In each iteration, a decision stump is trained under the sample distribution and then the sample distribution is adjusted so that it is orthogonal to the classification results of all the generated stumps. Because a decision stump can also be regarded as one selected feature, BoostFS is capable to select a subset of features that are irrelevant to each other as much as possible. Experimental results on synthetic datasets, five UCI datasets and a real malware detection dataset all show that the features selected by BoostFS help to improve learning algorithms in classification problems, especially when the original feature set contains redundant features.