Deep neural networks (DNNs) have been successfully applied to a great variety of applications, ranging from small IoT devices to large scale services in a data center. In order to improve the efficiency of processing these DNN models, dedicated hardware accelerators are required for all these scenarios. Theoretically, there exists an optimized acceleration architecture for each application. However, considering the cost of chip design and corresponding tool-chain development, researchers need to trade off between efficiency and generality. In this work, we demonstrate that it is practical to use a unified architecture, called Ascend, to support those applications, ranging from IoT devices to data-center services. We provide a lot of design details to explain that the success of Ascend relies on contributions from different levels. First, heterogeneous computing units are employed to support various DNN models. And the datapath is adapted according to the requirement of computing and data access. Second, when scaling the Ascend architecture from a single core to a cluster containing thousands of cores, it involves design efforts, such as memory hierarchy and system level integration. Third, a multi-tier compiler, which provides flexible choices for developers, is the last critical piece. Experimental results show that using accelerators based on the Ascend architecture can achieve comparable or even better performance in different applications. In addition, various chips based on the Ascend architecture have been successfully commercialized. More than 100 million chips have been used in real products.

The code summarization task aims to generate brief descriptions of source code automatically. It is beneficial for developers to understand source code. However, almost all of current code summarization approaches may generate low-quality (BLEU4<40) summaries, which will mislead developers. Previous work has shown that it is possible to conduct quality assurance for document generation (QA4DG) and improve the practicability of document generation approaches. Code summarization can also be regarded as a document generation task. This work aims to investigate whether QA4DG approaches can be leveraged to improve code summarization. Specifically, we first investigate whether existing QA4DG approaches can be plugged in code summarization approaches. We find that an automated quality assurance framework for commit message generation named QACom performs best. In-spired by the idea behind QAcom, we propose an ensemble code summarization approach called Ensum. Precisely, given a code snippet, Ensum first uses current code summarization approaches to generate candidate summaries. Then, Ensum predicts the quality of each candidate summary using a collaborative filtering-based component and a retrieval-based component and selects the best candidate summary as the output. Experimental results on two public datasets show that Ensum outperforms three state-of-the-art single approaches and one ensemble approach for code summarization in terms of BLEU-4, METEOR, and ROUGE-L.

Many aesthetic models in multimedia and computer vision suffer from two shortcomings: 1) the low descriptiveness and interpretability 1 of the hand-crafted aesthetic criteria (i.e., fail to indicate region-level aesthetics) and 2) the difficulty of engineering aesthetic features adaptively and automatically toward different image sets. To remedy these problems, we develop a deep architecture to learn aesthetically relevant visual attributes from Flickr, 2 which are localized by multiple textual attributes in a weakly supervised setting. More specifically, using a bag-of-words representation of the frequent Flickr image tags, a sparsity-constrained subspace algorithm discovers a compact set of textual attributes (i.e., each textual attribute is a sparse and linear representation of those frequent image tags) for each Flickr image. Then, a weakly supervised learning algorithm projects the textual attributes at image-level to the highly-responsive image patches. These patches indicate where humans look at appealing regions with respect to each textual attribute, which are employed to learn the visual attributes. Psychological and anatomical studies have demonstrated that humans perceive visual concepts in a hierarchical way. Therefore, we normalize these patches and further feed them into a five-layer convolutional neural network to mimic the hierarchy of human perceiving the visual attributes. We apply the learned deep features onto applications like image retargeting, aesthetics ranking, and retrieval. Both subjective and objective experimental results thoroughly demonstrate the superiority of our approach.1 In this paper, "describing" and "interpretability" means the ability of seeking region-level representation of each mined textual attribute, i.e., a sparse and linear representation of those frequent image tags. 2 https://www.flickr.com/.

Video saliency prediction is an indispensable yet challenging technique which can facilitate various applications, such as video surveillance, autonomous driving, and realistic rendering. Based on the popularity of embedded cameras, we in this paper predict region-level saliency from videos by leveraging human gaze locations recorded using a camera, (e.g., those equipped on an iMAC and laptop PC). Our proposed camera-assisted mechanism improves saliency prediction by discovering human attended regions inside a video clip. It is orthogonal to the current saliency models, i.e., any existing video/image saliency model can be boosted by our mechanism. First of all, the spatial-and temporal-level visual features are exploited collaboratively for calculating an initial saliency map. We notice that the current saliency models are not sufficiently adaptable to the variations in lighting, different view angles, and complicated backgrounds. Therefore, assisted by a camera tracking human gaze movements, a non-negative matrix factorization algorithm is designed to accurately localize the semantically/visually salient video regions perceived by humans. Finally, the learned human gaze locations as well as the initial saliency map are integrated to optimize video saliency calculation. Empirical results thoroughly demonstrated that: 1) our approach achieves the state-of-the-art video saliency prediction accuracy by outperforming 11 mainstream algorithms considerably and 2) our method can conveniently and successfully enhance video retargeting, quality estimation, and summarization.

Accurately detecting esthetic communities from a large number of Internet users (e.g., Flickr11www.flickr.com.or Picasa22picasa.google.com.users) is a useful technique that can facilitate several applications, such as image retargeting, visual esthetic assessment, and fashion recommendation. Conventional approaches cannot appropriately handle this task due to the following challenges: first, it is difficult to online update the detected esthetic communities since these photos may uploaded/removed frequently; second, human visual perception is essential to describe esthetic characteristics, but integrating it into an existing mining algorithm is challenging; and third, flat models cannot encode human visual perception precisely, especially for those sophisticated sceneries. To solve these problems, we propose deep perception graph analytics, an incremental pipeline where the esthetic relations among users are described by utilizing their gaze shifting paths (GSPs). Specifically, we first propose an aggregation-based deep network that formulates GSP representation into a unified framework. Afterward, the deep perception graph is constructed where the esthetic discrepancy between users is measured by their GSP distributions. Accordingly, we adopt a dense subgraph discovery algorithm that efficiently detects the communities belonging to each esthetic style. Finally, an online Gaussian mixture model (GMM) learning model is designed, which dynamically updates the GMM parameters in order to describe esthetic communities given the photos are uploaded/removed on the fly. Experiments on a million-scale image set crawled from Flickr demonstrate the efficiency and effectiveness of our method.

It is difficult to develop a computational model that can accurately predict the quality of the video summary. This paper proposes a novel algorithm to summarize one-shot landmark videos. The algorithm can optimally combine multiple unedited consumer video skims into an aesthetically pleasing summary. In particular, to effectively select the representative key frames from multiple videos, an active learning algorithm is derived by taking advantage of the locality of the frames within each video. Toward a smooth video summary, we define skimlet, a video clip with adjustable length, starting frame, and positioned by each skim. Thereby, a probabilistic framework is developed to transfer the visual cues from a collection of aesthetically pleasing photos into the video summary. The length and the starting frame of each skimlet are calculated to maximally smoothen the video summary. At the same time, the unstable frames are removed from each skimlet. Experiments on multiple videos taken from different sceneries demonstrated the aesthetics, the smoothness, and the stability of the generated summary.