Various attribution methods have been developed to explain deep neural networks (DNNs) by inferring the attribution/importance/contribution score of each input variable to the final output. However, existing attribution methods are often built upon different heuristics. There remains a lack of a unified theoretical understanding of why these methods are effective and how they are related. Furthermore, there is still no universally accepted criterion to compare whether one attribution method is preferable over another. In this paper, we resort to Taylor interactions and for the first time, we discover that fourteen existing attribution methods, which define attributions based on fully different heuristics, actually share the same core mechanism. Specifically, we prove that attribution scores of input variables estimated by the fourteen attribution methods can all be mathematically reformulated as a weighted allocation of two typical types of effects, i.e., independent effects of each input variable and interaction effects between input variables. The essential difference among these attribution methods lies in the weights of allocating different effects. Inspired by these insights, we propose three principles for fairly allocating the effects, which serve as new criteria to evaluate the faithfulness of attribution methods. In summary, this study can be considered as a new unified perspective to revisit fourteen attribution methods, which theoretically clarifies essential similarities and differences among these methods. Besides, the proposed new principles enable people to make a direct and fair comparison among different methods under the unified perspective.

With the rapid development of Artificial Intelligence, Optical Character Recognition(OCR) is applied to analyze and understand the contents of various images, which has a very important effect on online office and makes business more intelligent. As downstream tasks of OCR, information extraction and table analysis are indispensable for acquisition of target information. However, when the texts information are gained from an invoice image or a table image by detection and recognition methods, how to further extract necessary information from a mass of texts or to analyze table information by reconstruction method is still difficult and challenging. In the paper, based on gated graph convolutional networks (GatedGCNs), we propose a novel model to extract key information in documents and reconstruct table information from listing images. Different from manual methods, the GatedGCN-based model considers three kinds of features for the semantic entities, including the position of an entity, the box containing the entity and texts inside the box. The proposed model also considers the relationship between semantic entities, which is a key factor to improve the classification accuracy. Since the update of gated edges in GatedGCN can be treated as a new way to implement attention mechanism, the model can integrate more critical information and discard unnecessary information. Therefore, combining with the node features and the edge features we have extracted, when applying the model on key field extraction (which can be treated as node classification problems) and table reconstruction (that can be treated as link-prediction problems), the model reaches overall excellent results in terms of precision, recall, F1 score and accuracy, evaluated on Medical Invoice, Train Tickets, SciTSR, ICDAR2013 datasets.

Attribution methods have been developed to understand the decision making process of machine learning models, especially deep neural networks, by assigning importance scores to individual features. Existing attribution methods often built upon empirical intuitions and heuristics. There still lacks a general and theoretical framework that not only can unify these attribution methods, but also theoretically reveal their rationales, fidelity, and limitations. To bridge the gap, in this paper, we propose a Taylor attribution framework and reformulate seven mainstream attribution methods into the framework. Based on reformulations, we analyze the attribution methods in terms of rationale, fidelity, and limitation. Moreover, We establish three principles for a good attribution in the Taylor attribution framework, i.e., low approximation error, correct contribution assignment, and unbiased baseline selection. Finally, we empirically validate the Taylor reformulations, and reveal a positive correlation between the attribution performance and the number of principles followed by the attribution method via benchmarking on real-world datasets.

A low-power and fast-transient capacitor-less low dropout regulator (CL-LDO) has been proposed in this paper. A class-AB amplifier with adaptive embedded slew-rate enhancement (SRE) circuit is employed to improve both the transient response performance and load current range. The proposed CL- LDO has been implemented in a 55-nm standard CMOS process and occupies an active chip area of 0.012 mm2. It is capable of delivering 0-10 mA load current and recovering within 0.075 ps under maximum load current change with 1.3 V supply voltage and 0.1 V dropout, while consuming only 1-pA quiescent current, demonstrating its potential capability to be applied in low power duty-cycling wireless sensor applications.

Back-propagation based visualizations have been proposed to interpret deep neural networks (DNNs), some of which produce interpretations with good visual quality. However, there exist doubts about whether these intuitive visualizations are related to network decisions. Recent studies have confirmed this suspicion by verifying that almost all these modified back-propagation visualizations are not faithful to the model's decision-making process. Besides, these visualizations produce vague "relative importance scores", among which low values can't guarantee to be independent of the final prediction. Hence, it's highly desirable to develop a novel back-propagation method that guarantees theoretical faithfulness and produces a quantitative attribution score with a clear understanding. To achieve the goal, we resort to mutual information theory to generate the interpretations, studying how much information of output is encoded in each input neuron. The basic idea is to learn a source signal by back-propagation such that the mutual information between input and output should be as much as possible preserved in the mutual information between input and the source signal. In addition, we propose a Mutual Information Preserving Inverse Network, termed MIP-IN, in which the parameters of each layer are recursively trained to learn how to invert. During the inversion, forward relu operation is adopted to adapt the general interpretations to the specific input. We then empirically demonstrate that the inverted source signal satisfies completeness and minimality property, which are crucial for a faithful interpretation. Furthermore, the empirical study validates the effectiveness of interpretations generated by MIP-IN.

This work focuses on the problem of editing facial images by manipulating specified attributes of interest. To learn latent representations disentangled with respect to specified face attribute, a novel attribute-disentangled generative model is proposed by combining variational autoencoders (VAEs) and generative adversarial networks (GANs). In the proposed model, only two deep mappings are included: an encoder and a decoder, similarly as the counterparts in the context of VAEs. Latent space mapped by the encoder is split into two parts: style space and attribute space. The former represents attribute-irrelevant factors, such as identity, position, illumination and background, etc. The latter represents the attributes, such as hair color, gender, with or without glasses, etc, of which each dimension represents one single attribute. By regarding constraints on the output of the encoder as discriminative objectives, the encoder can act not only as a discriminator that is expected to discriminate a sample is a real or a generated one, but also as an attribute classifier that can discriminate whether a sample has the specified attributes or not. Combining reconstruction and Kullback-Leibler (KL) divergence regularization losses like in VAEs, the adversarial training loss defined for the style and the attribute in the latent space is introduced, which drives the proposed model to generate images whose distribution are close to the real data distribution in the latent space. Finally, the model was evaluated on the CelebA dataset and experimental results showed its effectiveness in disentangling face attributes and generating high-quality face images.

In this paper, an easily implemented semi-supervised graph learning method is presented for dimensionality reduction and clustering, using the most of prior knowledge from limited pairwise constraints. We extend instance-level constraints to space-level constraints to construct a more meaningful graph. By decomposing the (normalized) Laplacian matrix of this graph, to use the bottom eigenvectors leads to new representations of the data, which are hoped to capture the intrinsic structure. The proposed method improves the previous constrained learning methods. Furthermore, to achieve a given clustering accuracy, fewer constraints are required in our method. Experimental results demonstrate the advantages of the proposed method.