Deep neural networks have continually surpassed traditional methods on a variety of computer vision tasks. Though deep neural networks are very powerful, the large number of parameters and complex structures consume considerable storage and calculation time, making it hard to deploy with limited resources. To tackle this issue, many recently proposed knowledge distillation approaches are aimed at obtaining a small student network to imitate a large teacher network. However, the student network structure is pre-defined and may be hard to train. In this paper, we propose to distill knowledge with an evolutional student network population. The population is initialized with several basic structures and each network is evaluated by the imitation ability (i.e., fitness) to the teacher network. By reusing the weights, we provide five enhancement options to strengthen the networks with high fitness and abandon the weak ones. By changing the fitness criterion, we can select networks to meet different requirements, such as balancing size and accuracy. This allows one to find a superior student network structure that better imitates the teacher model from various aspects with easier training. The experimental results demonstrate the proposed method can achieve superior performance of knowledge distillation with flexible student structures.

It is desirable to include more controllable attributes to enhance the diversity of generated responses in open-domain dialogue systems. However, existing methods can generate responses with only one controllable attribute or lack a flexible way to generate them with multiple controllable attributes. In this paper, we propose a Progressively trained Hierarchical Encoder-Decoder (PHED) to tackle this task. More specifically, PHED deploys Conditional Variational AutoEncoder (CVAE) on Transformer to include one aspect of attributes at one stage. A vital characteristic of the CVAE is to separate the latent variables at each stage into two types: a global variable capturing the common semantic features and a specific variable absorbing the attribute information at that stage. PHED then couples the CVAE latent variables with the Transformer encoder and is trained by minimizing a newly derived ELBO and controlled losses to produce the next stage's input and produce responses as required. Finally, we conduct extensive evaluations to show that PHED significantly outperforms the state-of-the-art neural generation models and produces more diverse responses as expected.

Aspect-based sentiment classification is the task of predicting the sentiment tendency of a text toward a given aspect. Existing works on this task mainly focus on aspect-relevant information. In contrast, we design a model (BAT) which could extract overall Background information as well as Aspect-relevant informaTion. To make the BAT model learn better semantic representation of the given text, we introduce two auxiliary tasks (dependency neighborhood prediction and part-of-speech tagging). These auxiliary tasks are used to train the model together with the main sentiment classification task. Experiments on three benchmark datasets demonstrate that our method is effective and the proposed model achieves substantial performance improvements over comparison models.

Recently developed large pre-trained language models, e.g., BERT, have achieved remarkable performance in many downstream natural language processing applications. These pre-trained language models often contain hundreds of millions of parameters and suffer from high computation and latency in real-world applications. It is desirable to reduce the computation overhead of the models for fast training and inference while keeping the model performance in downstream applications. Several lines of work utilize knowledge distillation to compress the teacher model to a smaller student model. However, they usually discard the teacher's knowledge when in inference. Differently, in this paper, we propose RemERT to leverage the knowledge learned from the teacher, i.e., facilitating the pre-computed BERT representation on the reference sample and compressing BERT into a smaller student model. To guarantee our proposal, we provide theoretical justification on the loss function and the usage of reference samples. Significantly, the theoretical result shows that including the pre-computed teacher's representations on the reference samples indeed increases the mutual information in learning the student model. Finally, we conduct the empirical evaluation and show that our RemERT can beat the vanilla TinyBERT over 8.1 % and achieves more than 94% of the performance of$\mathbf{BERT}_{\mathbf{BASE}}$on the GLUE benchmark. Meanwhile, RemERT is$\mathbf{7.4x}$smaller and$\mathbf{9.5x}$faster on inference than$\mathbf{BERT}_{\mathbf{BASE}}$.

Emotion dynamics modeling is a significant task in emotion recognition in conversation. It aims to predict conversational emotions when building empathetic dialogue systems. Existing studies mainly develop models based on Recurrent Neural Networks (RNNs). They cannot benefit from the power of the recently-developed pre-training strategies for better token representation learning in conversations. More seriously, it is hard to distinguish the dependency of interlocutors and the emotional influence among interlocutors by simply assembling the features on top of RNNs. In this paper, we develop a series of BERT-based models to specifically capture the inter-interlocutor and intra-interlocutor dependencies of the conversational emotion dynamics. Concretely, we first substitute BERT for RNNs to enrich the token representations. Then, a Flat-structured BERT (F-BERT) is applied to link up utterances in a conversation directly, and a Hierarchically-structured BERT (H-BERT) is employed to distinguish the interlocutors when linking up utterances. More importantly, a Spatial-Temporal-structured BERT, namely ST-BERT, is proposed to further determine the emotional influence among interlocutors. Finally, we conduct extensive experiments on two popular emotion recognition in conversation benchmark datasets and demonstrate that our proposed models can attain around 5% and 10% improvement over the state-of-the-art baselines, respectively.

Automatically and accurately identifying user intents and filling the associated slots from their spoken language are critical to the success of dialogue systems. Traditional methods require manually defining the DOMAIN-INTENT-SLOT schema and asking many domain experts to annotate the corresponding utterances, upon which neural models are trained. This procedure brings the challenges of information sharing hindering, out-of-schema, or data sparsity in open domain dialogue systems. To tackle these challenges, we explore a new task of automatic intent-slot induction and propose a novel domain-independent tool. That is, we design a coarse-to-fine three-step procedure including Role-labeling, Concept-mining, And Pattern-mining (RCAP): (1) role-labeling: extracting key phrases from users’ utterances and classifying them into a quadruple of coarsely-defined intent-roles via sequence labeling; (2) concept-mining: clustering the extracted intent-role mentions and naming them into abstract fine-grained concepts; (3) pattern-mining: applying the Apriori algorithm to mine intent-role patterns and automatically inferring the intent-slot using these coarse-grained intent-role labels and fine-grained concepts. Empirical evaluations on both real-world in-domain and out-of-domain datasets show that: (1) our RCAP can generate satisfactory SLU schema and outperforms the state-of-the-art supervised learning method; (2) our RCAP can be directly applied to out-of-domain datasets and gain at least 76% improvement of F1-score on intent detection and 41% improvement of F1-score on slot filling; (3) our RCAP exhibits its power in generic intent-slot extractions with less manual effort, which opens pathways for schema induction on new domains and unseen intent-slot discovery for generalizable dialogue systems.

Recently, pre-trained language models such as BERT have shown state-of-the-art accuracies in text matching.When being applied to IR (or QA), the BERT-based matching models need to online calculate the representations and interactions for all query-candidate pairs.The high inference cost has prohibited the deployments of BERT-based matching models in many practical applications.To address this issue, we propose a novel BERT-based text matching model, in which the representations and the interactions are decoupled.Then, the representations of the candidates can be calculated and stored offline, and directly retrieved during the online matching phase.To conduct the interactions and generate final matching scores, a lightweight attention network is designed.Experiments based on several large scale text matching datasets show that the proposed model, called FASTMATCH, can achieve up to 100X speed-up to BERT and RoBERTa at the online matching phase, while keeping more up to 98.7% of the performance.

Text-to-SQL systems offers natural language interfaces to databases, which can automatically generates SQL queries given natural language questions.On the WikiSQL benchmark, state-ofthe-art text-to-SQL systems typically take a slot-filling approach by building several specialized models for each type of slot.Despite being effective, such modularized systems are complex and also fall short in jointly learning for different slots.To solve these problems, this paper proposes a novel approach that formulates the task as a question answering problem, where different slots are predicted by a unified machine reading comprehension (MRC) model.For this purpose, we use a BERT-based MRC model, which can also benefit from intermediate training on other MRC datasets.The proposed method can achieve competitive results on WikiSQL, suggesting it being a promising direction for text-to-SQL.

On the WikiSQL benchmark, state-of-the-art text-to-SQL systems typically take a slotfilling approach by building several dedicated models for each type of slots.Such modularized systems are not only complex but also of limited capacity for capturing interdependencies among SQL clauses.To solve these problems, this paper proposes a novel extraction-linking approach, where a unified extractor recognizes all types of slot mentions appearing in the question sentence before a linker maps the recognized columns to the table schema to generate executable SQL queries.Trained with automatically generated annotations, the proposed method achieves the first place on the WikiSQL benchmark.

We introduce a framework of Monte Carlo Tree Search with Double-q Dueling network (MCTS-DDU) for task-completion dialogue policy learning. Different from the previous deep model-based reinforcement learning methods, which uses background planning and may suffer from low-quality simulated experiences, MCTS-DDU performs decision-time planning based on dialogue state search trees built by Monte Carlo simulations and is robust to the simulation errors. Such idea arises naturally in human behaviors, e.g. predicting others' responses and then deciding our own actions. In the simulated movie-ticket booking task, our method outperforms the background planning approaches significantly. We demonstrate the effectiveness of MCTS and the dueling network in detailed ablation studies, and also compare the performance upper bounds of these two planning methods.

Facial emotion recognition is an inherently difficult problem, not only due to individual facial feature diversity but also racial and cultural differences. Compound facial emotion recognition makes the problem even more difficult because the discrimination between the dominant emotion and the complementary emotion is usually weak. To mitigate this problem, we propose the two-stage recognition method. The first stage is a coarse recognition stage and the second stage is a fine recognition stage. By doing this the classification for symmetrical emotion labels is enhanced. Beyond this, we make good use of the context information existing in the same label to correct some false-classification so that the recognition can be more robust. In this contest, our method can achieve very promising results.

Face anti-spoofing is an important problem for both academic research and industrial face recognition systems. Most of the existing face anti-spoofing methods take it as a classification task on individual static images, where motion pattern differences in consecutive real or fake face sequences are ignored. In this work, we propose a novel method to identify spoofing patterns using motion information. Different from previous methods, the proposed method makes the real or fake decision on the disentangled feature level, based on the observation that motion and spoofing pattern features could be disentangled from original image frames. We design a representation disentangling framework for this task, which is able to reconstruct both real and fake face sequences from the input. Meanwhile, the disentangled representations could be used to classify whether the input faces are real or fake. We perform several experiments on public face anti-spoofing datasets. The proposed method achieves SOTA results compared with existing methods.

Stock Trend Prediction(STP) has drawn wide attention from various fields, especially Artificial Intelligence. Most previous studies are single-scale oriented which results in information loss from a multi-scale perspective. In fact, multi-scale behavior is vital for making intelligent investment decisions. A mature investor will thoroughly investigate the state of a stock market at various time scales. To automatically learn the multi-scale information in stock data, we propose a Multi-scale Two-way Deep Neural Network. It learns multi-scale patterns from two types of scale-information, wavelet-based and downsampling-based, by eXtreme Gradient Boosting and Recurrent Convolutional Neural Network, respectively. After combining the learned patterns from the two-way, our model achieves state-of-the-art performance on FI-2010 and CSI-2016, where the latter is our published long-range stock dataset to help future studies for STP task. Extensive experimental results on the two datasets indicate that multi-scale information can significantly improve the STP performance and our model is superior in capturing such information.

The task of crowd counting is to estimate the accurate number of people in photos taken from unconstrained surveillance scenes. It is in general a challenging problem due to the input scale variations and perspective distortions. Previous methods make efforts to enhance the representation ability by using multi-scale features of the scene pictures. However, most of these methods directly add or fuse the features, in which the influences of different feature sizes are equally considered. In this paper, we propose a novel architecture called adaptive context learning network (ACLNet) to incorporate context of features in multiple levels. In this architecture, the original image features are enhanced by a multi-level feature generating module, and then the multi-level features are up-sampled to the same size and re-weighted for fusing. The ACLNet incorporates the context information existed in sub-regions of various scales adaptively, thus it is able to enhance the representative ability of multi-level features. We perform several experiments on public ShanghaiTech (A and B), UCF_CC_50 and NWPU-crowd datasets. Our proposed ACLNet achieves the state-of-the-art results compared with existing methods.

Visualizing dynamic graphs are challenging due to the difficulty to preserving a coherent mental map of the changing graphs. In this paper, we propose a novel layout algorithm which is capable of maintaining the overall structure of a sequence graphs. Through Laplacian constrained distance embedding, our method works online and maintains the aesthetic of individual graphs and the shape similarity between adjacent graphs in the sequence. By preserving the shape of the same graph components across different time steps, our method can effectively help users track and gain insights into the graph changes. Two datasets are tested to demonstrate the effectiveness of our algorithm.

With the exploding of Internet information, semantic technologies are proposed to help people find what they want. Semantic search requires domain-oriented ontology and user-defined workflow to process online data. Mostly, semantic search are focus on user query optimization. The semantic of online data is usually considered less than the domain ontology to processing query, and the search targets are mostly online service. Modeling approaches are required by semantic search to find and process online data in certain business scenario. So we proposed to build online services' semantics from its' application level, business level and information level. These semantic aspects of the service semantic are used to enhance the smartness by application semantic and business semantic. Concept semantic is for optimizing accuracy and precision when querying online service combined with the domain ontology. In this paper, the related concepts and implementation framework of the multi-dimension service modeling are applied in a travel application for user customize their journey to enhance the smartness, accuracy and precision.

This paper presents a development environment for rapid development of multi-touch sand table applications. The environment is built on a sophisticated business logic model, and with an extended MVC software development framework. The development framework is consisted of multi-touch control layer, business logic layer and dynamic GUI layer, which support efficient collaborative development. The development environment has been developed with dynamic controller tag library, multi-touch control library, market simulation logic library, and domain case template library, to facilitate application maintenance and development. Four multi-touch sand table applications have been developed to be practical training products using the proposed environment.