This paper proposes a tree-based convolutional neural network (TBCNN) for discriminative sentence modeling.Our model leverages either constituency trees or dependency trees of sentences.The tree-based convolution process extracts sentences structural features, which are then aggregated by max pooling.Such architecture allows short propagation paths between the output layer and underlying feature detectors, enabling effective structural feature learning and extraction.We evaluate our models on two tasks: sentiment analysis and question classification.In both experiments, TBCNN outperforms previous state-of-the-art results, including existing neural networks and dedicated feature/rule engineering.We also make efforts to visualize the tree-based convolution process, shedding light on how our models work.

This paper aims to compare different regularization strategies to address a common phenomenon, severe overfitting, in embedding-based neural networks for NLP.We chose two widely studied neural models and tasks as our testbed.We tried several frequently applied or newly proposed regularization strategies, including penalizing weights (embeddings excluded), penalizing embeddings, reembedding words, and dropout.We also emphasized on incremental hyperparameter tuning, and combining different regularizations.The results provide a picture on tuning hyperparameters for neural NLP models.

Relation classification is an important research arena in the field of natural language processing (NLP).In this paper, we present SDP-LSTM, a novel neural network to classify the relation of two entities in a sentence.Our neural architecture leverages the shortest dependency path (SDP) between two entities; multichannel recurrent neural networks, with long short term memory (LSTM) units, pick up heterogeneous information along the SDP.Our proposed model has several distinct features: (1) The shortest dependency paths retain most relevant information (to relation classification), while eliminating irrelevant words in the sentence.(2) The multichannel LSTM networks allow effective information integration from heterogeneous sources over the dependency paths.(3) A customized dropout strategy regularizes the neural network to alleviate overfitting.We test our model on the SemEval 2010 relation classification task, and achieve an F 1 -score of 83.7%, higher than competing methods in the literature.

Deep learning has made significant breakthroughs in various fields of artificial intelligence. However, it is still virtually impossible to use deep learning to analyze programs since deep architectures cannot be trained effectively with pure back propagation. In this pioneering paper, we propose the “coding criterion” to build program vector representations, which are the premise of deep learning for program analysis. We evaluate the learned vector representations both qualitatively and quantitatively. We conclude, based on the experiments, the coding criterion is successful in building program representations. To evaluate whether deep learning is beneficial for program analysis, we feed the representations to deep neural networks, and achieve higher accuracy in the program classification task than “shallow” methods. This result confirms the feasibility of deep learning to analyze programs.