Recently, pre-trained language models have achieved great success in the field of natural language processing (NLP). NLP downstream tasks have also seen remarkable improvement with the help of pre-trained models. However, syntactic information, such as dependency trees, is ignored in the process of fine-tuning, even though this prior information naturally exists in sentences. In this paper, we propose a model to exploit syntactic information to boost the fine-tuning of pre-trained models on downstream tasks. Our model encodes the syntactic information using a graph convolutional network (GCN) and then integrates it into text representations explicitly. In particular, we build a dependency graph from the word connections in the dependency tree obtained from an off-the-shelf dependency parser, aiming to preserve the dependency and syntactic structure in a sentence. We conduct extensive experiments on four types of downstream tasks. The empirical results show that our method significantly outperforms the strong pre-trained baseline models. Of note, our base-scale models outperform the large-scale baselines, with ob-vious performance gains, and an approximately 3-fold difference in the size of the model parameters is observed.

Multimodal models have been proven to outperform text-based models on learning semantic word representations. According to psycholinguistic theory, there is a graphical relationship among the modalities of language, and in recent years, the graph convolution network (GCN) has been proven to have substantial advantages in the extraction of non-European spatial features. This inspires us to propose a new multimodal word representation model, namely, GCNW, which uses the graph convolutional network to incorporate the phonetic and syntactic information into the word representation. We use a greedy strategy to update the modality-relation matrix in the GCN, and we train the model through unsupervised learning. We evaluated the proposed model on multiple downstream NLP tasks, and various experimental results demonstrate that the GCNW outperforms strong unimodal baselines and state-of-the-art multimodal models. We make the source code of both models available to encourage reproducible research.

Multimodal models have been proven to outperform text-based models on learning semantic word representations. According to psycholinguistic theory, there is a graphical relationship among the modalities of language, and in recent years, the graph convolution network (GCN) has been proven to have substantial advantages in the extraction of non-European spatial features . This inspires us to propose a new multimodal word representation model, namely, GCNW, which uses the graph convolutional network to incorporate the phonetic and syntactic information into the word representation. We use a greedy strategy to update the modality-relation matrix in the GCN, and we train the model through unsupervised learning . We evaluated the proposed model on multiple downstream NLP tasks, and various experimental results demonstrate that the GCNW outperforms strong unimodal baselines and state-of-the-art multimodal models. We make the source code of both models available to encourage reproducible research.