Sentence completion is a challenging semantic modeling task in which models must choose the most appropriate word from a given set to complete a sentence.Although a variety of language models have been applied to this task in previous work, none of the existing approaches incorporate syntactic information.In this paper we propose to tackle this task using a pair of simple language models in which the probability of a sentence is estimated as the probability of the lexicalisation of a given syntactic dependency tree.We apply our approach to the Microsoft Research Sentence Completion Challenge and show that it improves on n-gram language models by 8.7 percentage points, achieving the highest accuracy reported to date apart from neural language models that are more complex and expensive to train.