We extend and improve upon recent work in structured training for neural network transition-based dependency parsing.We do this by experimenting with novel features, additional transition systems and by testing on a wider array of languages.In particular, we introduce set-valued features to encode the predicted morphological properties and part-ofspeech confusion sets of the words being parsed.We also investigate the use of joint parsing and partof-speech tagging in the neural paradigm.Finally, we conduct a multi-lingual evaluation that demonstrates the robustness of the overall structured neural approach, as well as the benefits of the extensions proposed in this work.Our research further demonstrates the breadth of the applicability of neural network methods to dependency parsing, as well as the ease with which new features can be added to neural parsing models.