We propose a novel data-augmentation technique for neural machine translation based on ROT-k ciphertexts.ROT-k is a simple letter substitution cipher that replaces a letter in the plaintext with the kth letter after it in the alphabet.We first generate multiple ROT-k ciphertexts using different values of k for the plaintext which is the source side of the parallel data.We then leverage this enciphered training data along with the original parallel data via multi-source training to improve neural machine translation.Our method, CipherDAug, uses a co-regularization-inspired training procedure, requires no external data sources other than the original training data, and uses a standard Transformer to outperform strong data augmentation techniques on several datasets by a significant margin.This technique combines easily with existing approaches to data augmentation, and yields particularly strong results in low-resource settings.1

We propose a novel technique that combines alternative subword tokenizations of a single source-target language pair that allows us to leverage multilingual neural translation training methods. These alternate segmentations function like related languages in multilingual translation. Overall this improves translation accuracy for low-resource languages and produces translations that are lexically diverse and morphologically rich. We also introduce a cross-teaching technique which yields further improvements in translation accuracy and cross-lingual transfer between high- and low-resource language pairs. Compared to other strong multilingual baselines, our approach yields average gains of +1.7 BLEU across the four low-resource datasets from the multilingual TED-talks dataset. Our technique does not require additional training data and is a drop-in improvement for any existing neural translation system.

This work describes the first thorough analysis of "header" signs in proto-Elamite, an undeciphered script from 3100-2900 BCE.Headers are a category of signs which have been provisionally identified through painstaking manual analysis of this script by domain experts.We use unsupervised neural and statistical sequence modeling techniques to provide new and independent evidence for the existence of headers, without supervision from domain experts.Having affirmed the existence of headers as a legitimate structural feature, we next arrive at a richer understanding of their possible meaning and purpose by (i) examining which features predict their presence; (ii) identifying correlations between these features and other document properties; and (iii) examining cases where these features predict the presence of a header in texts where domain experts do not expect one (or vice versa).We provide more concrete processes for labeling headers in this corpus and a clearer justification for existing intuitions about document structure in proto-Elamite.

This paper presents a novel analysis of clausal coordination with shared arguments using synchronous tree adjoining grammar (STAG), a pairing of a tree adjoining grammar (TAG) for syntax and a TAG for semantics. In clausal coordination, one or more arguments can be shared by the verbal predicates of the conjuncts, as in Sue likes and Kim hates Pete, where an object argument Pete is shared by likes and hates. As the predicate-argument structure must be represented within each predicative elementary tree in TAG, modelling argument sharing across clauses poses an interesting challenge for TAG. A widely adopted approach within the TAG literature at present is to employ the conjoin operation (Sakar and Joshi, 1996, Proceedings of COLING ’96, 610–615), a non-standard tree-composing operation in TAG. This operation applies across elementary trees to identify and merge the arguments from each clause, yielding a derivation structure in which the shared arguments are combined with multiple elementary trees and a derived tree in which the shared arguments are dominated by multiple verbal projections. In contrast, our STAG analysis pairs a syntactic elementary tree that participates in the derivation of clausal coordination with a semantic elementary tree that includes a |$\lambda$|-term to abstract over the shared argument. This allows the sharing of arguments in coordination to be instantiated in semantics, without being represented in syntax in the form of multiple dominance, utilizing only the standard TAG operations, substitution and adjoining.

We show that an ε-free, chain-free synchronous context-free grammar (SCFG) can be converted into a weakly equivalent synchronous tree-adjoining grammar (STAG) which is prefix lexicalized.This transformation at most doubles the grammar's rank and cubes its size, but we show that in practice the size increase is only quadratic.Our results extend Greibach normal form from CFGs to SCFGs and prove new formal properties about SCFG, a formalism with many applications in natural language processing.