This paper explores how Generative Adversarial Networks (GANs) learn representations of phonological phenomena.We analyze how GANs encode contrastive and non-contrastive nasality in French and English vowels by applying the ciwGAN architecture (Beguš, 2021a).Beguš claims that ciwGAN encodes linguistically meaningful representations with categorical variables in its latent space and manipulating the latent variables shows an almost one to one corresponding control of the phonological features in ciwGAN's generated outputs.However, our results show an interactive effect of latent variables on the features in the generated outputs, which suggests the learned representations in neural networks are different from the phonological representations proposed by linguists.On the other hand, ci-wGAN is able to distinguish contrastive and noncontrastive features in English and French by encoding them differently.Comparing the performance of GANs learning from different languages results in a better understanding of what language specific features contribute to developing language specific phonological representations.We also discuss the role of training data frequencies in phonological feature learning.

This work treats the paradigm discovery problem (PDP)-the task of learning an inflectional morphological system from unannotated sentences.We formalize the PDP and develop evaluation metrics for judging systems.Using currently available resources, we construct datasets for the task.We also devise a heuristic benchmark for the PDP and report empirical results on five diverse languages.Our benchmark system first makes use of word embeddings and string similarity to cluster forms by cell and by paradigm.Then, we bootstrap a neural transducer on top of the clustered data to predict words to realize the empty paradigm slots.An error analysis of our system suggests clustering by cell across different inflection classes is the most pressing challenge for future work.Our code and data are available at https://github.com/ alexerdmann/ParadigmDiscovery.

Classical accounts of child language learning invoke memory limits as a pressure to discover sparse, language-like representations of speech, while more recent proposals stress the importance of prediction for language learning.In this study, we propose a broadcoverage unsupervised neural network model to test memory and prediction as sources of signal by which children might acquire language directly from the perceptual stream.Our model embodies several likely properties of real-time human cognition: it is strictly incremental, it encodes speech into hierarchically organized labeled segments, it allows interactive top-down and bottom-up information flow, it attempts to model its own sequence of latent representations, and its objective function only recruits local signals that are plausibly supported by human working memory capacity.We show that much phonemic structure is learnable from unlabeled speech on the basis of these local signals.We further show that remembering the past and predicting the future both contribute to the linguistic content of acquired representations, and that these contributions are at least partially complementary.

We present the first unsupervised LSTM speech segmenter as a cognitive model of the acquisition of words from unsegmented input.Cognitive biases toward phonological and syntactic predictability in speech are rooted in the limitations of human memory (Baddeley et al., 1998); compressed representations are easier to acquire and retain in memory.To model the biases introduced by these memory limitations, our system uses an LSTMbased encoder-decoder with a small number of hidden units, then searches for a segmentation that minimizes autoencoding loss.Linguistically meaningful segments (e.g.words) should share regular patterns of features that facilitate decoder performance in comparison to random segmentations, and we show that our learner discovers these patterns when trained on either phoneme sequences or raw acoustics.To our knowledge, ours is the first fully unsupervised system to be able to segment both symbolic and acoustic representations of speech.

Deep neural networks (DNNs) have become a standard component in supervised ASR, used in both data-driven feature extraction and acoustic modelling. Supervision is typically obtained from a forced alignment that provides phone class targets, requiring transcriptions and pronunciations. We propose a novel unsupervised DNN-based feature extractor that can be trained without these resources in zero-resource settings. Using unsupervised term discovery, we find pairs of isolated word examples of the same unknown type; these provide weak top-down supervision. For each pair, dynamic programming is used to align the feature frames of the two words. Matching frames are presented as input-output pairs to a deep autoencoder (AE) neural network. Using this AE as feature extractor in a word discrimination task, we achieve 64% relative improvement over a previous state-of-the-art system, 57% improvement relative to a bottom-up trained deep AE, and come to within 23% of a supervised system.

Analyses of filler-gap dependencies usu-ally involve complex syntactic rules or heuristics; however recent results suggest that filler-gap comprehension begins ear-lier than seemingly simpler constructions such as ditransitives or passives. Therefore, this work models filler-gap acquisition as a byproduct of learning word orderings (e.g. SVO vs OSV), which must be done at a very young age anyway in order to extract meaning from language. Specifically, this model, trained on part-of-speech tags, rep-resents the preferred locations of semantic roles relative to a verb as Gaussian mix-tures over real numbers. This approach learns role assignment in filler-gap constructions in a manner con-sistent with current developmental findings and is extremely robust to initialization variance. Additionally, this model is shown to be able to account for a characteristic er-ror made by learners during this period (A and B gorped interpreted as A gorped B). 1

We investigate the order of mention for objects in relational descriptions in visual scenes.Existing work in the visual domain focuses on content selection for text generation and relies primarily on templates to generate surface realizations from underlying content choices.In contrast, we seek to clarify the influence of visual perception on the linguistic form (as opposed to the content) of descriptions, modeling the variation in and constraints on the surface orderings in a description.We find previously-unknown effects of the visual characteristics of objects; specifically, when a relational description involves a visually salient object, that object is more likely to be mentioned first.We conduct a detailed analysis of these patterns using logistic regression, and also train and evaluate a classifier.Our methods yield significant improvement in classification accuracy over a naive baseline.

We present a cognitive model of early lexical acquisition which jointly performs word segmentation and learns an explicit model of phonetic variation.We define the model as a Bayesian noisy channel; we sample segmentations and word forms simultaneously from the posterior, using beam sampling to control the size of the search space.Compared to a pipelined approach in which segmentation is performed first, our model is qualitatively more similar to human learners.On data with variable pronunciations, the pipelined approach learns to treat syllables or morphemes as words.In contrast, our joint model, like infant learners, tends to learn multiword collocations.We also conduct analyses of the phonetic variations that the model learns to accept and its patterns of word recognition errors, and relate these to developmental evidence.

When multiple conversations occur simultaneously, a listener must decide which conversation each utterance is part of in order to interpret and respond to it appropriately. We refer to this task as disentanglement. We present a corpus of Internet Relay Chat dialogue in which the various conversations have been manually disentangled, and evaluate annotator reliability. We propose a graph-based clustering model for disentanglement, using lexical, timing, and discourse-based features. The model's predicted disentanglements are highly correlated with manual annotations. We conclude by discussing two extensions to the model, specificity tuning and conversation start detection, both of which are promising but do not currently yield practical improvements.