From clark.3664 at buckeyemail.osu.edu Fri Nov 4 11:40:11 2022 From: clark.3664 at buckeyemail.osu.edu (Clark, Christian) Date: Fri, 4 Nov 2022 15:40:11 +0000 Subject: [CaCL] Plans for next two weeks Message-ID: Hi, fellow CaCLers, We will not be meeting next Thursday, November 10. The week after that, November 17, I will lead discussion on Warstadt and Bowman 2022. Title: What Artificial Neural Networks Can Tell Us About Human Language Acquisition Link: https://arxiv.org/pdf/2208.07998.pdf Abstract: Rapid progress in machine learning for natural language processing has the potential to transform debates about how humans learn language. However, the learning environments and biases of current artificial learners and humans diverge in ways that weaken the impact of the evidence obtained from learning simulations. For example, today?s most effective neural language models are trained on roughly one thousand times the amount of linguistic data available to a typical child. To increase the relevance of learnability results from computational models, we need to train model learners without significant advantages over humans. If an appropriate model successfully acquires some target linguistic knowledge, it can provide a proof of concept that the target is learnable in a hypothesized human learning scenario. Plausible model learners will enable us to carry out experimental manipulations to make causal inferences about variables in the learning environment, and to rigorously test poverty-of-the-stimulus-style claims arguing for innate linguistic knowledge in humans on the basis of speculations about learnability. Comparable experiments will never be possible with human subjects due to practical and ethical considerations, making model learners an indispensable resource. So far, attempts to deprive current models of unfair advantages obtain sub-human results for key grammatical behaviors such as acceptability judgments. But before we can justifiably conclude that language learning requires more prior domain-specific knowledge than current models possess, we must first explore non-linguistic inputs in the form of multimodal stimuli and multi-agent interaction as ways to make our learners more efficient at learning from limited linguistic input. **** Cacklingly, Christian ---- Christian Clark Ph.D. Student Department of Linguistics The Ohio State University -------------- next part -------------- An HTML attachment was scrubbed... URL: From oh.531 at buckeyemail.osu.edu Tue Nov 22 16:13:20 2022 From: oh.531 at buckeyemail.osu.edu (Oh, Byung-Doh) Date: Tue, 22 Nov 2022 21:13:20 +0000 Subject: [CaCL] 12/1: Linguistic Frameworks Go Toe-to-Toe at Neuro-Symbolic Language Modeling Message-ID: Dear CaCLers, Next week, we'll discuss the following paper: Linguistic Frameworks Go Toe-to-Toe at Neuro-Symbolic Language Modeling https://aclanthology.org/2022.naacl-main.325.pdf We examine the extent to which, in principle, different syntactic and semantic graph representations can complement and improve neural language modeling. Specifically, by conditioning on a subgraph encapsulating the locally relevant sentence history, can a model make better next-word predictions than a pretrained sequential language model alone? With an ensemble setup consisting of GPT-2 and ground-truth graphs from one of 7 different formalisms, we find that the graph information indeed improves perplexity and other metrics. Moreover, this architecture provides a new way to compare different frameworks of linguistic representation. In our oracle graph setup, training and evaluating on English WSJ, semantic constituency structures prove most useful to language modeling performance?outpacing syntactic constituency structures as well as syntactic and semantic dependency structures. Happy Thanksgiving! Byung-Doh ================= Byung-Doh Oh (he/him/his) Ph.D. Student Department of Linguistics The Ohio State University -------------- next part -------------- An HTML attachment was scrubbed... URL: