From clark.3664 at buckeyemail.osu.edu Sun Apr 3 22:10:53 2022 From: clark.3664 at buckeyemail.osu.edu (Clark, Christian) Date: Mon, 4 Apr 2022 02:10:53 +0000 Subject: [CaCL] Reading for 4/7 Message-ID: Hi CaCLers, On Thursday we will discuss Ryu and Lewis (2021). Title: Accounting for Agreement Phenomena in Sentence Comprehension with Transformer Language Models: Effects of Similarity-based Interference on Surprisal and Attention Paper link: https://arxiv.org/abs/2104.12874 Zoom link: https://osu.zoom.us/j/95536921111?pwd=TG9YdVZ0Wk45R2hCdHhTYk5ubkhIQT09 Abstract: We advance a novel explanation of similarity-based interference effects in subject-verb and reflexive pronoun agreement processing, grounded in surprisal values computed from a pretrained large-scale Transformer model, GPT-2. Specifically, we show that surprisal of the verb or reflexive pronoun predicts facilitatory interference effects in ungrammatical sentences, where a distractor noun that matches in number with the verb or pronoun leads to faster reading times, despite the distractor not participating in the agreement relation. We review the human empirical evidence for such effects, including recent meta-analyses and large-scale studies. We also show that attention patterns (indexed by entropy and other measures) in the Transformer show patterns of diffuse attention in the presence of similar distractors, consistent with cue-based retrieval models of parsing. But in contrast to these models, the attentional cues and memory representations are learned entirely from the simple self-supervised task of predicting the next word. ---- Christian Clark Ph.D. Student Department of Linguistics The Ohio State University https://christian-clark.github.io/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From cheung.179 at buckeyemail.osu.edu Mon Apr 18 14:55:39 2022 From: cheung.179 at buckeyemail.osu.edu (Cheung, Willy) Date: Mon, 18 Apr 2022 18:55:39 +0000 Subject: [CaCL] Reading for 4/21 Message-ID: Hi CaCLers, On Thursday we will discuss Jiant et al 2021. Title: How can we know when language models know? On the calibration of language models for question answering. Paper link: https://direct.mit.edu/tacl/article-pdf/doi/10.1162/tacl_a_00407/1962628/tacl_a_00407.pdf Zoom link: https://osu.zoom.us/j/95536921111?pwd=TG9YdVZ0Wk45R2hCdHhTYk5ubkhIQT09 Abstract: Recent works have shown that language models (LM) capture different types of knowledge regarding facts or common sense. However, because no model is perfect, they still fail to provide appropriate answers in many cases. In this paper, we ask the question, ??How can we know when language models know, with confidence, the answer to a particular query??? We examine this question from the point of view of calibration, the property of a probabilistic model?s predicted probabilities actually being well correlated with the probabilities of correctness. We examine three strong generative models?T5, BART, and GPT-2?and study whether their probabilities on QA tasks are well calibrated, finding the answer is a relatively emphatic no. We then examine methods to calibrate such models to make their confidence scores correlate better with the likelihood of correctness through fine-tuning, post-hoc probability modification, or adjustment of the predicted outputs or inputs. Experiments on a diverse range of datasets demonstrate the effectiveness of our methods. We also perform analysis to study the strengths and limitations of these methods, shedding light on further improvements that may be made in methods for calibrating LMs. -------------- next part -------------- An HTML attachment was scrubbed... URL: From melsner0 at gmail.com Thu Apr 28 08:42:57 2022 From: melsner0 at gmail.com (Micha Elsner) Date: Thu, 28 Apr 2022 08:42:57 -0400 Subject: [CaCL] Fwd: UCI Department of Language Science Summer School on Computational Cognitive Modeling for Language In-Reply-To: References: Message-ID: For anyone interested. ---------- Forwarded message --------- From: SCiL Mail Date: Thu, Apr 28, 2022 at 7:49 AM Subject: UCI Department of Language Science Summer School on Computational Cognitive Modeling for Language To: >From Xin Xie (xxie14 at uci.edu) on behalf of the UCI Language Science Summer School Organizing Committee: The UCI Department of Language Science will be hosting a Summer School on Computational Cognitive Modeling for Language , August 1-5. This was originally going to be in 2020 but now it?s this year. We will immerse students for five days in four daily tutorials: 1. Computational modeling of language learning and adaptation with a focus on phonetics and phonology, 2. Bayesian models of pragmatic language use, 3. Computational models of online language processing using information theory, and 4. Modern neural-network-based machine learning as applied to natural language in the field of natural language processing (NLP). These will be taught by Xin Xie, Connor Mayer, Greg Scontras, Sameer Singh and Richard Futrell. We will teach the theory behind these approaches as well as the practicalities of applying them using datasets and experiments. Students will do hands-on programming exercises in class. The summer school is for postdocs, graduate students, advanced undergraduate students, and other researchers in cognitive science, linguistics, and related fields, especially people who do not have access to resources for learning these skills where they are now. We will also have a series of evening talks and panel discussions with invited speakers, talking about controversial questions and potential directions for the field of computational cognitive science. *Tuition* There is no tuition. *Travel & Accommodation* The Department will provide funding up to $500 for students? travel to and from Irvine, and will provide them with lodging in UC Irvine?s campus housing for the duration of the summer school. *APPLICATION IS OPEN!* Apply here by May 23, 2022. -- To unsubscribe from this group and stop receiving emails from it, send an email to scil+unsubscribe at groups.umass.edu. -------------- next part -------------- An HTML attachment was scrubbed... URL: From joseph.1 at osu.edu Wed Apr 27 14:35:25 2022 From: joseph.1 at osu.edu (Joseph, Brian) Date: Wed, 27 Apr 2022 18:35:25 +0000 Subject: [CaCL] Possible Digital Humanities talk by Chiara Zanchi Message-ID: <321116A8-E965-4A01-B150-CC036AE58902@contoso.com> Hi all ? Chiara Zanchi, an historical linguist from U of Pavia who was a visiting scholar here at OSU 5 years ago for a semester, will be in town for a brief visit next week. She has offered to give a talk if there is interest (though her visit is largely personal and a talk is not a necessary part of her visit). So, would there be sufficient interest in asking her to give a talk on a current project of hers, specifically ?WordNets for Ancient Indo-European Languages?, in which she would introduce two new WordNets for Ancient Greek and Sanskrit (or possibly just the one for Greek); these are part of a family of WordNets for ancient Indo-European languages which also includes the Latin WordNet. Given her schedule, the talk (for which a one-hour slot is needed) would have to be either Tuesday (3 May) between 2:00 ? 3:30 or Wednesday (4 May) between 1:00 ? 3:30. So please let me know by noon on Saturday if such a talk would be of interest to you and if you have a preference regarding the time. If there is sufficient interest, I will work on the logistics. Best, --Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: