[CaCL] Fwd: [Cpl-lab] CPL lab meeting Wednesday 9/23 2pm (EDT) Allyson Ettinger (University of Chicago) "Holding AI to a human standard: Applying cognitive science to evaluation of NLP models"

[Cory Shain]

An upcoming guest lecture by Allyson Ettinger in Roger Levy's lab seems to
be of potential interest to many of us. See below for details.


---------- Forwarded message ---------
From: Peng Qian <pqian at mit.edu>
Date: Mon, Sep 21, 2020 at 9:10 AM
Subject: [Cpl-lab] CPL lab meeting Wednesday 9/23 2pm (EDT) Allyson
Ettinger (University of Chicago) "Holding AI to a human standard: Applying
cognitive science to evaluation of NLP models"
To: cpl-lab <cpl-lab at mit.edu>


Dear CPL-ers,

Our next lab meeting will be* 2pm EDT Wednesday, Sept 23*. Our guest
speaker, Allyson Ettinger, from the University of Chicago, will give a
talk. Details can be found below.

We will be meeting virtually on Zoom at https://urldefense.com/v3/__https://mit.zoom.us/j/517629851__;!!KGKeukY!jmkTjnjkJH5GzRANqHGS2y2WTJHgh_nfsUFzdaiCdILO4P5-cABED8G-gr5NKeR-$   with
password 756945


*Holding AI to a human standard: Applying cognitive science to evaluation
of NLP models*

*Abstract*: In recent years, the field of natural language processing (NLP)
has made what appears to be incredible progress, with models even
surpassing human performance on certain benchmark evaluations. How should
we interpret these advances? Have these models achieved so-called language
"understanding"? For addressing these questions, we can draw critical
insights from the study of language in human cognition -- both because
language "understanding" can only reasonably be defined based on human
language comprehension, and because the controlled methodologies of
psycholinguistics and cognitive science are well-suited to asking
fine-grained linguistic questions about black box systems. In this talk I
discuss a series of projects examining the linguistic capabilities of NLP
models, by comparing against language processing behaviors at a number of
different levels of human cognition, using tests inspired by experimental
methods in cognitive science and psycholinguistics. I focus in particular
on work probing for systematic capturing of compositional meaning
information in models' internal representations, and on work assessing
models' word prediction patterns against human predictive responses of
varying linguistic sensitivity. The results show that by leveraging these
kinds of controlled tests grounded in human language comprehension
patterns, we can identify clear room for growth in state-of-the-art NLP
models, including ways in which optimization of the models' objectives may
ultimately be poorly-suited for achieving human-like language
understanding. I will discuss implications of the findings both for NLP and
for the study of language processing in humans.

*Bio*: Allyson Ettinger is an Assistant Professor in the Department of
Linguistics at the University of Chicago. Her interdisciplinary work draws
on methods and insights from cognitive science, linguistics, and computer
science to examine meaning extraction and predictive processes executed
during language processing in humans and in artificial intelligence
systems. She received her PhD in Linguistics from the University of
Maryland, and spent a year as research faculty at the Toyota Technological
Institute at Chicago (TTIC) before beginning her appointment at the
University of Chicago. She holds an additional courtesy appointment at TTIC.


Best,
Peng


_______________________________________________
Cpl-lab mailing list
Cpl-lab at mit.edu
https://urldefense.com/v3/__http://mailman.mit.edu/mailman/listinfo/cpl-lab__;!!KGKeukY!jmkTjnjkJH5GzRANqHGS2y2WTJHgh_nfsUFzdaiCdILO4P5-cABED8G-goTe-1JG$ 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osu.edu/pipermail/cacl/attachments/20200921/679606ab/attachment.html>