From oh.531 at buckeyemail.osu.edu Thu Nov 2 14:19:56 2023 From: oh.531 at buckeyemail.osu.edu (Oh, Byung-Doh) Date: Thu, 2 Nov 2023 18:19:56 +0000 Subject: [CaCL] CaCL 11/16: Human-like systematic generalization through a meta-learning neural network Message-ID: Hello everyone, Next week (11/9), CaCL will not meet. On the following week (11/16), I will lead discussion of the following paper, which coincidentally seems to have a lot of connections to the paper we discussed today: Human-like systematic generalization through a meta-learning neural network https://www.nature.com/articles/s41586-023-06668-3 The power of human language and thought arises from systematic compositionality?the algebraic ability to understand and produce novel combinations from known components. Fodor and Pylyshyn famously argued that artificial neural networks lack this capacity and are therefore not viable models of the mind. Neural networks have advanced considerably in the years since, yet the systematicity challenge persists. Here we successfully address Fodor and Pylyshyn?s challenge by providing evidence that neural networks can achieve human-like systematicity when optimized for their compositional skills. To do so, we introduce the meta-learning for compositionality (MLC) approach for guiding training through a dynamic stream of compositional tasks. To compare humans and machines, we conducted human behavioural experiments using an instruction learning paradigm. After considering seven different models, we found that, in contrast to perfectly systematic but rigid probabilistic symbolic models, and perfectly flexible but unsystematic neural networks, only MLC achieves both the systematicity and flexibility needed for human-like generalization. MLC also advances the compositional skills of machine learning systems in several systematic generalization benchmarks. Our results show how a standard neural network architecture, optimized for its compositional skills, can mimic human systematic generalization in a head-to-head comparison. Best, Byung-Doh ================= Byung-Doh Oh (he/him/his) Ph.D. Candidate Department of Linguistics The Ohio State University -------------- next part -------------- An HTML attachment was scrubbed... URL: From oh.531 at buckeyemail.osu.edu Sat Nov 11 10:58:19 2023 From: oh.531 at buckeyemail.osu.edu (Oh, Byung-Doh) Date: Sat, 11 Nov 2023 15:58:19 +0000 Subject: [CaCL] CaCL 11/16: Human-like systematic generalization through a meta-learning neural network In-Reply-To: References: Message-ID: Hi everyone, Just a reminder of the paper we're discussing this upcoming week: ================= Byung-Doh Oh (he/him/his) Ph.D. Candidate Department of Linguistics The Ohio State University ________________________________ From: Oh, Byung-Doh Sent: Thursday, November 2, 2023 2:19 PM To: cacl at lists.osu.edu Subject: CaCL 11/16: Human-like systematic generalization through a meta-learning neural network Hello everyone, Next week (11/9), CaCL will not meet. On the following week (11/16), I will lead discussion of the following paper, which coincidentally seems to have a lot of connections to the paper we discussed today: Human-like systematic generalization through a meta-learning neural network https://www.nature.com/articles/s41586-023-06668-3 The power of human language and thought arises from systematic compositionality?the algebraic ability to understand and produce novel combinations from known components. Fodor and Pylyshyn famously argued that artificial neural networks lack this capacity and are therefore not viable models of the mind. Neural networks have advanced considerably in the years since, yet the systematicity challenge persists. Here we successfully address Fodor and Pylyshyn?s challenge by providing evidence that neural networks can achieve human-like systematicity when optimized for their compositional skills. To do so, we introduce the meta-learning for compositionality (MLC) approach for guiding training through a dynamic stream of compositional tasks. To compare humans and machines, we conducted human behavioural experiments using an instruction learning paradigm. After considering seven different models, we found that, in contrast to perfectly systematic but rigid probabilistic symbolic models, and perfectly flexible but unsystematic neural networks, only MLC achieves both the systematicity and flexibility needed for human-like generalization. MLC also advances the compositional skills of machine learning systems in several systematic generalization benchmarks. Our results show how a standard neural network architecture, optimized for its compositional skills, can mimic human systematic generalization in a head-to-head comparison. Best, Byung-Doh ================= Byung-Doh Oh (he/him/his) Ph.D. Candidate Department of Linguistics The Ohio State University -------------- next part -------------- An HTML attachment was scrubbed... URL: From lin.4434 at buckeyemail.osu.edu Thu Nov 16 20:06:17 2023 From: lin.4434 at buckeyemail.osu.edu (Lin, Yi Chien) Date: Fri, 17 Nov 2023 01:06:17 +0000 Subject: [CaCL] Reading for 11/30: Emergent analogical reasoning in large language models Message-ID: Hi all, Next week (11/23), CaCL will not meet. On 11/30, we will discuss the paper ?Emergent analogical reasoning in large language models? by Webb et al. (2023). Link to paper: https://www.nature.com/articles/s41562-023-01659-w Abstract: The recent advent of large language models has reinvigorated debate over whether human cognitive capacities might emerge in such generic models given sufficient training data. Of particular interest is the ability of these models to reason about novel problems zero-shot, without any direct training. In human cognition, this capacity is closely tied to an ability to reason by analogy. Here we performed a direct comparison between human reasoners and a large language model (the text-davinci-003 variant of Generative Pre-trained Transformer (GPT)-3) on a range of analogical tasks, including a non-visual matrix reasoning task based on the rule structure of Raven?s Standard Progressive Matrices. We found that GPT-3 displayed a surprisingly strong capacity for abstract pattern induction, matching or even surpassing human capabilities in most settings; preliminary tests of GPT-4 indicated even better performance. Our results indicate that large language models such as GPT-3 have acquired an emergent ability to find zero-shot solutions to a broad range of analogy problems. Best, Yi-Chien -------------- next part -------------- An HTML attachment was scrubbed... URL: