Back to highlights 2023

Can neural networks learn fast, like children do?

Baroni, Marco (UPF)

Social & Behavioural Sciences

In the Nature article "Human-like systematic generalization through a meta-learning neural network", Brenden Lake and I introduced a method that allows modern AI systems for language (such as ChatGPT) to learn "compositionally", just like children (and humans in general) do. For example, if the network already knows the meaning of "twice", when it is introduced the new verb "to tiktok", the network can immediately understand the meaning of the expression "to tiktok twice". This result had a lot of resonance, generating a wide debate on the nature of learning in humans and machines, and what are the positive and possibly undesirable consequences of similar advancements in AI. Nature published a commentary titled "AI ‘breakthrough’: neural net has human-like ability to generalize language" and Scientific American an article about our work called "New training method helps AI generalize like people do". Our study had also great resonance in Spain and Catalonia, including a service during the telediario of RTVE, a live interview for the COPE radio and coverage by all the major newspapers (La Vanguardia, El País, Ara, ...).

During training of the novel meta-learning neural network, Episode A presents a set of study examples and a query instruction. The study examples demonstrate how to “jump twice”, “skip”, etc. The query instruction involves compositional usage of a word (“skip”) that is only presented in isolation in the study examples, and no intended output is provided. The network produces a query output that is compared with human behaviour. Episode B introduces the next word (“tiptoe”) and the network is asked to use it compositionally (“tiptoe backward around a cone”), and so on for many more training episodes, until the network learns to autonomously process new instructions by composing the meaning of their words.


REFERENCE

- Lake B & Baroni M 2023, 'Human-like systematic generalization through a meta-learning neural network', Nature, 623, 115-121.