EGG: A tool to simulate language emergence in deep networks.
Deep artificial neural networks are increasingly pervasive in our daily lives, helping us in everyday tasks such as tagging pictures or translating from one language to the other. However, these computational systems are not endowed with the ability to communicate with each other and with us, which makes them rather inflexible and opaque tools. Marco Baroni and his colleagues are interested in the question: what happens if we let a community of artificial deep network "invent" their own language in order to solve a task together. In order to encourage research in this interdisciplinary area, involving artificial intelligence, linguistics and cognitive science, in 2020 the team open-sourced the EGG toolkit. Using the toolkit (currently starred more than 100 times on GitHub, and presented at the prestigious EMNLP conference) Baroni and colleagues were able to highlight several interesting properties of the emergent system developed by deep networks to communicate. For example, the networks are not subject to the same energy saving constraints that shape the communication systems of humans and animals. Consequently, they might associate very long forms, such as "fjksjkgjrgjkrgksfkeeeeeeff", to very frequent words (e.g., those meaning "the" or "it"). Conversely, neural networks can invent very clever ways to communicate about the world (e.g., by referring to the relative intensity of different pixels) that lead to languages that are completely obscure for us, but allow extremely efficient information transmission. The research goal set for 2020 is to find a common ground between the language spoken by deep networks and the one spoken by people!