Can AI Mimic Human Compositional Thinking?
Researchers advanced the capabilities of neural networks to make compositional generalizations, similar to how humans grasp and expand on new concepts
This new technique, named Meta-learning for Compositionality (MLC), challenges decades-old skepticism about the capacity of artificial neural networks.
MLC involves training the network through episodic learning to enhance its generalization skills.
Remarkably, in various tasks, MLC matched or even surpassed human performance.
Key Facts:
- The MLC technique focuses on episodic training of neural networks, allowing them to better generalize new concepts compositionally.
- In tasks involving novel word combinations, MLC performed on par with or superior to human participants.
- Despite its advancements, popular models like ChatGPT and GPT-4 have challenges with this kind of compositional generalization, but MLC might be a solution to enhance their capabilities.
Source: NYU
Humans have the ability to learn a new concept and then immediately use it to understand related uses of that concept—once children know how to “skip,” they understand what it means to “skip twice around the room” or “skip with your hands up.”
But are machines capable of this type of thinking?
In the late 1980s, Jerry Fodor and Zenon Pylyshyn, philosophers and cognitive scientists, posited that artificial neural networks—the engines that drive artificial intelligence and machine learning— are not capable of making these connections, known as “compositional generalizations.”
However, in the decades since, scientists have been developing ways to instill this capacity in neural networks and related technologies, but with mixed success, thereby keeping alive this decades-old debate.
Researchers at New York University and Spain’s Pompeu Fabra University have now developed a technique—reported in the journal Nature—that advances the ability of these tools, such as ChatGPT, to make compositional generalizations.
This technique, Meta-learning for Compositionality (MLC), outperforms existing approaches and is on par with, and in some cases better than, human performance.
MLC centers on training neural networks—the engines driving ChatGPT and related technologies for speech recognition and natural language processing—to become better at compositional generalization through practice.
Developers of existing systems, including large language models, have hoped that compositional generalization will emerge from standard training methods, or have developed special-purpose architectures in order to achieve these abilities.
MLC, in contrast, shows how explicitly practicing these skills allow these systems to unlock new powers, the authors note.
“For 35 years, researchers in cognitive science, artificial intelligence, linguistics, and philosophy have been debating whether neural networks can achieve human-like systematic generalization,” says Brenden Lake, an assistant professor in NYU’s Center for Data Science and Department of Psychology and one of the authors of the paper.
“We have shown, for the first time, that a generic neural network can mimic or exceed human systematic generalization in a head-to-head comparison.”
In exploring the possibility of bolstering compositional learning in neural networks, the researchers created MLC, a novel learning procedure in which a neural network is continuously updated to improve its skills over a series of episodes.
In an episode, MLC receives a new word and is asked to use it compositionally—for instance, to take the word “jump” and then create new word combinations, such as “jump twice” or “jump around right twice.” MLC then receives a new episode that features a different word, and so on, each time improving the network’s compositional skills.
To test the effectiveness of MLC, Lake, co-director of NYU’s Minds, Brains, and Machines Initiative, and Marco Baroni, a researcher at the Catalan Institute for Research and Advanced Studies and professor at the Department of Translation and Language Sciences of Pompeu Fabra University, conducted a series of experiments with human participants that were identical to the tasks performed by MLC.
In addition, rather than learn the meaning of actual words—terms humans would already know—they also had to learn the meaning of nonsensical terms (e.g., “zup” and “dax”) as defined by the researchers and know how to apply them in different ways.
MLC performed as well as the human participants—and, in some cases, better than its human counterparts. MLC and people also outperformed ChatGPT and GPT-4, which despite its striking general abilities, showed difficulties with this learning task.
“Large language models such as ChatGPT still struggle with compositional generalization, though they have gotten better in recent years,” observes Baroni, a member of Pompeu Fabra University’s Computational Linguistics and Linguistic Theory research group.
“But we think that MLC can further improve the compositional skills of large language models.”
See more here neurosciencenews.com
Please Donate Below To Support Our Ongoing Work To Defend The Scientific Method
PRINCIPIA SCIENTIFIC INTERNATIONAL, legally registered in the UK as a company incorporated for charitable purposes. Head Office: 27 Old Gloucester Street, London WC1N 3AX.
Trackback from your site.
Tom
| #
LMAO. The only purpose of A/I is to exert more control over humans.
Reply
Wisenox
| #
They don’t mention what systems design and structures MLC is based on. Admittedly, my coding skills are hobbyist, but the descriptions remind me of hash trees, which were outlined in Dr Dobbs Journal at least 18 years ago.
If so, then MLC novel word composition can be seen in applications such as Hashcat.
If this is indeed the model proposed for machine simulated consciousness, then I disagree. The approach is too brute force, like conditionals overload.
Personally, I feel the best approach is to view human consciousness as an interface. Programmatically this is best implemented through abstraction and extension. Throw code reflection into the mix, and now you have a powerhouse of an interface.
Humans think with pathways. In my head, these become landscapes, and I am able to “skip” ahead to locations and start from there. Brute force hash tree parent-child node transversal, no matter how efficient, is sluggish, resource hungry, and predictable. You can remove the predictability to some degree, but it will be at the expense of the other two.
The machine has to learn to skip to locations in the landscape, and you need an interface for that. The “learning” and personification attributes can be realized with abstraction and reflection. We don’t continually add brain cells, nodes, when we learn. We reform pathways.
I am, however, just a hobbyist and don’t have any experience developing an AI. Just speculating.
Reply
Howdy
| #
“how explicitly practicing these skills allow these systems to unlock new powers, the authors note.”
“a neural network is continuously updated to improve its skills over a series of episodes.”
Machines do not have ‘skills’, they follow root instructions, while the program may adapt, it does not develop a skill.
So you see how detached from reality these people are, like talking about a machine program as if it’s a life-form? Intoxicated, as I previously said.
Just Frankenstein’s monster of the ‘modern’ age, but the madness has many more in it’s grip this time.
Reply
Alan
| #
AI will only think like humans when it ignores reality and accepts a fantasy world.
Reply
Wisenox
| #
That’s the post of the day. Made me laugh.
Reply