Brain-Inspired "Machine Memory Intelligence" Could Address the Biggest Issues with LLMs, Gen AI
Researchers propose M²I as a means to fix "catastrophic forgetting" and "a deficiency in logical reasoning" in current AI efforts.
Researchers from Xi'an Jiaotong University, Shanghai Fourth People’s Hospital, Tongji University, Tsinghua University, and Iowa State University have been looking into solving some of the drawbacks of large language models (LLMs), like their forgetfulness and tendency to hallucination, by taking inspiration from how the human brain remembers things to create machine memory intelligence (M²I).
"Large models, exemplified by [OpenAI's] ChatGPT, have reached the pinnacle of contemporary artificial intelligence (AI)," the researchers claim. "However, they are plagued by three inherent drawbacks: excessive training data and computing power consumption, susceptibility to catastrophic forgetting, and a deficiency in logical reasoning capabilities within black-box models. To address these challenges, we draw insights from human memory mechanisms to introduce 'machine memory,' which we define as a storage structure formed by encoding external information into a machine-representable and computable format."
Using the idea of "machine memory," which draws inspirations from how the human brain both remembers and forgets things, the researchers propose a new framework dubbed machine memory intelligence, or M²I. " M²I aims to liberate machine intelligence from the confines of data-centric neural networks," the team says, "and fundamentally break through the limitations of existing large models, driving a qualitative leap from weak to strong AI."
The AI bubble shows little sign of bursting, yet core issues remain largely unaddressed. Increasing resources are going to building chips dedicated to training and inference for large language models (LLMs) and other generative AI technologies, which then draw ever-higher amounts of power while running before being replaced a year later by parts twice as fast. The quality of output from these models increases, but fundamental issues like "hallucination" — where the token-chain returned by the model forms the shape of an answer, as it should, but has no basis in reality — have yet to be solved.
M²I, the researchers say, can help. "Research in the direction of machine memory aims to address the path dependency on traditional artificial neural networks for achieving machine intelligence, thereby overcoming the issues of brute-force computation, catastrophic forgetting, and weak logical reasoning capabilities inherent in current LLMs and multimodal models," the team writes. "Furthermore, leveraging M2I can help validate theories and hypotheses in brain science, particularly those related to human brain memory, thus promoting the deep integration, cross-validation, and collaborative development of AI and brain science."
The full paper, which identifies a range of challenges between the concept and real-world deployment, has been published in the journal Engineering under open-access terms.
