STELLAR System Will Enable Autonomous Systems to Learn for Life

The STELLAR machine-learning architecture will emulate interactions of the hippocampus to consolidate lifelong learning. © 2018 HRL Laboratories.

HRL Laboratories Joins DARPA’s Lifelong Learning Machines (L2M) Program to Develop Breakthrough Machine-Learning Architecture that Remembers Mistakes, Improves, and Evolves Over Its Lifetime

HRL Laboratories, LLC, will help the Defense Advanced Research Project Agency (DARPA) develop what could be a breakthrough in machine-learning architecture for autonomous systems. With funding from the Lifelong Learning Machines (L2M) program, which is led by Dr. Hava Siegelmann from DARPA, the proposed system will continually improve performance and update its knowledge based on experience, without human supervision.

Inspired by the neuromodulatory systems in the adult human brain, we’ll incorporate mechanisms of structural and functional plasticity that are capable of continual learning

If successful, HRL’s Super Turing Evolving Lifelong Learning ARchitecture (STELLAR) project will achieve significant improvement over current technology, enabling autonomous systems to rapidly adapt to unforeseen situations, remember and learn from each experience, and consolidate new tasks with previous ones. Current machine learning systems forget old tasks when learning new ones and are not capable of learning online and responding to new situations not presented during offline training.

For a new skill to be consolidated in a way that does not interfere with previous learning requires the researchers to mimic how the human brain consolidates old and new learning. Tasks humans learn during the day are replayed in their brains at night. During sleep, learning is consolidated between the areas of the brain called the hippocampus—important in rapid encoding of waking experience—and the neocortex—involved in long-term memory storage, semantic memory extraction, and higher-order brain functions such as sensory perception, decision-making, action selection, cognition, language, and others.

“We intend to mimic the learning consolidation process of the brain using agents augmented with a memory system that is inspired by the interactions between the neocortex and hippocampus,” said Dr. Praveen Pilly, HRL’s Principal Investigator on the STELLAR project. “We will have a machine learning algorithm that is active while awake, such as when driving an autonomous car, optimizing its ability to predict the next few time steps. Later, when the car is parked and the algorithm is essentially sleeping, it will be selectively replaying the recent experiences it has gained and consolidating them with its existing knowledge in a non-destructive manner. This will give the car a heretofore unseen ability to respond to and eventually predict some unplanned events while driving. The system will be able to respond safely to surprises such as a tire blow-out or unforeseen weather conditions without shutting down.”

The HRL STELLAR team L to R: Charles Martin, Praveen Pilly, Soheil Kolouri, Aashish Patel, Nicholas Ketz, Mohammad Rostami. © 2018 HRL Laboratories.

“Current machine learning algorithms, including deep neural networks, are uniformly plastic and do not include mechanisms for preserving previously learned knowledge. Inspired by the neuromodulatory systems in the adult human brain, we’ll incorporate mechanisms of structural and functional plasticity that are capable of continual learning,” said Dr. Soheil Kolouri, STELLAR’s co-Principal Investigator.

“To achieve L2M, we’ll leverage a technique called neuro-evolution, creating an evolutionary methodology in which multiple learning architectures will compete for survival through generations. An element of the system called a fitness score will measure how well each architecture learns tasks in a curriculum, as well as how they adapt to surprise situations safely. The architectures that are best at these tasks will be programmed to move on to the next generation, spawning new general-purpose flexible architectures that can better consolidate new with old learning,” said Dr. Nicholas Ketz, STELLAR’s key team member.

STELLAR will go beyond current Turing systems by allowing adaptation of its program in response to new and unforeseen situations. “The combination of selective plasticity, offline memory optimization, neurogenesis, and neuromodulation allow the creation and selection of heterogeneous plastic neural networks to accelerate online adaptation leading to Super-Turing capability,” said Dr. Pilly.

The HRL Team also comprises collaborators at Stanford University (Prof. James McClelland), University of California Irvine (Profs. Jeffrey Krichmar and Emre Neftci), University of Texas Austin (Prof. Risto Miikkulainen), Loughborough University (Prof. Andrea Soltoggio), IT University of Copenhagen (Prof. Sebastian Risi), and The French Institute for Research in Computer Science and Automation (INRIA, Prof. Jean-Baptiste Mouret) with world-leading expertise in neuroevolution, brain-inspired learning and memory models, neuromodulation, and autonomous systems.

HRL Laboratories, LLC, Malibu, California (hrl.com) is a corporate research-and-development laboratory owned by The Boeing Company and General Motors specializing in research into sensors and materials, information and systems sciences, applied electromagnetics, and microelectronics. HRL provides custom research and development and performs additional R&D contract services for its LLC member companies, the U.S. government, and other commercial companies.

Media Inquiries: media[at]hrl.com, (310) 317-5000