HRL Researchers Receive IARPA Award for Hybrid Forecasting Competition

HRL Laboratories, LLC researchers in the Center for Computational Network Intelligence, Information and Systems Sciences Laboratory received an award from the Intelligence Advanced Research Projects Activity (IARPA) to enter the Hybrid Forecasting Competition. This contest will test designs for hybrid forecasting systems that integrate human and machine forecasting components for the most accurate, flexible, and scalable performance.

HRL’s proposed system, Machine-aided Analytic Triage with Intelligent Crowd Sourcing (MATRICS), directly engages human participants and minimizes crowd attrition with incentives to maximize participant effort and maintain engagement.

“The idea is that the human forecasters will be good and bad at certain things, as will the machine forecasters. So we hope to capture what is best about each of these elements, help them mitigate each other, and throw away the bad things,” said David Huber, HRL’s principal investigator.

Previous research by others found that approximately the top 2% of crowds are exceptionally good at forecasting events. It is possible that these top forecasters are motivated to get better as they participate if they have fun, become very interested and spend time learning about the subject.

“Not much is known about what factors make a person a good forecaster and one of the research questions of this program is to determine if there is something more quantifiable about them,” Huber said. “With more analytical knowledge about what makes consistently exceptional forecasters ‘tick’, we hope to determine ways to leverage the remaining 98% of forecasters or discover knowledge that will help them improve. We also want to see if we can apply what we learn from human forecasters to improve the capabilities of machine forecasting systems, creating hybrid approaches to forecasting.”

“Typically human-machine interaction is one-on-one, one human interacting with a machine. Here we are talking about a group of humans interacting with each other toward a common goal,” said Tsai-Ching Lu, HRL project manager for MATRICS. “The interaction is also between the people and the machine, and within the machine there are multiple algorithms or systems that act just like people in a crowd.”

The MATRICS hypothesis is that individual human forecasters and machine algorithms can be interactive peers in a hybrid system. Behaviors of highly accurate human and machine forecasters may be used to improve forecaster behavior through minor changes in the survey process.

“IARPA will be recruiting thousands of human forecasters for the HFC program. One big challenge of the program is incentivizing participants with non-monetary motivations, making it fun and exciting for them, perhaps with some game elements, so our participants want to come back, keep doing it, and remain involved.” Huber said.

HRL’s institutional partners on the MATRICS project are Harvard University, Northeastern University, Massachusetts Institute of Technology, University of Houston, and Microsoft Research. Members of the public who are interested in participating as forecasters in the HFC research effort can sign up at https://www.hybridforecasting.com.

HRL Laboratories, LLC, Malibu, California (hrl.com) is a corporate research-and-development laboratory owned by The Boeing Company and General Motors specializing in research into sensors and materials, information and systems sciences, applied electromagnetics, and microelectronics. HRL provides custom research and development and performs additional R&D contract services for its LLC member companies, the U.S. government, and other commercial companies.

Media Inquiries: media[at]hrl.com, (310) 317-5000