Human-Machine System to Enhance Software Analyst Performance
The Cognitive Aid for Vulnerability Analysis project will develop human-machine tools for reducing analyst mental workload and fatigue for improved performance. © Getty Images.
HRL Laboratories Human-Machine Tools Aim to Reduce Analyst Mental Workload and Fatigue for Improved Performance
HRL Laboratories, LLC, has begun development of a system that could potentially assist program analysts searching for vulnerabilities in compiled software. The Cognitive Aid for Vulnerability Analysis (CAVA) project design aims to use a multi-tiered approach to accelerate reverse-engineering and software comprehension by addressing a gap in human-machine systems that limits their recognition and adaptation to human thought processes—such as attention decay and fatigue—that reduce performance during detection of accidental or malicious vulnerabilities in compiled binaries. The unique knowledge base and demanding workload of program vulnerability analysts using current software tools indicates that future tools will likely induce high workload if they are not properly tuned to the operator’s cognitive state.
“Underlying this project is the concept of sensemaking,” said Principal Investigator Rajan Bhattacharyya. “This is the cognitive process analysts use to deal with difficult tasks by forming, testing, and revising hypotheses behind the causes of what they observe. This involves decision making at many levels: which hypotheses to test first, which information to pay attention to, and which information is trustworthy or deceptive. We will be looking into these thought processes at an unprecedented scope to try to create software tools and a graphical interface with the goal of reducing analysts’ mental workload and fatigue, and improving their performance.”
“Within the CAVA project, we have five challenges,” Bhattacharyya said. First, we will analyze operator behavior to understand the finer-grained cognitive operations that underlie our observations of their performance in reverse-engineering software for vulnerability discovery. Next, we will hope to build a a sensing system to measure neurocognitive metrics, such as attention and workload, in this real-world task. If that is successful, we hope to use the data and measurements to build a model that simulates an analyst’s cognitive processing and decision-making. At that point we will design and begin building a neuroscience-inspired graphical user interface to improve our measurements going forward. If each of these steps is achieved, the final step will be creation of a system that optimizes the neuroergonomics of our overall analysis using an algorithm that learns from our collected data and adapts the user interface to improve detection and discovery of software vulnerabilities and ease stress for analysts.”
During this 12-month seedling project, HRL will work with the Naval Information Warfare Center and Carnegie Mellon University to co-design the experiments, utilize subject-matter experts as experimental subjects, and perform the experiments on site to address the first four challenges, while assessing the feasibility of challenge 5 for future work.
This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Contract No. HR0011-20-C-0141. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of DARPA.
HRL Laboratories, LLC, Malibu, California (hrl.com) is a corporate research-and-development laboratory owned by The Boeing Company and General Motors specializing in research into sensors and materials, information and systems sciences, applied electromagnetics, and microelectronics. HRL provides custom research and development and performs additional R&D contract services for its LLC member companies, the U.S. government, and other commercial companies.
Media Inquiries: media[at]hrl.com, (310) 317-5000