AI System Will Use “Capture the Flag” to Train Space Operators for Contested Space

AIOCS is a project to create an AI program to help train operators who may be required to control and monitor craft in future contested outer space operations. © 2022 Getty.

HRL Laboratories Will Create AI Proxy Adversaries to Help Train Satellite Technicians by Incorporating Correct Physics, Timescales, and Action Spaces in Games

HRL Laboratories, LLC, has launched Artificial Intelligence Opponents for Contested Space (AIOCS), a project to create an AI program to help train operators who may be required to control and monitor craft in future contested outer space operations. Funded by the Air Force Research Laboratory (AFRL) Space Vehicles Directorate, which is part of the newly established United States Space Force (USSF), the project will use spiral development, integration and evaluation of reinforcement learning, and game theory for strategic management and execution in a space environment contested by multiple agents.

“The game we are using for the project is the familiar ‘capture the flag’ game, where one team tries to cross the field, capture the opposing team’s flag, and return with it to their own side without getting tagged out.  We have set up the same game, except that it is played in orbits around the earth using proper astrodynamics for the motion.  The ‘flags’ are kept on satellites that cannot maneuver, and must stay in their original orbit for the entire game,” said Deepak Khosla, HRL’s Principal Investigator. “The players trying to capture the flag on both sides are satellites that have thrusters, and can maneuver to change their orbits, allowing them to move around and interact with the flag stations and the other satellites.  Essentially, the goal of the game is maneuver one of your satellites to the opponents flag station satellite and get within a certain distance of it, and then maneuver back to your own flag station satellite, while avoiding getting ‘tagged’ out by an opponent’s satellite.  The game also includes realistic constraints such as tracking fuel consumption for the satellites as they maneuver – if one runs out of fuel, it can no longer maneuver to try to win the game.”

AFRL, in collaboration with HRL, is bringing novel concepts and designs from other domains to space

Dr. Richard Scott Erwin, the AFRL project manager, describes the relevance of the game to the newly stood up United States Space Force.  “One of the major drivers that was behind the stand-up of the Space Force as a separate service was the recognition that our adversaries understand the dependence of our military on the services, communications, and data that satellites provide, and that the Space Force must be able to provide these services in the face of threats against the satellites that provide them.  At the same time, AI and machine learning have made huge strides in the last decade in the ability to play complex games, such as Go, StarCraft 2, and Poker, at or above the ability of the best human players.  ‘Space Capture the Flag’ is a completely made-up game, but it captures the relevant physics that control satellite motion, provides an adversarial setting that allows many-against-many scenarios, and incorporates important constraints such as the limited amount of fuel that spacecraft have.  It will allow us to evaluate how humans and AI players approach these kinds of situations, and to understand where human operators have advantages, where AI agents may offer advantages, and where blending both is the optimal solution.

As the AIOCS program advances, HRL will collaborate with government experts to refine algorithms and develop new simulation capabilities. The AI will be analyzed and evaluated as it faces increasingly capable adversaries and progressively more sophisticated and realistic scenarios.

“The AI can be used as a proxy adversary to help operators hone their skills and understand the implications of their decisions in contested space environments. Ultimately, as confidence in the AI grows and it is shown to be able to deal with more complex, realistic situations, it could be used as a decision-aid tool that is able to rapidly evaluate a very large number of possible blue team moves over a long time horizon, and suggests a top few courses of action with estimated outcomes and prediction confidences as well as relative benefits and disadvantages,” Khosla said. “The user can then vet this suggested list quickly and select the course of action that works best for them, reducing the required number of operators, the time to make a decision, and the need for humans to perform tedious calculations.”

“There are interesting aspects to this game that separate it from others we have looked at in the past,” Khosla continued.  “For example, some of the games can last weeks due to the vast distances that must be traversed by the satellites, but at the same time, the maneuvers to intercept the flag station or an opponent’s satellite can happen very quickly – in hours or even less.  It will be interesting to see how AI and human players deal with such a wide disparity in time constants as the effort progresses.  We also hope to incorporate important features such as uncertainty in the positions of satellites and information delays, which are features of real space operations, and which will be challenging for human and AI players to deal with.”

“Space is a congested and contested domain, and one that is critical for national security, civil exploration, and commercial development for the United States and the rest of the world. Developing the capabilities to autonomously manage satellite constellations within domain, safely and securely, even when in a contested area is a growing need for US Space Force,” said Mr. Benjamin Seibert, AFRL Space Control Mission Lead. “AFRL, in collaboration with HRL, is bringing novel concepts and designs from other domains to space. These kinds of cross-domain collaborative relationships are core to the AFRL mantra of ‘One Lab, Two Services’ under which AFRL supports both the Air Force and Space Force as a single laboratory serving the needs of the air, space, and cyber domains.”

HRL Laboratories, LLC, Malibu, California (hrl.com) is a corporate research-and-development laboratory owned by The Boeing Company and General Motors specializing in research into sensors and materials, information and systems sciences, applied electromagnetics, and microelectronics. HRL provides custom research and development and performs additional R&D contract services for its LLC member companies, the U.S. government, and other commercial companies.

Media Inquiries: media[at]hrl.com, (310) 317-5000