Lifelong Learning Machines

Enabling intelligent systems that continuously adapt to changing conditions and missions in the real world

Our Contribution

Engineers and scientists in APL’s Intelligent Systems Center are developing systems that continuously improve in the field while applying previously acquired proficiencies to perceive, decide, act, and team in new situations.

Related Publication

Vallabha, G. K., J. Markowitz, “” SPIE AI/ML for Multi-Domain Operations IV (2022).

Related Areas of Impact

Mission Area

Research and Exploratory Development

Novel Environments for Exercising Lifelong Learning Agents

Training and evaluating a lifelong learning agent requires the ability to generate a diverse range of challenges, in a variety of environments, with rich systematic variation.

MetaArcade is a highly parameterized environment for constructing custom 2D arcade games with controlled variation of perceptual features, actions, and rewards (, , ).
L2Explorer is a Unity-based first-person-view 3D exploration environment that can be configured on the fly to generate a range of tasks and task variants that can be structured into complex and evolving evaluation curricula (, , ).

Related Publications

Johnson, E. C., E. Q. Nguyen, B. Schreurs, C. S. Ewulum, C. Ashcraft, N. M. Fendley, M. M. Baker, A. New, G. K. Vallabha, “L2Explorer: A Lifelong Reinforcement Learning Assessment Environment,” (2022).
Staley, E. W., C. Ashcraft, B. Stoler, J. Markowitz, G. Vallabha, C. Ratto, K. D. Katyal, “Meta Arcade: A Configurable Environment Suite for Meta-Learning,” Deep RL Workshop NeurIPS 2021.

Novel Strategies for Evaluating Lifelong Learning

Evaluating a lifelong learning agent requires characterizing the tasks encountered by the agent over its mission, identifying the capabilities of the agent (how well it can learn multiple tasks and exploit task relationships), and exercising the agent across different mission scenarios in a consistent yet flexible way.

RLBlocks is a framework of building blocks for lifelong learning agents ().
TELLA is a framework for Training and Evaluating Lifelong Learning Agents (, ).
CLAMP is an algorithm to identify the capabilities of a learning agent using its performance data (, ).
L2Metrics is a Python library for calculating lifelong learning metrics from performance data (, ).

Related Publications

Baker, M., et al., “A domain-agnostic approach for characterization of lifelong learning systems,” , 160, pp. 274–296 (2023).
Fendley, N., C. Costello, E. Nguyen, G. Perrotta, C. Lowman “Continual Reinforcement Learning with TELLA,” Conference on Lifelong Learning Agents (CoLLAs) 2022, (2022).
Rivera, C., C. Ashcraft, A. New, J. Schmidt, G. Vallabha “Estimating Latent Properties of Lifelong Learning Systems,” Conference on Lifelong Learning Agents (CoLLAs) 2022, (2022).

Novel Algorithms to Handle Perceptual and Task Variation

The appearance of a ship can vary significantly depending on weather, lighting, and orientation relative to the camera.

Artificial agents struggle when asked to identify objects under different visual conditions, act upon input observations that are superficially different (such as different lighting conditions in natural imagery), or adapt to significant changes to their given task.

Related Publications

Emmanuel, P. R., C. R. Ratto, N. G. Drenkow, J. J. Markowitz, J. J., “Semi-supervised domain transfer for robust maritime satellite image classification,” Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications V, Vol. 12538, pp. 296–309), SPIE (June 2023).
Markowitz, J., R. Sheffield, G. Mullins, “.” SPIE AI/ML for Multi-Domain Operations IV (2022).

��ֱ��