Inverse Reinforcement Learning for Early Nuclear Proliferation Detection
PIs: Dennis Thomas, Ben Wilson
While data-driven methods are increasingly being developed for early detection of nuclear proliferation threats, analysts are limited from integrating data-driven recommendations within their analytic workflows due to partial observability of early indicators of nuclear proliferation activities and a lack of domain-guided learning and algorithmic explainability.
This project will address these limitations in two ways: (1) by developing an inverse reinforcement learning algorithm and computational pipeline to infer competing hypotheses (goals) from partially observed sequences of early proliferation activity indicators found in open-source text data, and (2) by applying structural causal methods to explain the reinforcement learning outcomes.
Our approach includes adopting a data-driven decomposition approach that allows for modeling latent goal variables at the state level and further studies at the researcher (agent) level. Our goals will be defined based on subject matter expert inputs, and we will mine open-source text data from technical publications and patents. To begin, data and insights from the team’s recent prior work for NA-22 on dynamic network analysis with nuclear science research collaboration and citation networks will serve as inputs for defining state-level goals, researcher-level actions, and/or research evolution over time. The algorithm results—expressed as probabilistic estimates of a state pursuing a goal over time—will be validated against subject matter expert feedback with uncertainty bounds.
Finally, we are developing an algorithm and computational pipeline for inferring competing hypotheses associated with early detection of nuclear proliferation from open-source text data using inverse reinforcement learning. Our analysis of competing hypotheses for early nuclear proliferation detection will be supplemented by explanations of algorithm results via causal methods for verification and further exploration.