PNNL

Long before the questions of COVID’s origins, the United States Intelligence Advanced Research Projects Activity began developing and refining a suite of tools to improve detecting signs of biological engineering. The program, known as Finding Engineering-Linked Indicators, or FELIX, spanned across organizations, including Pacific Northwest National Laboratory (PNNL), who tested and evaluated these tools to verify they could find a range of engineered biology—even in the presence of other natural organisms.

PNNL Statistician Kelly Stratton led project management for PNNL’s contributions to FELIX beginning in 2018. Stratton led a PNNL team that included statisticians, biologist, and chemists who understood different organisms and what type of engineering would be important to detect, in addition to which organisms could be engineered and how.

“There are many altruistic reasons to genetically engineer an organism, such as developing medicines or benefiting crop production,” said Stratton, “but there’s also a potential for something unintended or something deliberately harmful to happen. So being able to track genetic engineering is important to help us more quickly understand a situation or create a new therapeutic.”

PNNL’s role from the beginning was to be the statistical center of the evaluation team’s efforts in generating and curating samples. Statistics were essential to guide the development of sample sets to get a granular enough view of how the technologies were performing. Knowing how the technologies performed could help developers and researchers determine whether the tools met the specificity necessary to be reliable in the field.

The evaluation team sent four rounds of 50 samples each to the developers so they could test their technology. Some developers worked from a physical sample while others worked computationally with genetic sequences. In either case, it was essential to send unique samples that would be difficult to determine because of their mix of wild strains and strains from a laboratory—engineered or not.

At the beginning of the program, the performers only determined if there was evidence of engineering in a simple sample. As the tools improved, the testing team created more complex samples to get a more nuanced view of the tool’s strengths and weaknesses. By the end of the program, the team also upped the sample’s difficultly level to determine whether the tools were detecting the presence of genetic engineering within the metagenome of a more realistic community.

“Our internal team was so incredible,” said Stratton. “Technically strong and also extremely supportive of each other, inquisitive in terms of understanding the moving parts of the biology and the implications from the statistical side.”

Broad Institute of the Massachusetts Institute of Technology and Harvard University, Ginkgo Bioworks Inc., Wyss Institute at Harvard, Noblis, Draper Laboratory, and Raytheon BBN Technologies directly developed the tools that could provide early alerts to the presence of engineered organisms. Argonne National Laboratory, Lawrence Berkeley National Laboratory, and the United States Department of Agriculture’s National Wildlife Research Center served as testing and evaluation partners with PNNL.