ACMD staff contributed to 30 years of atmospheric data collection
In support of the Department of Energy’s Atmospheric Radiation Measurement user facility, staff from Pacific Northwest National Laboratory updated ingests and worked on science data products for the 2019–2020 Multidisciplinary Drifting Observatory for the Study of Arctic Climate expedition.
(Photo by Matthew Shupe | University of Colorado Boulder)
Over the past 30-plus years, the Atmospheric Radiation Measurement (ARM) user facility has collected measurements from all seven continents and five oceans. In remote and urban regions, more than 460 ARM instruments gather data on clouds, precipitation, solar and infrared energy, aerosols, and other environmental elements.
All of that data produces a unique picture of how Earth systems are changing—and how they might have looked in preindustrial times. This is an attractive draw for the more than 1,000 users who download data on average each year from ARM, a Department of Energy (DOE) Office of Science user facility.
Every day, the Advanced Computing, Mathematics, and Data (ACMD) Division at Pacific Northwest National Laboratory (PNNL) help ARM deliver high-quality, continuous atmospheric data at no cost to scientists worldwide.
Most of the ACMD staff who support ARM are spread across three teams within the division’s Software and Data Systems Engineering Group, including the Data Services, Infrastructure and Systems Deployment, and Scientific Software Engineering teams.
Staff prepare ingest software for ARM to process data from instruments in the field. Before an ARM mobile deployment, software developers routinely update more than 60 ingests.
ACMD staff are also responsible for developing higher-order science data products. Staff employ algorithms to combine measurements from multiple instruments, creating more usable and meaningful data products.
These products, known as value-added products (VAPs), typically compute geophysical variables that instruments do not measure directly, such as liquid water path. Some VAPs are “best estimate” products or packaged for modelers. ACMD staff are responsible for 50 active VAPs, including one that uses artificial intelligence to produce a cloud mask from lidar data.
In addition, ACMD staff create and support tools and processes to aid data flow and discoverability.
To help streamline ingest and VAP development, Software Engineer Krista Gaustad led a group of developers, including Software Engineers Sherman Beus, Brian Ermold, and Timothy Shippert, in creating the ARM Data Integrator. This open-source framework makes it much easier to process raw instrument data and combine and transform diverse data sets into higher-order products.
Earth Scientist Jennifer Comstock is a team leader and also serves as ARM’s first associate director for research. In her ARM role, Comstock oversees research and science product development across multiple DOE national laboratories.
ARM’s organization chart also features a trio of ACMD software engineers in technical leadership positions. Beus oversees ARM’s data flow software and website development, Ermold leads ingest development, and Gaustad is responsible for the data product software and development workflow.
Early in ARM’s history, several present-day ACMD staff managed and supported ARM’s Data Management Facility at PNNL. This facility helped ARM standardize and centralize data processes that were previously handled by different sites around the world.
As technology and user needs have evolved, so have ARM’s approaches to collecting, processing, evaluating, storing, and distributing data. Thanks to the work of ACMD staff and colleagues from PNNL and other DOE national laboratories, ARM remains the world’s premier ground-based observations facility in its fourth decade.
Published: July 31, 2023