PNNL

Pacific Northwest National Laboratory (PNNL) will be at the SPIE Defense + Commercial Sensing:  Algorithms, Technologies, and Applications for Multispectral and Hyperspectral Imaging XXIX Conference and CBRNE Sensing XXIV Conference May 2–4, 2023, in Orlando, Florida. 

SPIE is the international society for optics and photonics, with the mission to “partner with researchers, educators, and industry to advance light-based research and technologies for the betterment of the human condition” (spie.org). SPIE has contributed over $22 million to the international optics community in the past five years. This year’s Defense + Commercial Sensing conference will feature cutting-edge research in sensors, infrared, spectral imaging, radar, and more. PNNL staff members will be involved in six presentations highlighting research that enhances capabilities for defense and security applications.

PNNL Presentations and Talks  

Quantifying the robustness of deep multispectral segmentation models against adversarial examples and data poisoning

PNNL Authors: Elise Bishoff – Eleanor Byler – Charles Godfrey

Abstract: While including additional spectral bands beyond the traditional RGB channels can improve performance in overhead image segmentation tasks, it is still unclear how this additional data impacts adversarial robustness. We seek to characterize the performance and robustness of a multispectral image segmentation model subjected to adversarial attacks, focusing on two novel attack implementations designed to exploit multi-band information: data poisoning and perturbations (realistic adversarial examples) that coherently and self-consistently perturb the input data. We find modest improvements in model performance and robustness over an RGB baseline, highlighting the potential of multispectral data for creating more reliable models.

Architecture impacts robustness and interpretability of multispectral deep learning models

PNNL Authors: Charles Godfrey – Elise Bishoff – Myles McKay – Eleanor Byler

Abstract: Including information from additional spectral bands can improve deep learning model performance for many vision-oriented tasks. There are many ways to incorporate additional spectral bands into a model, but the optimal fusion can vary between applications. We characterize the performance of multispectral deep learning models with different fusion approaches and quantify their relative reliance on different input bands, showing that even when different fusion architectures achieve near-identical performance, they leverage information from the various spectral bands to varying degrees.

Hyperspectral data augmentation for boosting downstream analysis performance

PNNL Authors: Noriaki Kono – Madison Blumer – Timothy Doster – Tegan Emerson

Abstract: Hyperspectral imaging has seen a recent uptick in the use of machine learning tools for many tasks. Data-driven approaches often demonstrate improved performance on well curated datasets but fail to generalize or perform in real-world environments. Data augmentation is a common technique used to help bolster the performance of machine learning models by teaching them the types of variation to ignore, but the suite of standard augmentation tools used in image processing do not necessarily transfer to spectral domains. The presented augmentation methods capture the types of meaningful variation seen in real-world settings and preserve underlying domain knowledge.

Accounting for scattering via Hadamard-based hyperspectral reference set augmentation

PNNL Authors: Michael Rawson – Timothy Doster – Tegan Emerson

Abstract: Hyperspectral measurements give exquisite details about a scene, but analysis can be expensive due to high spatial/spectral resolutions; thus, sparsity is an important enabler of spectral compression and analytics. Environmental and atmospheric effects, including scattering, can produce nonlinear effects posing challenges for existing compression methods. In this work we leverage Hadamard products of spectra to augment reference sets in the presence of scattering-based nonlinearities. We present a novel framework for constructing sparse representations by combining subspace projection with non-negative least squares and benchmark against conventional matching pursuit and neural-inspired algorithms in terms of spectral reconstruction error and abundance coding error.

A stochastic differential equation formulation for modeling atmospheric absorption and scattering in hyperspectral images

PNNL Authors: James Koch – Brenda M. Forland – Timothy Doster – Tegan Emerson

Abstract: The physical interactions between the atmosphere and light are typically dominated by the absorbance and scattering. In this work, we recast these physics as learnable deterministic drift and additive noise of a Neural Stochastic Differential Equation. Upon successful training of these physics surrogate models, forward evaluation of the model equates to a noising process, whereas the reverse evaluation is a de-noising process. We show the utility of the trained models through canonical atmospheric correction tasks.

Comparison of methods to derive infrared optical constants for organic materials for optical modeling

PNNL Authors: Kelly A. Peterson – Tanya L. Myers – Bruce E. Bernacki – Charmayne E. Lonergan – Timothy J. Johnson

Abstract: Reflectance spectroscopy, especially at infrared wavelengths, is often used for contact, standoff, and remote sensing of solid materials. The reflectance spectra of solids, however, are complex, relying on many factors, even for the same material. Such phenomena can be modeled if the optical constants as a function of wavelength or wavenumber, n(ν) and k(ν), are known. Methods to measure the optical constants of solids, however, are challenging, particularly for powdered materials. For powdered materials, a pressed pellet of the neat material is often used when a crystalline specimen is unavailable. Three techniques, including ellipsometry, single-angle reflectance and KBr transmission spectroscopy, are applied to several organic materials for comparison, and the effects on the modeled spectra are shown.

Using synthetic infrared spectra derived from n/k optical constants for remote chemical detection

PNNL Authors: Charmayne E. Lonergan – Bruce E. Bernacki - Oliva M. Primera-Pedrozo - Jeremy D. Erickson - Sarah D. Burton - Timothy J. Johnson - Tanya L. Myers

Abstract: In this work optical constants n and k, (the real and imaginary part of the refractive index respectively) are used to model the infrared (IR) spectra for chemicals (e.g. acetaminophen, methyl phosphonic acid [MPA], and tributyl phosphate) on various substrates (e.g. glass and aluminum). The resulting spectra are compared to those measured via hyperspectral imaging (HSI) data acquired in the field. Modeled IR spectra were nicely captured in the corresponding n/k-derived spectra. This is promising and suggests the optical constants can be used to model the reflectance spectra and capture changes in material form.

Preparation and characterization methods of thin layer samples for remote detection

PNNL Authors: Oliva M. Primera-Pedrozo - Jeremy D. Erickson - Charmayne E. Lonergan - Sarah D. Burton - Bruce E. Bernacki - Tanya L. Myers - Timothy J. Johnson

Abstract: Development of methods to deposit thin layers (e.g. ~5 to 100 µm) of chemicals is important for both system design and field studies. In this work, solid and liquid analytes were deposited on painted and bare substrates including aluminum, glass, plastic, and concrete using an ExactaCoat ultrasonic spray coater. Laboratory hemispherical reflectance data in the infrared were collected for samples with deposition conditions to characterize both the composition and layer thickness. Preliminary results demonstrate that to prepare homogenous layers on surfaces, parameters such as substrate type, analyte solubility, vapor pressure, paint color, surface porosity, and surface roughness are all important.

Spectroscopic signatures of oxyanions and halides relevant to the stewardship of radioactive nuclear waste

PNNL Authors: Timothy J. Johnson

Abstract: Evaluating the speciation of oxyanions and halides upon perturbing solution conditions and/or exposure to irradiation is necessary for stewardship of radioactive nuclear waste, such as the waste stored at the Hanford Site in Washington State. Example techniques to populate speciation models include the use of vibrational spectroscopies, such as Raman spectroscopy and Fourier Transform Infra-red (FTIR) spectroscopy, which can be unified with orthogonal experimental techniques, such as Nuclear Magnetic Resonance (NMR) spectroscopy and Ultraviolet-Visible (UV-Vis) spectroscopy, to provide a molecular-level description of the speciation of oxyanions and halides.