Researchers present latest on deep learning models and invariance in neural networks
Image by Jeff London | Pacific Northwest National Laboratory
For more than a decade, Pacific Northwest National Laboratory (PNNL) researchers have been applying artificial intelligence (AI) to solve problems in areas like national security and Earth systems.
As leaders in AI and machine learning, PNNL experts are sharing their latest findings at the 36th annual Neural Information Processing Systems (NeurIPS) Conference, Nov. 28–Dec. 9, 2022.
NeurIPS is considered one of the top conferences on AI, machine learning, and computational neuroscience. Two key conference papers, authored by PNNL researchers, will be highlighted at the international event.
Charles Godfrey, Davis Brown, Tegan Emerson, and Henry Kvinge are presenting “On the Symmetries of Deep Learning Models and their Internal Representations.”
The paper discusses how symmetries intrinsic to deep learning models affect their function. The authors identified and calculated a set of fundamental symmetry groups of a neural network architecture, called intertwiners, and show how they result in symmetries of the model’s internal representation of data. These findings shed light on the question of whether different deep learning models learn the same features from a given dataset and inform the design of methods to make neural networks more interpretable to humans.
This project was funded by PNNL’s Mathematics for Artificial Reasoning in Science (MARS) Initiative as part of the Laboratory Directed Research and Development Program.
The second paper, “In What Ways Are Deep Neural Networks Invariant and How Should We Measure This?,” is authored by Kvinge, Emerson, Grayson Jorgenson, Scott Vasquez, and Timothy Doster.
This paper explores invariance and equivariance of deep learning models. The authors evaluated the two most popular methods used to build invariance into networks: data augmentation and equivariant layers.
A unique element of NeurIPS is its dedicated competition track. Each year, the competitions encourage participants to collaborate while using their skills to have an impact on important, real-world challenges.
The CityLearn Challenge 2022 is one of the 25 competitions offered this year. Working in teams, competitors address energy challenges by leveraging CityLearn, an OpenAI Gym environment, to develop AI agents for demand response. Participants must coordinate the energy consumed by each of the buildings within a simulated microgrid.
PNNL’s Ján Drgoňa is one of the organizers, collaborating with industry and research partners: Purdue University, the Intelligent Environments Laboratory, the Electric Power Research Institute, Lawrence Berkeley National Laboratory, Manifold, AICrowd, and the University of Colorado Boulder.
At NeurIPS, affinity groups promote the ideas and voices of marginalized communities while raising awareness of issues that affect their members. Affinity group workshops are geared to promote diversity and inclusion.
Supporting these goals, PNNL researchers are leading discussions on two topics during the affinity workshops.
Ellyn Ayton, data scientist, and Svitlana Volkova, chief scientist, will discuss “Graph Transformer Networks for Nuclear Proliferation Detection in Urban Environments.”
Anika Halappanavar, PNNL intern, and Maria Glenski, senior data scientist, will be presenting “Characteristics of White Helmets Disinformation vs COVID-19 Misinformation.”
Drgoňa is also co-organizing a workshop, titled “Tackling Climate Change with Machine Learning.” The organizers will lead a discussion on how machine learning can be a valuable tool in reducing greenhouse gas emissions while helping society adapt to the impacts of climate change.
Whether during poster sessions or workshops, learn more about opportunities to engage with PNNL researchers during the 2022 NeurIPS Conference. This year, PNNL is an official sponsor of the conference.
Published: November 28, 2022