Mega AI, an internal investment at PNNL, aims to develop next-generation AI capabilities unique to the Department of Energy’s national lab complex to address research gaps in large-scale multimodal representation learning, multitask inferences, and the need for increased generalizability, rapid adaptivity, and usability of AI technologies. In this newsletter, we highlight recent developments in the research community on next-generation AI technologies focusing on massive-scale model development, deployment and evaluation, data and code availability, model interactions, and new features and capabilities that are relevant to Mega AI’s goals and science and security applications.

June 2022

New Models and Capabilities 

• May 26 - Blog: Graphcore and Hugging Face Launch New Lineup of IPU-Ready Transformers. READ MORE
• May 24 - Paper: Large Language Models are Zero-Shot Reasoners. READ MORE
• May 6 - Article: Google AI sparks a revolution in Machine Learning. READ MORE
• May 3 - Blog: Democratizing access to large-scale language models with OPT-175B. READ MORE
• May 1 - Article: AI2 Open-Sources ‘LM-Debugger’: An Interactive Tool For Inspection And Intervention In Transformer-Based Language Models. READ MORE
• April 28 - Blog: Tackling multiple tasks with a single visual language model. READ MORE
• April 28 - Blog: Studying the brain to build AI that processes language as people do. READ MORE
• April 12 - Paper: A Review on Language Models as Knowledge Bases. READ MORE
• Website: Imagen – unprecedented photorealism x deep level of language understanding. READ MORE

New Reports and Studies

• May 25 - Article: AI The dark secret behind those cute AI-generated animal images. READ MORE
• May 25 - Article: Eye to eye in AI: Developing artificial intelligence for national security and defense. READ MORE
• May 19 - Paper: A Generalist Agent. READ MORE
• May 13 - Paper: Data Distributional Properties Drive Emergent In-Context Learning in Transformers. READ MORE
• May 13 - Article: Google’s Universal Pretraining Framework Unifies Language Learning Paradigms. READ MORE
• May 10 - Paper: Unifying Language Learning Paradigms. READ MORE
• March 7 - Article: Dual use of artificial-intelligence-powered drug discovery. READ MORE
• May 4 - Paper: Reframing Human-AI Collaboration for Generating Free-Text Explanations. READ MORE
• April 20 - Blog: Amazon release 51 – language dataset for language understanding. READ MORE
• April 13 - Blog: Measuring Goodhart’s Law. READ MORE
• Paper: Training Language Models with Natural Language Feedback. READ MORE

Events

Pacific Northwest National Laboratory’s Mega AI Innovator Series LEARN MORE

• April 20 - Pushing NLP to the Edge
  Alexander “Sasha” Rush, Cornell Tech
  View on YouTube
• April 13 - Creating Trustworthy AI/ML for Weather and Climate
  Amy McGovern, University of Oklahoma
  View on YouTube
• March 20 - Advances in ML for Cyber-Safety
  Srijan Kumar, Georgia Institute of Technology
  View on YouTube