Data Lakehouse
PermitAI turns environmental review and permitting records into structured data that people and software can use. Data lakehousing solutions developed by Pacific Northwest National Laboratory bring together information from separate repositories to help agencies, researchers, and developers discover relevant past work and compile information that supports new environmental review.
Our Approach
Decades of permitting information is recorded in documents, maps, and agency databases. Differences in storage formats and metadata make that information difficult to discover, compare, and reuse.
PermitAI combines original records with extracted content and enriched metadata in a shared data architecture. Our approach includes:
- Collecting records: Gather documents and available metadata through agency partnerships, data transfers, and public repositories.
- Extracting content: Separate documents into usable text, images, and geographic information while retaining their relationship to the source.
- Organizing information: Connect projects, review processes, documents, and files using a common metadata framework informed by Council on Environmental Quality standards.
- Improving quality: Use expert annotations and feedback to evaluate automated extraction and refine the processing pipeline.
Processing PermitAI’s large and varied collection of documents, images, and metadata requires close coordination between data engineering and data science. Data engineering manages ingestion, storage, and processing at scale, while data science extracts, enriches, and evaluates the information. Orchestrating these workflows turns incoming records into consistent, usable data for search, analysis, and AI applications.
This foundation supports document discovery, analysis of past reviews, and AI applications, such as SearchNEPA, that work with environmental information. The architecture is designed to accommodate additional permitting processes as PermitAI expands.
NEPATEC
The National Environmental Policy Act Text Corpus (NEPATEC) brings historical federal environmental reviews into a structured, machine-readable format. It includes categorical exclusions, environmental assessments, environmental impact statements, and associated records, enriched with metadata and extracted content.
The latest release, NEPATEC3.0, includes 81,347 projects, 188,330 files, and 42 metadata attributes, alongside extracted text, map images, and available GeoTIFFs. Release statistics are preliminary.PermitTEC
The Permitting Text Corpus complements environmental review data with structured information about federal litigation involving NEPA and related environmental statutes. Records describe courts, parties, ruling dates, and outcomes, with field-level provenance documenting extraction and review.
The latest release, PermitTEC v0.1, includes 761 cases, with 223 linked to projects in NEPATEC v2.0. These links help researchers examine environmental reviews alongside the legal challenges associated with them.Both releases use CC0 1.0. Accessing dataset files requires a Hugging Face account and acceptance of the repository’s access conditions.