Electric utilities, system operators, and balancing authorities are rapidly deploying new, high-speed data collection devices throughout their systems to gain far better operational information and situational awareness than what was possible with conventional SCADA systems. These devices, especially synchrophasors (or phasor measuring units, PMUs) generate far more data about the electric system than was ever generated in the past, and often the sheer quantity of data to analyze can outstrip a utility's ability to effectively process the information, much less act on it to ensure stable operations. 

PNNL's data curation software dramatically improves the processing of large-scale datasets associated with an electric power grid, primarily that from PMUs. Using filters, the unique statistical software framework can remove erroneous data and process the remaining data to detect events of interest such as power spikes, usage trends, and real-time consumption. Data associated with identified events may be stored in a repository for additional analysis, including answering queries from users. The flexible, scalable software, which has been extensively tested on real electric grid data, enables a more comprehensive, iterative analysis of large-scale datasets. The software can support real-time power pricing.

The framework consists of a statistical analysis package, running in a robust parallel environment, which is used to define rules that identify subsets of data, for example bad data or data indicating events of interest. These rules can be combined in arbitrary ways, for example, multiple rules may be required to remove all erroneous data from the original dataset. When the software identifies events of interest, it classifies them within known event types and stores the collection of event metadata and underlying data references in a relational database. These higher-level metadata descriptions of the events can then be used to quickly respond to queries from users or applications, or this information can be displayed in a visual format.