Applying Key Performance Indicators

Introduction
Discussion
Solutions and Actions
Conclusions and Next Steps
Definitions
Sources of Information

Introduction

Developing and applying key performance indicators (KPIs) is a valuable way for building owners and their operations and maintenance (O&M) organizations to measure O&M performance and identify areas in need of improvement.

There is an ever-increasing number, variety, and complexity of KPIs related to O&M. These complexities can result in a challenging KPI selection and limited usage by operations, maintenance, and facility managers.

In general, a KPI can be defined as a numeric value used to measure and evaluate progress toward a defined goal or objective. KPIs are typically generated by combining one or more relevant metrics into a mathematical relationship to express the performance, condition, or efficiency of a part, system, facility, or organization.

With the multitude of KPIs available, it becomes important to spend the time up front before selection to determine the balance between tactical and strategic KPIs, operational and financial KPIs, and those that help capture real-time performance while anticipating the future (MIT 2018).

This Best Practice strives to present federal-sector-relevant KPIs, offer guidance on KPI selection and development, and provide an understanding of how to use KPIs for evaluating an O&M program.

Discussion

By some counts, there are over 100 KPIs that can be related to O&M. These span the topic areas of planning, scheduling, execution, reliability, response, organization, cost, performance, and safety, among others. While many KPIs exist and are well defined, there may be a need to develop KPIs to account for the specifics or intricacies of an organization.

KPIs can be grouped in a number of ways, including by the fundamental elements of the business organization, its economic metrics, and its relevant operational and technical aspects. There can be crossover in KPI application between these elements, leading to intraorganizational indicators residing in one or more groups.

However they are organized, KPIs fall into one of two distinguishing categories— (1) a leading KPI or (2) a lagging KPI. A leading KPI is usually process specific and is a measure of performance before change or degradation. An example of a leading KPI is the percentage of maintenance work orders requiring rework. A lagging KPI is a measure of what has already happened and how it affects the system, process, or business. An example of a lagging KPI is a MTBF. Leading KPIs are used most often to track and define future progress and success; lagging KPIs provide an indication of past success. A well-run organization will have a balanced set of both leading and lagging KPIs.

The tables below present a subset of relevant KPIs (FEMP 2010; NASA 2000 and 2015; Kumar et al. 2011; BSI 2007; Peach et al. 2016) organized by leading or lagging and then into major categories. The categories chosen are typical of service-based organizations with similarities to the federal government.

For many of the KPIs below, targeted values and ranges for acceptance (often referred to as benchmarks) can be established. These benchmarks are typically industry or system specific. The benchmarks that are more generic and/or span industries have been included for reference. For those benchmarks that are industry or system specific, only the recommended trend is provided; these indications include trending upward (i.e., the desired outcome has an upward trend) or trending downward (where the desired trend is downward). In both cases, the O&M management team should strategically track and trend these KPIs over time and record changes month-over-month and year-over-year. These changes will become important to assess, validate, and act upon.

Table 1. Leading KPIs

Area	Topic	Target Value
Work Planning	Planning intensity: Planned work hours over available work hours	Target > 75%, trending upward
	Schedule intensity: Scheduled work hours over available work hours	Target > 90%, trending upward
	Planned downtime: Planned or scheduled downtime or shutdown over total uptime	Target < 10%, trending downward
Work Execution	Schedule: Work orders with defined schedule over total work orders	Target > 95%, trending upward
	Backlog percentage: Work orders in backlog status over total work orders	Target < 5%, trending downward
	Rework percentage: Percentage of work orders in rework status; work orders in rework status over total work orders	Target < 3%, trending downward
Work Administration	Training: Number of training events or days/hours over number of staff	Industry specific, trending upward
Work Administration	Proficiency: Staff carrying certification over total number of staff	Industry specific, trending upward

Table 2. Lagging KPIs

Area	Topic	Target Value
Equipment Effectiveness	Availability: Equipment uptime over expected operating time including planned downtime	Target >90%, trending upward
	Downtime: Unplanned equipment failures/shutdowns over planned equipment failures/shutdowns	Target <25%, trending downward
	MTBF: Total operating time over number of failures	Industry/system specific, trending upward
	Occupant satisfaction: Occupant complaints totaled over time interval (month, quarter, year)	Industry specific, tending downward
	Resource use intensity: Resource use (energy, water, other) over facility area (ft²)	Industry specific, trending downward
Maintenance Cost/ Financial Effectiveness	Maintenance Cost: Total maintenance cost over prior year’s cost	Industry specific, trending downward
	Facility Condition: Total maintenance cost over replacement asset value of equipment	Target < 3%, trending downward
	Cost of Staff: Total internal maintenance personnel cost over total maintenance cost	Industry specific, trending steady/downward
Programmatic Effectiveness	Corrective Maintenance: Corrective maintenance hours over total maintenance hours	Target < 10% to 20%, trending downward
	Preventive Maintenance: Preventive maintenance hours over total maintenance hours	Target > 25% to 35%, trending steady to upward
	Condition-based/Predictive Maintenance: Predictive maintenance hours over total maintenance hours	Target > 45% to 55%, trending upward
Safety and Environment Effectiveness	Condition-based/Predictive Maintenance: Predictive maintenance hours over total maintenance hours	Target > 45% to 55%, trending upward
	Safety Severity: Hours lost to injuries for maintenance personnel over total hours worked by maintenance personnel	Industry specific, trending downward
	Environmental Effectiveness: Number of reportable environmental events over total hours worked by maintenance personnel	Industry specific, trending downward

Solutions and Actions

KPI Selection Criteria

Organizing KPIs into topic areas relevant to the application is an important first step in KPI assessment. In many cases, certain KPIs can be eliminated from further consideration depending on the organization’s type. For example, a service organization (or some parts of the federal government) will have little use for a KPI that relates maintenance cost to units of production.

Ideally, an organization should have as few KPIs as possible while allowing for complete assessment to detect issues, highlight solutions, and offer opportunities for improvement.

The selection process begins with defining the objectives and goals of the maintenance organization across all levels of maintenance activity (NASA 2000). Those responsible for O&M of the targeted system or process should be consulted for input to allow for more complete development and ownership in the process and its implementation.

While goals and objectives will vary with an organization’s maintenance competence, a list of sample goals and objectives are presented as follows:

Improve system availability
Better define current allocation of maintenance time/cost across maintenance types—reactive, preventive, condition-based/predictive
Improve O&M safety
Improve maintenance cost effectiveness
Identify equipment risks that potentially affect reliability
Understand monitoring technology to better track performance
Reduce overall downtime
Better define spare and critical parts availability and management
Improve environmental compliance
Be able to support both leading and lagging assessment needs
Reduce occupant complaints and dissatisfaction.

Once goals and objectives are defined, some level of assessment needs to take place to align these with existing or proposed KPIs. As a general screen, KPI alignment needs to take into account the ability of the KPI components (i.e., the metrics) to be measured. Key questions to answer include the following:

How easy will it be to acquire the necessary metric data?
What will it cost to collect these data?
What tools will be necessary?
How often should the metrics be collected?
How will they be communicated and stored?

Answering these questions will be important in determining the feasibility of any proposed KPI.

KPI Benchmarks

After the proposed KPIs are selected, the identification of some level of comparison or benchmark is required. As discussed, industry benchmarks exist for a number of standard KPIs; however, they may not always be available or applicable. Therefore, an organization should be prepared to track their own KPI values and develop an internal or process-specific benchmark. These internal benchmarks can begin with the goal of a trend, e.g., the proposed KPI value should trend downward over time. As this KPI is assessed over time, values will be generated and easily trended; furthermore, these values then become input for a future benchmark.

In many cases, industry standard benchmarks will be presented as a value range or a series of ranges with identifiers such as below-average, average, above-average, and best in class. Other presentations make use of color-coding outputs in a dashboard function to allow for quick assessments of issues.

KPI Use

The application of KPIs becomes a management exercise. Because most KPIs are time sensitive, acting on the information in a timely manner is important. The interpretation of KPI results should be included in regular O&M procedures and actions. Once interpreted, any resulting or recommended actions should be incorporated into normal O&M work orders and acted upon.

The sharing of KPI results across management teams, system operators, and other stakeholders is critical to improved performance. Highlighting KPIs through public displays, online and interactive score cards, or simple email messages will increase visibility and continue support. Gaining a better understanding within an organization of the frequency of reporting and level of detail will be critical to KPI usage and program success. Depending on the KPI and mode for display (static report, score card, or interactive display), integrating KPI findings into monthly management reporting should be a goal for maximum visibility.

KPIs should not be thought of as static measures used for periodic comparison, but should be continually reviewed for relevancy, accuracy, and frequency of collection and reporting. It is quite common to have KPI calculations changed or modified as systems are improved and benchmarks achieved. It is recommended that KPI designation and calculation be reviewed at least annually for accuracy and appropriateness.

Conclusions and Next Steps

The development and application of KPIs offers valuable methods for building owners and their O&M organizations to measure O&M performance and identify areas in need of improvement.

With the magnitude of KPIs available to choose from, it is easy to select KPIs that provide data and information that is not always actionable. KPIs are only useful if they are based on objectives and have been well designed with measurable and comparable outputs.

Successful O&M programs make use of a strategic combination of leading and lagging KPIs and have active benchmarks to assess performance.

The method and mode of KPI reporting is critical to a program’s success. Dashboards and score cards offer quick assessments of performance and system change. It is these changes that will become important to assess, validate, and act upon.

Definitions

Asset. Maintenance term commonly taken to be any item of physical plant or equipment.
Availability. The proportion of total time that an item of equipment is capable of performing its specified functions, normally expressed as a percentage. It can be calculated by dividing the equipment available hours by the total number of hours in any given period.
Average Life. How long, on average, a component will last before it suffers a failure.
Backlog. Work that has not been completed by the nominated 'required by date.' The period for which each Work Order is overdue is defined as the difference between the current date and the 'required by date.'
Benchmarking. The process of comparing building energy performance to that of similar buildings (cross-sectional benchmarking) or its own historic performance (longitudinal benchmarking). Benchmarking may be performed at the system or component level.
Breakdown. A specific type of failure in which a plant or equipment item is completely unable to function.
Component. A subassembly of an asset, usually removable in one piece and interchangeable with other, standard components.
Computerized Maintenance Management System. A computerized system to assist with the effective and efficient management of maintenance activities through the application of computer technology. It generally includes elements such as a computerized work order system, capabilities for scheduling routine maintenance tasks, recording and storing standard jobs, bills of materials, applications parts lists, and equipment manuals.
Condition Monitoring. The use of specialized equipment to measure the condition of equipment. Vibration analysis, tribology (oil analysis), and thermography are all examples of condition monitoring techniques.
Corrective Maintenance. Any maintenance activity that is required to correct a failure that has occurred or is in the process of occurring. This activity may consist of repair, restoration, or replacement of components.
Downtime. The time that an item of equipment is out of service as a result of equipment failure.
Emergency Maintenance. A maintenance task carried out in order to avert an immediate safety or environmental hazard, or to correct a failure with significant economic impact.
Engineering Work Order. The prime document used to initiate an engineering investigation, engineering design activity or engineering modifications to an item of equipment.
Estimated Plant Replacement Value. The estimated cost of capital works required to replace all the existing assets with new assets capable of producing the same quantity and quality of output. This is a key value often used in benchmarking activities.
Failure. A cessation of proper function or performance, or an inability to meet a standard, or a nonperformance of what is requested or expected.
Key Performance Indicators. A select number of key measures that enable performance against targets to be monitored.

Leading Key Performance Indicators. A measure of performance prior to change or degradation and used to track future performance.
Lagging Key Performance Indicators. A measure of performance of what has already happened and how it impacts the system, process, or business; used to track past performance.
Maintenance. Any activity carried out on an asset in order to ensure that the asset continues to perform its intended functions or to repair the equipment.
Maintenance Schedule. A list of planned maintenance tasks to be performed during a given time period together with the expected start times and durations of each of these tasks.
Maintenance Strategy. A long-term plan that covers all aspects of maintenance management, sets the direction for maintenance management, and contains firm action plans for achieving a desired future state for the maintenance function.
Mean Time Between Failures (MTBF). A measure of equipment reliability equal to the number of failures in a given time period, divided by the total equipment uptime in that period.
Mean Time To Repair. A measure of maintainability equal to the total equipment downtime in a given time period divided by the number of failures in that period.
Planned Maintenance. Any maintenance activity for which a predetermined job procedure has been documented and all labor, materials, tools, and equipment required to carry out the task have been estimated and their availability assured before commencement of the task.
Predictive Maintenance. An equipment maintenance strategy based on measuring the condition of equipment in order to assess whether it will fail during some future period and then taking appropriate action to avoid the consequences of that failure.
Preventive Maintenance. An equipment maintenance strategy based on replacing, overhauling, or remanufacturing an item at a fixed interval, regardless of its condition at the time; also known as scheduled maintenance.
Proactive Maintenance. The collection of efforts to identify, monitor and control future failure with an emphasis on the understanding and elimination of the cause of failure.
Reactive Maintenance. The work required to restore a component to a condition substantially equivalent to its originally intended and designed capacity, efficiency, or capability (NASA 2000). Reactive maintenance is also known as corrective maintenance, and a reactive maintenance strategy is also known as run-to-failure. No actions or efforts are taken to maintain the equipment as the designer originally intended to ensure design life is reached.
Reliability. The probability that a system will continue to function for the amount of time needed to fulfill its mission.
Reliability Centered Maintenance. The process used to determine the most effective approach to maintenance. It involves identifying actions that, when taken, will reduce the probability of failure and that are the most cost-effective. It seeks the optimal mix of condition-based actions, other time- or cycle-based actions, or run-to-failure approach.
Repair. The work required to restore a facility or component thereof, including installed equipment, to a condition substantially equivalent to its originally intended and designed capacity, efficiency, or capability.
Resource Use Intensity. The key performance indicator relating a resource use (e.g., electricity, natural gas, or water) with another building metric (e.g., total square footage or occupancy) and a time metric. Most commonly listed as an energy use intensity (EUI) in kBtu/ft²/year.
Risk. The potential for an unwanted outcome resulting from an event or sequence of event and is quantified in terms of the event likelihood and associated consequences.
Routine Maintenance Task. Any maintenance task that is performed at a regular, predefined interval.
Schedule Compliance. One of the KPIs often used to monitor and control maintenance. Schedule compliance is the number of scheduled work orders completed in a given time period (normally one week) divided by the total number of scheduled work orders that should have been completed during that period according to the approved maintenance schedule for that period.
Scheduled Maintenance. Any maintenance work that has been planned and included on an approved maintenance schedule.
Scheduled Operating Time. The time during which an asset is scheduled to be operating according to a long-term production schedule.
Scheduled Work Order. A work order that has been planned and included on an approved maintenance schedule.
Shutdown. That period of time when equipment is out of service.
Shutdown Maintenance. Maintenance that can only be performed while equipment is shutdown.
Standing Work Order. A work order that is left open either indefinitely or for a predetermined period of time for the purpose of collecting labor hours costs and/or history for tasks for which it has been decided that individual work orders should not be raised.
Unplanned Maintenance. Any maintenance activity for which a predetermined job procedure has not been documented or for which all labor, materials, tools, and equipment required to carry out the task have not been estimated and their availability assured before commencement of the task.
Unscheduled Maintenance. Any maintenance work that has not been included on an approved maintenance schedule prior to its commencement.
Uptime. The time that an item of equipment is in service and operating.
Useful Life. The maximum length of time that a component can be left in service before it will start to experience a rapidly increasing probability of failure.
Utilization. The proportion of available time that an item of equipment is operating. Calculated by dividing equipment operating hours by equipment available hours. Generally expressed as a percentage.
Work Order. The prime document used by the maintenance function to manage maintenance tasks. It may include such information as a description of the work required, the task priority, the job procedure to be followed, the parts, materials, tools, and equipment required to complete the job, the labor hours, costs and materials consumed in completing the task, as well as key information on failure causes, what work was performed, etc.
Work Request. The prime document raised by user departments requesting the initiation of a maintenance task. This is usually converted to a work order after the work request has been authorized for completion.
Workload. The amount of labor hours required to carry out specified maintenance tasks.

Sources of Information

BSI – British Standards Institution. 2007. Maintenance – Maintenance Key Performance Indicators (BS EN 15341:2007). Based on European Standard EN 15341:2007. Brussels, Belgium. https://webstore.ansi.org/standards/bsi/bsen153412007

FEMP ─ Federal Energy Management Program. 2010. Operations & Maintenance Best Practices: A Guide to Achieving Operational Efficiency. Release 3.0. Prepared by Pacific Northwest National Laboratory for FEMP, Richland, WA. at https://www.wbdg.org/FFC/DOE/DOECRIT/femp_omguide.pdf.

Kumar, Uday, Diego Galar, Aditya Parida, Christer Stenström, and Luis Berges-Muro. 2011. Maintenance Performance Metrics: A State of the Art Review. Manufacturing Engineering and Advanced Metrology Group, Lulea University of Technology, Lulea, Sweden. http://ltu.diva-portal.org/smash/get/diva2:994674/FULLTEXT01.pdf

LBNL – Lawrence Berkeley National Laboratory. 2017. Using EMIS to Identify Top Opportunities for Commercial Building Efficiency. LBNL-1007250, Lawrence Berkeley National Laboratory, Berkeley, CA. https://eta.lbl.gov/sites/all/files/publications/1007250.pdf

MIT – Massachusetts Institute of Technology. 2018. Leading with Next-Generation Key Performance Indicators. MIT Sloan Management Review Research Report: 60180. Cambridge, MA. https://sloanreview.mit.edu/wp-content/uploads/2018/06/9dbcc7b64e.pdf

NASA – National Aeronautics and Space Administration. 2000. Reliability Centered Maintenance Guide for Facilities and Collateral Equipment. National Aeronautics and Space Administration, Washington, D.C. https://www.hq.nasa.gov/office/codej/codejx/Assets/Docs/RCMGuideMar2000.pdf

NASA – National Aeronautics and Space Administration. 2015. Procedural Requirements: Facilities Maintenance and Operations Management – Appendix G Performance Measurement. National Aeronautics and Space Administration, Washington, D.C. https://nodis3.gsfc.nasa.gov/displayDir.cfm?Internal_ID=N_PR_8831_002F_&page_name=AppendixG

NE-SpinTech. 2018. Glossary of Maintenance Terms. New England SpinTech: Reliability and Maintenance Solutions. Spencer, MA. Accessed at https://www.ne-spintech.com/wp-content/uploads/2018/04/Glossary-of-Maintenance-Terms.pdf

Peach, Rina, Herman JP Ellis, and Krige Visser. 2016. “A Maintenance Performance Measurement Framework that Includes Maintenance Human Factors: A Case Study from the Electricity Transmission Industry.” South African Journal of Industrial Engineering 27(2):177–189. https://doi.org/10.7166/27-2-149

[1] References for definitions: FEMP 2010; LBNL 2017; NASA 2000; NE-SpinTech 2018

Published December 2020