This overview contains information regarding the Background, Prospective Indicators for State Use in Performance Agreements, Background: Environmental Indicators, Evaluation Criteria, Conceptual Frameworks, Sources, To the Reader/Reviewer and Selection Criteria References.
The State Environmental Goals and Indicators Project (SEGIP) is a cooperative agreement between the Florida Center for Public Management (FCPM) of the Florida State University and the U.S. Environmental Protection Agency (U.S. EPA) and represents an initiative to ensure the effective participation of state environmental management agencies in the development of environmental goals and indicators. The purpose of this project is to:
assist state environmental agencies in improving their environmental management capabilities by providing procedural, technical and financial assistance in the development of environmental goals and indicators and in integrating such goals and indicators into their environmental management systems.Within the context of this mission SEGIP has volunteered to provide U.S. EPA with a group of prospective indicators that are appropriate for use in structuring state agreements as conceptually proposed for Performance Agreements. These agreements provide the foundation for a fundamental reorientation in the way U.S. EPA and the states conduct their business by focusing U.S. EPAs funding processes on the achievement of explicit environmental results rather than on the more traditional administrative, process and program measures that currently drive funding. The cornerstone of making this new partnership work is the capacity to provide measures of strategic environmental results that reflect concerns intrinsic to the U.S. EPA-State relationship. The selection of such indicators is a matter of considerable importance since they will be used to measure whether states are making environmental progress in their use of U.S. EPA reonds.
On June 15, 1995 FCPM delivered to U.S. EPA Prospective Indicators for State Use in Performance Partnership Agreements -- a listing and documentation for 17 initial indicators (list provided in Appendix B).
The indicators provided in that document represented an initial list of measures that SEGIP believes could have utility in structuring Performance Agreements. The list provided was conservative: many other indicators could have been listed as candidates. The indicators that were provided, however, were those which SEGIP believed to be the best currently available and with little or no qualification could be used to support Performance Agreements. In identifying these indicators the following Qualification Standards were applied:
The Qualification Standards for these indicators were restrictive. This was done to ensure that this initial group was of the highest quality possible. To develop further indicators to meet the full needs of Performance Agreements requires the identification of indicators of uncertain quality or less direct application. In June, no attempt was made to identify such indicators. The development of these less certain, lower quality, and/or surrogate indicators was identified as the next step in the process.
It is obvious that many additional indicators will be needed to meet the measurement requirements of Performance Agreements or national goals or any other future use of environmental indicators. SEGIP was aware of numerous other candidate indicators that may have utility and other indicators that meet the requirements set out in the above paragraph. SEGIP knew of several strong candidates that were not included because short time constraints prevented an evaluation of the indicators characteristics sufficient to provide the confidence necessary to include them in this initial group.
Further, there are numbers of good indicators that come from
national
databases that are not comparable across states, but that
are quite
useful for individual states. Still further, there is a rich
potential
for indicators in a number of smaller, more limited
databases held by
U.S. EPA and other federal agencies. Restricted to smaller
groups of
states and collected on uncertain schedules, these data sets
have the
potential to provide quality indicators for some states and,
with further
development, perhaps for all states. These indicators were
not presented
in June due to their:
Given the breadth of the topics requiring measurement in Performance Agreements, such restrictive criteria could not yield a sufficient number of high quality indicators to meet all needs. At the request of U.S. EPA, SEGIP has, in this document, taken the next step and provide additional indicators that can be considered as potential measures in Performance Agreements, indicators that may or do violate one or more of the Qualification Standards or have unknown metadata (information about the data). In providing such indicators, however, SEGIP has structured them into lists based upon their known characteristics to allow potential users at least some basic ability to differentiate the relative capacity of individual indicators to meet intended needs.
Indicators presented in this document will be classified at three different levels. At the first level indicators will be broadly grouped based upon their present availability for use.
FCPM classifies each of its indicators according to its immediate availability for use, sorting each indicator into one of three groups based on its current utility. Classifying indicators in this way allows the identification of indicators based on when they will be of direct use in an indicator system and sets directions for future growth in available indicators.
Type A: Indicators for which adequate data are available now and can be used to support the indicator without significant additional cost considerations. To be classified as a Type A, an indicator:
Type B: Indicators which are presently feasible, but cannot be provided due to inordinate cost, analytical complexity or time constraints. Type B indicators are those that could be made available if some barrier can be overcome. The data needed to produce the indicator exist but because of cost concerns, analytical difficulties, time constraints, manpower issues or some other impediment, the indicator cannot be provided.
Type C: Prospective indicators for which there is no reasonable prospect of development without some extraordinary expenditure of resources. Type C indicators are purely prospective. The data do not exist and there is no clear intent to collect. Type C indicators exist as designs only.
At this first level all indicators will be grouped into one of these three broad categories. Virtually all indicators included in this analysis are Type A indicators. Some Bs and Cs have been included to accommodate U.S. EPA efforts to establish new data collection processes.
At the second level all indicators classified as As are placed into separate lists based upon what is known about:
Five separate lists of Type A indicators responding to these concerns are provided:
Fact sheets for the Type A, List 1 indicators are presented as Appendix A.
Finally, at the third level, within each list indicators are grouped according into subjective, broad categories:
Over the last three years, the U.S. EPA Office of Water has been developing a set of environmental indicators in conjunction with a variety of stakeholders. At a recent National Water Environmental Indicators Workshop June 22-23, 1995, candidate indicators were discussed by participants and 16 indicators are moving forward for use in Performance Agreements and other management agreements. To demonstrate and encourage consistency, the Office of Water environmental indicators are crosswalked with the SEGIP prospective indicators as presented in Appendix E.
Environmental indicators describe, analyze, and present scientifically-based information on environmental conditions, trends, and their significance. Environmental indicators and aggregated environmental indices top an information pyramid whose base is primary data derived from monitoring and data analysis. Indicators and indices must be analytically sound and have a methodology behind their measurements.
An index is an aggregation of indicators which summarizes often large quantities of related information by using a systematic procedure to weight, scale and aggregated multiple variables into a single summary output. Although environmental indices have been discussed and conceptualized since the mid-1960's, their widespread acceptance has been limited to only a few environmental areas, most notably biological water quality measurement.
The relationships among indicators, data and information are illustrated in the figure below, noting that effective indicators need to have a communicative format that is designed for an explicit group while retaining the essential data necessary to represent complex environmental processes.
Ideally, each indicator finally included in a Performance Agreement should meet a series of standards designed to ensure high and consistent quality. Listed below are the selection criteria employed by SEGIP in all of its indicator work. The Selection Criteria were drawn from a fairly rich and widely-accepted literature on selection criteria (see references at the end of this section). SEGIP prepared a matrix of the criteria referenced and selected criteria most appropriate to its work with the states, dividing them into two basic types:
Essential Criteria include:
Preferable Criteria include:
To provide additional information regarding the individual indicators, each indicator will be summarized in a matrix format that will allow the application of two different conceptual indicator classifications.
Pressure-State-Response. A widely-used framework for environmental indicators has been developed from a seemingly simple set of questions: What is happening to the state of the environment or natural resources? Why is it happening? What are we doing about it? Indicators of changes or trends in the physical or biological state of the natural world answer the first question; indicators of stresses or pressures from human activities that cause environmental change answer the second, and measures of programs and policies created in response to environmental problems answer the third.
This pressure-state-response framework, developed by the Organization for Economic Cooperation and Development (OECD) from earlier work by the Canadian government, is a means for organizing indicators that reflects three different, but related and integrated, types of indicators for each environmental issue. They are:
The value of this approach is that it allows a group of indicators to show for each issue the source of the problems affecting the issue, the environmental status of the issue or resource, as well as the measures that society is taking to deal with the issue.
In particular, pressure indicators are useful in formulating policy objectives and in evaluating policy performance because they show short-term changes and more direct cause and effect relationships in the environment. State indicators involve longer lag times in data collection and less direct cause and effect relationships, but are essential to long-term evaluations of the environment and programs to improve environmental conditions.
In this reports matrix, each indicator has been assigned one of these three classifications as a demonstration of the use of this framework and to show what types of indicators are being recommended.
A graphic below shows the linkage among pressure, state, and response indicators and that, on a time scale, responses to environmental trends should ideally be developed after pressure and state data are known.
Hierarchy of Indicators. Initially created by the U.S. EPA Office of Policy, Planning and Evaluation, the Chesapeake Bay Program has developed an indicator-driven planning process that successfully uses a broad and extensive range of environmental indicators that focus actions on the improvement of the resource. To measure the level of quality of each indicator with respect to the strength of the type of data, they have developed a six point scale for rating indicators. That system is summarized below:
Hierarchy of Indicators |
|||||
|---|---|---|---|---|---|
| Administrative | Environmental | ||||
| 1 | 2 | 3 | 4 | 5 | 6 |
| Actions by Federal or State Regulatory Agency | Responses of the Regulatory Community or Society | Changes in Discharge of Emission Quantities | Changes in Ambient Conditions | Changes in Uptake and/or Assimilation | Changes in Health, Ecology of Other Effects |
Indicators rated as a 1 or a 2 (Administrative) do not reflect environmental measures and are generally not acceptable for the present purposes. In the pressure-state-response context, all of the indicator levels have value in describing environmental trends and conditions and societal responses to them. However, if we are lacking pressure (Level 3) or state (Levels 4, 5, 6) indicators, response (Levels 1 and 2) indicators have little meaning or validity. In contrast, a measure with a rating of 6 is able by itself to directly describe a human health or ecological health value, and presents a very powerful type of information. These ratings are provided to show the general quality of data supporting the indicators included in this document.
A chart showing the relationship between the hierarchy of indicators and the Pressure-State-Response framework follows the text of this section.
SEGIP has evaluated environmental indicators from the following documents for consideration in the annotated listing:
States
U.S. Environmental Protection Agency
Other Agencies
As this document is aimed at readers and reviewers who have environmental management responsibilities at the state and federal levels, a suggestion is offered that the questions posed by the U.S. EPA Office of Water in their Draft Report for Discussion--National Indicators for Water, June 16, 1995, be used as guidance for considering the indicators presented:
Also, it is well to remember that even if an indicator has been identified for a certain issue area, the issue may not be fully-described using indicators due to a lack of information or quality data.
This annotated listing does not mean we should not develop or identify other indicators that effectively describe environmental trends and conditions. The listing is simply a quickly produced set of indicators potentially usable in Performance Agreements. The State Environmental Goals and Indicators Project will endeavor to monitor the environmental indicators used by the states in Performance Agreements and report on their usefulness as accountability measurements on an annual basis. SEGIP will also work with the states over the transition year of the National Environmental Performance Partnership System to identify environmental indicators that can be used in future years.