Introduction

Last Update April 20, 1996

Many governmental agencies operate programs for years and spend millions of dollars without ever attempting to make any assessment of the impacts that they are having or without accurately documenting the status and trends of the subject of their efforts. The result can be the loss of focus for the program, the inefficient use of financial and personnel resources, and the loss of public and political support. For the past five years planning professionals at all levels of government, particularly environmental planners, have been aggressively working on redesigning their policy planning processes and have been actively building an intergovernmental partnership to improve their joint public policy management skills by adding measurement of progress, by increasing accountability, and by focusing on results.

A major area of importance is the development of environmental and growth management indicator systems. Indicators are useful tools for a wide variety of management purposes and the capacity of individual public organizations to develop policy is greatly enhanced by the availability of good indicator systems.

The multiple uses of indicators and the pivotal role they play in any serious attempt to improve public management have focused much attention on procedural and technical issues concerning the development of indicator systems at all governmental levels. International, national, and regional conferences and a variety of publications on indicators and indicator systems development have created increasingly high levels of interest, especially among state and regional agencies. Three years ago interest in environmental and growth management indicators was scant and sporadic. Today interest in such indicators has exploded with indicator development activities occurring at numerous locations in the federal government, most of the states, many regional organizations, in local governments, and even among private corporations.

Florida Assessment of Coastal Trends (FACT)

The Florida Coastal Management Program (FCMP) has taken steps to add some important management tools that can improve its vision in dealing with the future of Florida's coastal areas. The most important tool being added is an indicator system that provides a comprehensive perspective of the important environmental, economic, and social values associated with the coast. Such a system provides a means of evaluating Florida's progress in protecting its coastal areas, provides a basis for making strategic decisions about programs and financial resources, and provides information about coastal issues and problems to other decision-makers and the general public.

The Florida Coastal Management Program has contracted with the Florida Center for Public Management (FCPM) of the Florida State University to assist in the design and development of such a system. The contract required that:

By September 30, 1995 prepare a report containing a structured collection of environmental, growth management, economic and social indicators that collectively describe the status of Florida's coastal areas, that portray the historic trends affecting coastal Florida and that project Florida's coastal future.

FCPM was required to accomplish the following to successfully complete this objective:

work intensively with Florida Coastal Management Program staff to develop a design and structure for the system that completely reflects what is needed and desired for the system,
assess existing sources of indicator data and information already known and accessible to FCPM and utilize that which is appropriate,
conduct research as needed to identify other desirable data and information needed to develop additional key indicators,
prepare specific technical documentation to support each indicator to include a discussion of the concept of the indicator, the source of the data, a contact person with address and phone number, the format in which the data resides, any costs associated with its acquisition, the frequency of collection of the data, the geographic coverage, an assessment of the strengths and limitations of the indicator and its data, a discussion of how the data will be collected, a brief analysis of the data, and a graphic or tabular display.

This Florida Assessment of Coastal Trends (FACT) document represents the product of these activities. Structured across its nine issue areas are 98 indicators that reflect important measurements of key environmental, social, economic, cultural, and aesthetic issues affecting Florida’s coastal areas.

FACT Methodology and Outcome

The process of developing FACT included the following steps:

Issue Development. Indicator systems should be developed to explicitly support or reinforce known planning or policy management structures. Ideally such structures should support issues that reflect end-result concerns. In this case the indicator process itself represented the initiation of a planning process, so a fully appropriate issue structure did not pre-exist. The first step in the process, therefore, was to initiate an issue development process designed to identify the critical, strategic issues facing Florida’s coastal future over the next five to twenty years. These issues would then be used to provide the organizational framework for identifying the required indicators. On January 24, 1995 a workshop was convened and a number of coastal experts representing a range of coastal policy interests spent a day discussing coastal priorities. The principal product of their deliberations was the development of a list of nine strategic issues facing Florida’s coastal areas.

Sub-issue Development. The broad strategic issues provide general direction concerning the development of indicators. To ensure, however, that the indicators selected are focused to best reflect the issue, an additional process of further refining each issue into its principal dimensions was completed. Workshop participants reviewed each issue and identified two to four sub-issues or components of each issue. These sub-issues then became the final framework around which indicators were developed. The nine issues and their associated sub-issues are as follows:

  1. Impact of Growth in the Coastal Zone
  2. Disruption of Coastal Physical Processes
  3. Responding to Coastal Threats and Hazards
  4. Degradation and Restoration of Coastal Ecosystems
  5. Managing Fresh Water Allocation
  6. Sustaining the Human Uses of the Coast
  7. Balancing Public and Private Uses of Resources
  8. Preservation of Cultural and Aesthetic Resources
  9. Encouraging Public Awareness and Involvement

Preliminary Indicator Identification. Having established the substantive framework within which indicators would be developed, the next step was to brainstorm the desired indicators. This process involved two separate objectives:

determining what are the desired indicators required to effectively measure the dimensions of each issue, regardless of the current availability of data to support the indicators, and
identifying data sources that are currently capable of supporting acceptable indicators.

The final activity of the workshop was to examine each issue and brainstorm potential indicators to be included in the system. This input was used by staff as the beginning point for the actual development of indicators.

Indicator Development. Using this base of input, staff then began systematically investigating individual indicators and, where appropriate, developing support documentation for indicators that were finally selected for inclusion. Most of the ind icators initially suggested in the brainstorming session could not be included. Reasons for elimination from consideration included lack of data and the existence of a better indicator for the same dimension, among others. Hundreds of potential indicato rs were pared down to the 98 in the present system. The process of indicator development was greatly aided by the assistance of four students from the Department of Urban and Regional Planning of the Florida State University who used a studio held in co njunction with FACT as a means of meeting their capstone requirement for their Master’s degrees. Their assistance provided an invaluable extension of staff capacity in conducting research.

Indicator Review. Initial products were reviewed at several points by Florida Coastal Management Program staff and by individuals responsible for the information used in the indicators.

Final Publication. The publication was assembled in its present form. Each indicator is displayed in a standardized format that reflects the reasons for its inclusion, that identifies technical information regarding its source, format and limitat ions, and that provides a brief analysis and/or graphical display of any trends associated with the indicator. In addition, each indicator displays certain conceptual information commonly used to characterize indicators concerning the availability of dat a for the indicator and the strength of the information supporting the indicator. Discussion of those conceptual frameworks and others used to support this system are included in the following section.

Conceptual Frameworks

Definitions. Basic indicator-related definitions include:

Parameter: A property that is measured or observed.
Indicator: A parameter, or a value derived from a parameter, which points to/provides information about/describes the state of a phenomenon/environment/area with a significance extending beyond that dire ctly associated with a parameter value.
Index: A set of aggregated or weighted parameters or indicators.

Uses. It is axiomatic that environmental agencies at all levels should use indicators to make key strategic decisions concerning the success of their efforts and to make important program and budget adjustments. Startlingly, this is much too oft en not the case. Federal environmental agencies and their associated state agencies are driven by huge, monolithic programs which over the years have taken on importance unto themselves. Most of the measurement and reporting that occurs deals with progr am activities rather than measures that deal with what is actually happening. With leadership from some federal agencies and a few of the states, this is starting to change with the initiation of projects that are beginning to use results-based data and information as the foundation for making decisions.

Indicators represent components or processes of real world systems. This means that they function as models and have all of the possibilities and limitations that models offer. The numerical values of indicators tend to have special meaning to particula r observers -- a meaning that goes beyond the numerical value itself. For example, the number of top trophic level predator birds could be used to represent the vitality of a whole ecosystem based on the species habitat requirements. In other words, in dicators generally simplify in order to make complex phenomena quantifiable in such a manner that communication is either enabled or promoted.

Indicators are only as useful as the information they are based upon. The measuring devices that are used to evaluate problems, their causes, and the steps taken to address those problems need to be selected carefully from this information and presented to decision-makers and to the general public in ways that are relevant to them and can be readily understood.

Indicators are useful tools for a wide variety of management purposes. The availability of indicator systems at the state level of government can increase the capacity of individual states to develop policy. Indicators can be used as a:

mission-level tool to provide a broad evaluation of an agency’s performance,
measurement foundation for structuring goals,
basis for measuring and communicating achievement and progress,
basis for making strategic planning and budgeting decisions,
means of evaluating the performance of individual programs and activities,
tool in building particular constituencies,
basis for the development of education programs, and
tool for public relations and information dissemination.

By measuring success and failure of environmental protection programs by their environmental results rather than by bureaucratic inputs, regulators, the regulated community, and the public should be encouraged to find the least costly, most effective ways to achieve those results within a geographic jurisdiction.

The relatively young and emerging science and art of developing indicators and indicator systems is evolving some simple conceptual tools to provide clarity and order to the process. The following presents several conceptual approaches presently in use b y FCPM in describing and displaying the indicator systems with which they are associated.

Qualification Standards. For each individual indicator system being developed, it is useful to list with as much precision and with as much completeness as possible the specific criteria used to define an acceptable indicator for that system. Des cribed in such criteria might be such concerns as:

the geographic scope of the indicators (national, statewide, regional, ecosystem, local),
the selection criteria used,
the acceptable types of indicators (environmental, program, administrative),
the availability of data (is it available now or is it a prospective indicator?), or
how it is intended to be used (its purpose).

By clearly identifying such standards early in the process and constantly comparing the selection of indicators against them, attention can remain focused on indicators appropriate for the system to be developed.

Indicator qualification standards employed in the development of FACT include:

the geographic scope of the indicators must measure an activity or condition that affects Florida’s coast or its 35 coastal counties,
the indicator must reflect an important dimension of one of the nine strategic issues, and
any Type A indicator (see below) must meet FCPM Indicator Selection Criteria (see below).

Classification of Availability. FCPM classifies all of its indicators according to their immediate availability for use, sorting indicators into one of three groups based on their current utility. Classifying indicators in this way allows the ide ntification of indicators based on when they will be of direct use in an indicator system and sets directions for future growth in available indicators. This classification is used in the almost universal circumstance where development of the indicator s ystem lacks a sufficient number of existing indicators with data that meet FCPM Indicator Selection Criteria to implement a full indicator system. The use of availability classification allows the presentation of indicators that may not currently exist a s part of the system in order show what will be needed to establish a fully functioning system. Three types of indicators are identified:

Type A: Indicators for which adequate data are available now and can be used to support the indicator without significant cost considerations. To be classified as a Type A, an indicator:

meets all essential selection criteria and most preferred criteria,
is presently available for use in its present condition, and
can be acquired easily at little or no cost.

Type B: Indicators which are presently feasible, but cannot be provided due to inordinate cost, analytical complexity or time constraints. Type B indicators are those that could be made available if some barrier can be overcome. The data needed to produce the indicator exist but because of cost concerns, analytical difficulties , time constraints, manpower issues or some other impediment, the indicator cannot be provided.

Type C: Prospective indicators for which there is no reasonable prospect of development without some extraordinary expenditure of resources. Type C indicators are purely prospective. The data do not exist and there is no clear intent to collect. Type C indicators exist as designs only.

Selection Criteria. Ideally, each indicator finally included in an indicator system should meet a series of standards designed to ensure high and consistent quality. Listed below are the selection criteria employed by FCPM in all of its indicator work. Selection criteria are of two types:

  1. essential -- criteria an indicator must meet, and
  2. preferable -- criteria an indicator should meet

Essential Criteria include:

Measurable: The indicator measures a feature of the environment that can be quantified simply using standard methodologies with a known degree of performance and precision.

Data Quality: The data supporting the indicators are adequately supported by sound collection methodologies, data management systems, and quality assurance procedures to ensure that the indicator is accurately repre sented. The data should be clearly defined, verifiable, scientifically acceptable, and easy to reproduce.

Importance: The indicator must measure some aspect of environmental quality that reflects an issue of major national importance to states and to the federal government in demonstrating the current and future conditi ons of the environment.

Relevance: The indicator should be relevant to a desired significant policy goal, issue, legal mandate, or agency mission (e.g., contaminated fish fillets for consumption advisories; species of recreational or comme rcial value) that provides information of obvious value that can be easily related to the public and decision-makers.

Representative: Changes in the indicator are highly correlated to trends in the other parameters or systems they are selected to represent.

Appropriate scale: The indicator responds to changes on an appropriate geographic (e.g., national or regional) and/or temporal (e.g., yearly) scale.

Trends: The data for the indicator should have been collected over a sufficient period of time to allow some analysis of trends or should provide a baseline for future trends. The indicator should show reliability over time, bringing to light a representative trend, preferably annual.

Decision support: The indicator should provide information to a level appropriate for making policy decisions. Highly specific and special parameters, useful to technical staff, will not be of much significance to policy staff or management decision-makers.

Preferable Criteria include:

Results: The indicator should measure a direct environmental result (e.g., an impact on human health or ecological conditions). Indicators expressing changes in ambient conditions or changes in measures reflect ing discharges or releases are acceptable, but not preferred. Process measures (e.g., permits, compliance and enforcement activities, etc.) are not acceptable.

Understandable: The indicator should be simple and clear, and sufficiently nontechnical to be comprehensible to the general public with brief explanation. The indicator should lend itself to effective and appealing display and presentation.

Sensitivity: The indicator is able to distinguish meaningful differences in environmental conditions with an acceptable degree of resolution. Small changes in the indicator show measurable results.

Integrates effects/exposures: The indicator integrates effects or exposure over time and space and responds to the cumulative impacts of multiple stressors. It is broadly applicable to many stressors and sites.

Data comparability: The data supporting an indicator can be compared to existing and past measures of conditions to develop trends and define variation.

Cost effective/availability: The information for an indicator is available or can be obtained with reasonable cost and effort and provides maximum information per unit effort.

Anticipatory: The indicator is capable of providing an early warning of environmental change.

Hierarchy of Indicators. The Hierarchy of Indicators was developed to measure the quality of indicators intended to reflect environmental values. The ability of individual indicators to provide information varies significantly. The very best ind icators are those that provide direct measures of the health of humans, animals, and plants. Unfortunately, there are relatively few areas where such definitive, end-result kinds of measures are available. Where such quality indicators are not available other, less direct, measures must be used. The table below identifies a hierarchy of indicator quality that reflects gradations in the value received by different types of indicator data.

Data types 1 and 2 measure administrative, bureaucratic, or activity measures and are only weakly associated or completely un-associated with the capacity to measure environmental quality. Some of the environmental indicators in FACT are types 1 or 2, and those that do exist may be eventually replaced by higher quality indicators.

Data types 3-6 are measures that reflect indicators of environmental performance. As the indicators progress from type 3 to type 6, the quality of environmental information increases. Ideally, all indicators should be type 6 indicators, and future versi ons of FACT should focus on improving the overall quality of the indicators in the system.

Some indicators are included that measure neither administrative concerns nor environmental issues, but instead measure closely related concerns or issues that are indirectly associated with environmental performance or portray coastal concerns unrelated to the environment. The measures are coded "I" for Indirect.

Hierarchy of Inidcators
Administrative Environmental Indirect
1 2 3 4 5 6 I
Actions by Federal or State Regulatory Agency Responses of the Regulatory Community or Society Changes in Discharge of Emission Quantities Changes in Ambient Conditions or in the Quantities of Natural Resources Changes in Uptake and/or Assimilation Changes in Health, Ecology of Other Effects Measures Which Reflect Concerns of Significance to Environmental Issues or Policy But Are Not Themselves Indicators of Environmental Performance or Reflet Indicators Unrelated to Environmental Issues