Adaptive Management Tools: "Limiting Factor" Evaluation

A recent article by Raymond Gullison and Jared Hardner defines an alternative approach to the design of evaluations, which is intended to address the complexities of long time horizons for changes from natural resource management, and identifying the key measurable changes with which to assess the success of a natural resource decision or program. The author's call the approach "Limiting Factor Analysis" based on identifying those factors most likely to impede success of a program, and evaluating the success of a policy based on its impact on these limiting factors. This approach could be directly used in water planning - where often the success of a plan or planning process may not be known for some time, and the changes to water availability directly related to the plan are often difficult to untangle.

What is Limiting Factor Analysis?

The authors borrow the term "limiting factor" from agricultural and ecological research, where it refers to factors that have the greatest influence in limiting the growth or abundance of an organism. In their approach to evaluation, the term can be used for those factors that present the greatest threat to the successful outcome from a natural resource management intervention, plan or program. Here we will consider the approach in relation to water planning. The authors have identified a transferable but core list of potential limiting factors, which include:

Inadequate or incomplete scientific understanding
Changes or impediments in policy
Changes or insufficient legislative protection
Inadequate institutional capacity or access to resources
Economic pressures that lead to changes or compromises in the planning process
Insufficient enforcement of laws, regulations or monitoring requirements
Inadequate stakeholder support
Insufficient access to funding or other resources necessary to achieve the outcome

They recommend a three step approach to limiting factor evaluation:

First, identify limiting factors. Review the core list of limiting factors and suggest which factors might impede the achievement of the water planning objectives in the plan. This step produces a list of limiting factors as specific as possible to the water planning process.
Next, score limiting factors. By working in conjunction with stakeholder groups impacted by the plan, the planner should rank the status of each limiting factor at the beginning of the planning process, and then again at the release of the draft, and finally at some agreed point after the plan has been finalised. The authors tend to use simple categories ranging from "presents a complete impasse to success" to "does not limit the outcome in any way".
Then, it is essential to understand the management response. The planner and others involved in the evaluation then identify all of the other stakeholders (such as irrigation groups, conservation groups, government agencies, and so on) that are working at overcoming the various limiting factors identified, and how they are going about it, and their likelihood of success. This helps to clarify what occurs as a consequence of the plan or the planning process, and what is beyond the control or influence of the water plan.

Download the full article here.

Abstract

In recent years, donors to biodiversity conservation projects have sought greater accountability, using professional evaluators to help assess the degree to which grantees are achieving conservation objectives. One of the most formidable challenges evaluators face is the time required both for ecological systems to respond to management interventions and over which grantees must maintain their biodiversity conservation gains. The authors present a simple methodology, called limiting factors analysis, which was developed during the course of evaluating several large portfolios of conservation projects. The method is a practical basis for rapidly assessing whether current conditions are likely to prevent grantees from achieving their long-term objectives. Use of the methodology is illustrated with examples from recent evaluations of the Andes Amazon Initiative and the Columbia Basin Water Transactions Program.

Citation

Gullison, R., & Hardner, J. (2009). Using limiting factors analysis to overcome the problem of long time horizons. In M. Birnbaum & P. Mickwitz (Eds.), Environmental program and policy evaluation: Addressing methodological challenges. New Directions for Evaluation, 122, 19–29.