By completing steps 1-6 of the CAP Basic Practices process you and your project team will have done much of the work needed to design an effective program to measure your conservation results. These earlier steps outline the desired outcomes and key assumptions underlying your project that serve as the basis for your measures plan. Well-defined objective statements and strategic actions (Step 6: Develop Strategies) identify threat-based and action-based indicators to measure. The threat summaries (Step 4: Identify Critical Threats) and diagrams from your situation analyses (Step 5: Complete Situation Analysis) will suggest additional candidate indicators. Initial viability analyses identify candidate key ecological attributes and indicators for measuring the impacts of actions on targets or for periodically tracking their status (Step 3: Assess Viability). If your project team invests the up-front time in these design steps, then deciding what to measure and developing your measures plan will be straightforward. Commonly used methods follow the sequence of numbered steps in Figure 1. Figure 1: Commonly used method for developing CAP measures plan
1. Determine Strategy Effectiveness Information NeedsYou should start by identifying what information is needed to track the effectiveness of strategies being implemented by your project. Resources are being spent to achieve desired results and you need to know whether the current course of action is showing progress and should continue or is not showing progress and should be revised. Every stated objective should have at least one indicator used to track progress towards achieving the desired results. The number and type of strategy effectiveness indicators needed per objective will vary depending upon the complexity, risk, or uncertainty associated with the strategic actions being implemented. If your project team has developed good viability assessments and written clear and measurable objectives and described the strategic actions needed to achieve them, you will already have the most important information you need to identify priority strategy effectiveness needs and corresponding draft indicators. In many cases, the selection of strategy effectiveness indicators is very straightforward. Because most strategies focus on abating critical threats or improving degraded key ecological attributes, strategy effectiveness information needs are often associated with the highest ranked threats and the key ecological attributes of targets that are of greatest concern to the project team. Indicators for measuring the effectiveness of strategies should correspond to threats listed in the threat summary table and/or to key ecological attributes of concern listed in the viability assessment. Consider the following examples of strategy effectiveness indicators for objectives from actual Conservation Action Plans: Project: Cookson Hills Objective: Secure legal protection on 18,000 acres by 2015 Indicator: Acres in legal protection (addresses critical threat) Project: Bering Sea Objective: Reduce current (2005) number of albatross caught in longlines & nets
by 50% by 2010 in US waters and by 2015 in Russian waters Indicators: Short-tailed albatross incidental take (addresses critical threat),
Short-tailed albatross breeding population size (from target viability assessment) Project: Lake Wales Ridge Objective: By 2013, climbing ferns have been completely eradicated from within 10
miles of all conservation properties Indicator: Number and aerial extent of climbing fern locations (addresses critical
threat) Project: Current River Objective: Achieve and maintain less than 20% in-stream grazing in target creeks. Indicator: Percent of stream subjected to in-stream grazing (addresses critical
threat) Project: Cookson Hills Objective: Protect all caves in the Conservation Area from human intrusion by
2008 Indicators: Evidence of anthropogenic disturbance (addresses critical threat),
Bat diversity/ abundance (from target viability assessment) These five examples illustrate strategy effectiveness approaches that rely on indicators of critical threats and/or target viability. For situations where strategic actions are aimed at underlying causes behind critical threats, consider selecting indicators at key steps along the causal chain that connect the action(s) to the target. For example, consider the following objectives and strategic actions for the Andean Bear target from the Condor Bioreserve in Ecuador where bears are killed due to conflicts with livestock grazing. Goal: Andean Bear / Population size of at least one adult bear per km² of available habitat. Threat: Illegal hunting of Andean Bears Objective: By September 30 2007, 50% of Andean bear conflict hunting has
been reduced over 2000 hunting levels in three critical sites of the CBR:
Oyacachi, Cosanga and Cuyuja. Strategic Actions:
(1) Work with private land owners and communities to create separate land
use zones for cattle grazing and bear conservation areas to reduce the
hunting of Andean bears due to conflicts with cattle grazing, especially in
Oyacachi, Juan Montalvo and Cosanga.
(2) Allocate some ecotourism revenue to an Andean Bear fund to compensate
ranchers for cattle killed by bears. The project team has made the linkage between the actions they are taking and their desired results explicit through the use of a Results Chain diagram (WWF and FOS 2006) as shown in Figure 2. Figure 2: Results chain for the Andean Bear focal target at the Condor Bioreserve project in Columbia
|
The indicators listed below the factors are potential indicators for measuring the results of the actions being taken. The project team needs to decide which indicators are most important and/or feasible to monitor with available resources. If the objective is to cut in half the number of bears killed each year and you suspect the primary motivation is conflict with cattle, it seems you would at least want to keep track of # of bears killed, # of bear attacks on cattle, and some periodic estimate of the bear population itself. The Results Chains tool is very useful for guiding the selection of strategy effectiveness indicators. See Resources and Tools for instructions and examples of completed Results Chains (WWF-FOS 2005). ü The CAP Workbook includes a data entry wizard – the Strategy Identification Wizard – that makes is easy to associate threat-based, target-based, or any other indicators to your stated objectives. The threat summary table includes a utility that makes it easy to link your indicators to specific threats (double-click on threat-threat ranks to open the utility).
2. Determine Status Assessment NeedsYour draft strategy effectiveness indicators likely address most of your critical threats and a subset of the key ecological attributes of your focal targets. You should review the threats and key ecological attributes that are not the focus of your current action plan to identify additional potential status assessment indicators to monitor. Start by reviewing your threat summary table. Are there threats that are not the subject of current actions, perhaps with Medium or Low ranks, that raise sufficient concern to warrant measuring? Undesirable changes in these threats may trigger new conservation action. For example, you may currently consider illegal timber harvesting to be a Low-ranked threat but confidence in the threat rating was weak so you may ask rangers and other project staff to keep records of any newly cut stumps they observe on routine visits through the project area. ü Add additional threat-based status indicators into the CAP Workbook by double-clicking on target-threat ranks in the Threat Summary Table and using the “Add Indicators” utility.
Next, review the indicators from your target viability assessment. The viability assessment process often generates lists of Key Ecological Attributes and Indicators that exceed the capacity of project teams to assess on a regular basis. Thus, it is important to identify the indicators that are most important to regularly measure. Some of the viability indicators will have already been selected for strategy effectiveness purposes and likely represent the highest priority viability indicators for measuring (the monitoring table in the ü CAP Workbook lists all viability indicators and shows which ones are linked to the project’s objectives). Identify key ecological attributes and indicators where you have hypothesized a connection to potentially critical threats but where uncertainty in target status is serving as a barrier to taking action. Improving the understanding of the status of these key ecological attributes and indicators will inform pending conservation action decisions. For example, you may know that periodic flooding is necessary to lead to riparian forest recruitment events but you don’t yet know whether upstream water uses are sufficiently altering the flow regime to prohibit recruitment. Measuring an indicator of riparian tree recruitment will help you determine if strategies affecting upstream water management are needed. It will be less important to regularly measure viability indicators associated with key ecological attributes confidently assigned to Good or Very Good status ratings that are not associated with critical threats. Lower priority ranking may lead to the selection of less costly measurement techniques (qualitative vs. quantitative methods) or less frequent assessment intervals. 3. Review and Refine Draft Indicators and Explore MethodsWhen people think of measuring results, they also often think of complex methods involving quantitative indicators that require specialized skills – for example, mark-recapture population monitoring of an animal population or counts of plants in rectangular quadrats. Methods, however, do not need to be complex or sophisticated and indicators can be quantitative or qualitative. In fact, if you can get the information you need using a simple, inexpensive method, it is far preferable to do this than to choose a complex, expensive method. While the information you gather may be less precise, it may be sufficient for the types of decisions you are making. When planning for measuring results, you need to keep in mind that it should be a relatively small portion of your project budget – a general rule of thumb is about 5-15% of your overall budget (this will vary considerably depending on the project and the actions being taken). If your methods for measuring results are too complex, you will not have enough money to implement actions and measure the results. For example, consider the following alternative indicators and methods for tracking the abundance of an invasive plant population: - Indicator: Patch location and size. Method: Detailed mapping of all patches of the invasive species using a global position system and management of the data within a Geographic Information System.
- Indicator: Population size. Method: Total census of all plants in the population.
- Indicator: Mean density or cover. Method: Quantitative assessments of density or cover in randomly positioned quadrats to estimate the average plant density or average cover with confidence intervals around estimates.
- Indicator: Relative abundance rank. Method: Qualitative estimates of abundance based on wandering transect survey method.
- Indicator: Presence/absence. Method: Quick site visit to determine whether the invasive species is present or not.
All of these indicators and methods are valid, but each varies in its level of effort, cost, and accuracy. You will need to balance the need for greater accuracy and precision with considerations of the risk and uncertainty of anticipated results and resource availability. As you review the list of draft strategy effectiveness and status assessment indicators, consider the following tips for refining indicators and selecting potential methods: - Use existing data sources
- Consider alternative methods
- Pursue locally-based solutions
- Measure surrogate indicators
- Evaluate potential indicators and methods using desired criteria
- Record a brief description of selected methods for each indicator
Use existing data sources
Before you invest time and effort into developing and implementing your methods for measuring results, you should determine if the data you need is available from existing, reliable sources. Assuming these methods meet the criteria for good methods, you should try to use this data rather than spending your project resources on collecting primary data. In some cases, you may not be able to get exactly what you need from secondary sources, but you should evaluate whether what you can get would meet your needs. If so, you should consider modifying your indicator so that you can draw on this existing source. For example, if you have identified the need to measure river flows, you may discover that a government agency has an automated stream flow gauge 10 miles upstream that provides a reliable enough estimate of stream flows within your project area and you may be able to download annual flow data from the internet. You should be careful, however, that your new indicator does truly serve as a good measure of your information need. Good sources of data include ongoing research projects and routine monitoring by scientific institutes, universities or government agencies. Consider alternative methods If existing data sources cannot meet your needs, consider alternative methods before selecting a particular approach. There is typically a wide range of potential methods to assess a given indicator. These methods vary in terms of specific measurement techniques, the use of statistical sampling methods, and the degree to which management treatments are spatially replicated and compared to themselves and/or to untreated controls over time. There are typically many alternative measurement techniques. For example, measures of aerial extent of an ecological system can be measured directly on the ground by pacing or with measuring tapes, tracing polygons on aerial photographs and estimating cover with a grid-overlay, walking the perimeter of patches with a global positioning system, or collecting and analyzing geo-referenced satellite imagery. The size of animal populations can be assessed via a variety of methods including asking local villagers to report the number of animals they have seen or heard recently, conducting a complete census, using relative indices of abundance (e.g., track or scat counts), using mark-recapture techniques, or using distance sampling methods. Plant populations can be assessed via total counts, rank order estimates of abundance (e.g., 1-100, 101-1000, 1000-10,000, > 10,000 individuals), demographic techniques, simple or nested frequency measurements, biomass estimates, or estimates of plant cover (e.g., ocular estimates, point-intercept, line-intercept, or direct cover measurements). See Elzinga et al. 2001 in the Resources and Tools section for an overview of many common measurement techniques for plant and animal populations. You need to decide whether or not your methods will involve statistical sampling procedures. Sampling is the act or process of selecting a part of something with the intent of showing the quality, style, or nature of the whole (e.g., counts of plants gathered within randomly positioning quadrats used to estimate the overall population size with 95% confidence intervals). Many monitoring methods do not require sampling procedures. Sometimes, you can count or measure all individuals within a population. Other times, you may select qualitative approaches such as subjectively positioned permanent photo-points. If you do elect to use sampling procedures, there are many sampling decisions that must be considered including the selection of specific sampling units (e.g. quadrats, points, line transects), the size and shape of the sampling units, the arrangement of sampling units within the area of interest (e.g., simple random sampling, systematic sampling, stratified random sampling), whether sampling unit locations should be permanently marked or temporary, and the number of sampling units to sample. See the Resources and Tools section of this chapter for several good books to guide the selection of efficient sampling designs. Assessing the results of specific conservation actions with a high level of scientific certainty requires an experimental research design with adequate levels of replication and controls. Although it is desirable to achieve strong scientific inferences regarding the consequences of your actions, competing demands on limited resources typically limit the opportunity for full field experimentation to assess the impacts of most conservation actions. You should pursue more rigorous experimental research designs when the uncertainty or risk associated with your actions warrants this higher level of scientific certainty. Even without a fully replicated experiment design, you can markedly improve the probability of learning whether actions being implemented are leading to the desired results by measuring a combination of indicators located at different positions along a results chain (see WWF-FOS 2006 for more information on Results Chains). In many cases you or your colleagues will be aware of the range of methods available. If this is not the case you can learn about various methods by talking to experienced people, reviewing documents or manuals on the subject, taking courses, or scanning through examples of monitoring plans available through shared information systems such as TNC’s Conservation Project Database (http://conpro.tnc.org/). Explore the use of locally-based monitoring methods
Locally-based monitoring methods embraces a broad range of approaches, from censuses by local rangers, inventories by citizen scientists, or using economic or resource use/extraction data from the very actors that may be creating threats. There are many examples of manuals that guide the establishment of local volunteer monitoring programs. The U.S. Environmental Protection Agency (EPA)’s has published manuals for volunteer monitoring of lakes, streams, estuaries, and wetlands and the U.S. Forest Service’s 2006 guide on Broadening Participation in Biological Monitoring: Handbook for Scientists and Managers. Locally-based options may be particularly appropriate when local communities are actively using the natural resources within the project area. In some natural resource use cases, you can find an indicator that simultaneously informs threat status and target viability status. Consider a situation where you are concerned about the potential downward trend in the population of some fished species. You could focus strictly on target viability measures and do underwater surveys where counts of fish by estimated size class are made for a fixed duration of time at numerous monitoring stations. Alternatively, you could take a completely threat-based approach where you track the number of fishing boats working in the area, number of fishing trips, or the number of people employed as fisherman. However, you could simultaneously track the threat and the target if you work with the fisherman to keep and share good catch records. Measuring trends in the total weight or volume of the catch, the size distribution (or simply the average size) of fish caught, and the effort required to obtain their catch (number of hours spent fishing) could yield valuable threat and target status information. For example, if it is taking fisherman more time to catch the same quantity of fish and the average fish size is steadily declining, you have reason to be concerned about the status of the fish population. Alternatively, perhaps a large marine reserve has been established nearby and more and larger fish are being caught with less effort providing an indicator of stable or increasing population size. In either case, as long as you believe your catch records are accurate and complete, you may not need separate underwater fish counts to assess the status of the fish population. Similarly, if you are concerned about the over-harvesting of non-timber forest products, the most cost-effective assessment approach might be to measure harvest levels rather than directly measuring the population of plants in the forest. For example, if in year one it takes a villager an average of four hours to fill a basket with masuatake mushrooms and in year three it takes an average of 8 hours to collect the same volume of mushrooms, there is reason to be concerned about the status of the masuatake mushroom population, even without separate counts of mushrooms in the field. See Resources and Tools at the end of this chapter for a set of 15 locally-based monitoring case studies (Danielsen et al. 2005). Consider measuring proxy indicators In some cases you cannot collect the information you need directly because data are too difficult, too expensive, or culturally inappropriate to acquire. In these cases you should consider measuring proxy or surrogate indicators. For example, you might use the number of orangutan nests as a proxy for the orangutan population size. Or if you are working to control a non-native plant species by having volunteer work crews annually pull out all established plants, you may rely on the number of person-hours it takes each year to control the population as a proxy measure of the abundance of the non-native species. A steady decline in the annual control effort needed to treat the population suggests a reduction in the abundance of the non-native species. Evaluate potential indicators and methods using desired criteria
As you consider alternative indicators and methods, you should review and apply the criteria introduced in the “Defining Measuring Results” section, repeated here: Criteria for Indicators: - Measurable- Able to be recorded and analyzed in quantitative or in discreet qualitative terms.
- Clear- Presented or described in such a way that its meaning will be the same to all people.
- Sensitive- Changing proportionately in response to actual changes in the condition or item being measured.
Criteria for Methods: - Accurate- Gives minimal or no error.
- Reliable- Results obtained using the methods are consistently repeatable.
- Cost-Effective- Not overly expensive for the data the method yields or for the resources available to the project.
- Feasible- Project team has people who can use the method, as well as the material and financial resources to use the method.
- Appropriate- Appropriate to the environmental, cultural, and political context of the project.
Record a brief description of the method associated with each indicator
The proposed method should be briefly summarized in the monitoring plan. If the method is not well known to those doing the measurements, it may be necessary to define and describe the method more fully in a separate document (see Slapcinsky & Gordon 2003 and Slapcinsky et al. 2006 in the Resources and Tools section for examples of monitoring plans for quantitative and qualitative monitoring from TNC’s Florida Program). üThe monitoring table in the CAP workbook includes a field to record a brief description of the method and also includes a separate field to record the citation and location of a more detailed monitoring plan. 4. Set Priority Status for all IndicatorsYou have now developed the basic elements of a plan for measuring results by selecting strategy effectiveness indicators, status assessment indicators, and providing a brief description of the methods for measuring each indicator in a draft monitoring table. You should have also linked all indicators to objectives, targets, key ecological attributes, and threats. The previous steps for identifying indicators and methods incorporated many priority-setting criteria. We have emphasized the importance of covering the project’s strategy effectiveness needs before exploring the status assessment needs. We have placed higher priority on measuring high-ranked threats and the key ecological attributes of greatest concern. We have suggested ways of reducing the overall cost for measuring results, and thereby allow you to cover more of your measures needs, by using existing monitoring data collection efforts, engaging local participants in the data collection efforts, and considering qualitative approaches. Still, it may not be possible to implement all identified indicators in the early phases of a conservation project. Consider assigning a priority status to each indicator to help ensure that the most critical indicators are measured first. Within the ü CAP Workbook, each indicator can be assigned a Very High, High, Medium or Low status within the Monitoring worksheet. 5. Complete Monitoring TableYou are now ready to complete more details in the monitoring table to set the stage for implementing the plan for measuring results. The additional detail includes specifying the following categories of information for each priority indicator: - When (timeframe & frequency of data collection)
- Where (location of data collection)
- Who (people responsible for data collection, data management, and analysis)
- Cost (of monitoring the indicator)
- Funding source
- Current indicator status (measurement value and date)
- Complete monitoring plan (reference and date)
- Summary report (reference and date)
- Implementation status
Guidance associated with completing the monitoring table is covered as part of the next chapter, Step 8: Develop Work Plans. |