Get 20M+ Full-Text Papers For Less Than $1.50/day. Subscribe now for You or Your Team.

Learn More →

Mainstreaming Impact Evaluation in Nature Conservation

Mainstreaming Impact Evaluation in Nature Conservation Biodiversity; conservation policy; impact An important part of conservation practice is the empirical evaluation of pro- evaluation; payment for environmental gram and policy impacts. Understanding why conservation programs succeed services; protected areas. or fail is essential for designing cost-effective initiatives and for improving the Correspondence livelihoods of natural resource users. The evidence we seek can be generated Jan Borner, ¨ Center for Development Research, with modern impact evaluation designs. Such designs measure causal effects University of Bonn, Bonn, Germany. of specific interventions by comparing outcomes with the interventions to out- Tel: +49-228-73-1873; comes in credible counterfactual scenarios. Good designs also identify the con- Fax: +49-228-73-1869. ditions under which the causal effect arises. Despite a critical need for empirical E-mail: jborner@uni-bonn.de evidence, conservation science has been slow to adopt these impact evaluation Received designs. We identify reasons for the slow rate of adoption and provide sugges- 13 September 2014 tions for mainstreaming impact evaluation in nature conservation. Accepted 10 April 2015 Editor William Sutherland doi: 10.1111/conl.12180 Impact evaluation has developed into a research disci- Introduction pline with multiple fields of application including health, Conservation science is only slowly beginning to build education, and development (White 2009). Our notion a body of evidence on the impact of conservation poli- of impact evaluation goes beyond monitoring program cies (Ferraro & Pattanayak 2006; Fisher et al. 2014). inputs, outputs, or indicators over time. It measures the Many compelling reasons motivate impact evaluations of causal effect of a specific policy, program or intervention conservation policy instruments. Organizations want to vis-a-vis ` a credible counterfactual scenario and seeks to know where to invest scarce resources, while govern- understand the conditions under which this effect arises ments and donors seek tangible outcomes. Evidence of (Ferraro & Hanauer 2014). In a comprehensive impact why conservation initiatives succeed or fail is also essen- evaluation, evaluators will rule out alternative or rival tial for designing cost-effective programs and improving explanations of program outcomes (Ferraro 2009). One the livelihoods of natural resource users (Sutherland et al. might also examine past outcomes to forecast the poten- 2004; Cook et al. 2010). In this article, we propose steps tial impact of future interventions (Pfaff et al. 2009). To toward mainstreaming and improving conservation pol- obtain these insights, impact evaluations must be more icy impact evaluation. than abstract quantitative evaluations but rather build 58 Conservation Letters, January/February 2016, 9(1), 58–64 Copyright and Photocopying: 2015 The Authors. Conservation Letters published by Wiley Periodicals, Inc. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. K. Baylis et al. Mainstreaming impact evaluation on qualitative theories of change which help identify the wintering grounds, often distributed across multi- conditions in which the desired impacts arise (Morgan & ple ecosystems and political administrations (Brower Winship 2007). 1995; Naidoo et al. 2014). In the presence of multi- We are not the first to make the above points. Sev- ple objectives at multiple scales, it also becomes more eral recent papers call for improving the quality of difficult to articulate clear theories of change and impact evaluation in nature conservation (Ferraro & empirical strategies for impact evaluation. Different Pattanayak 2006; Miteva et al. 2012; Pullin 2012; Fisher choices about scale will also inhibit comparable repli- et al. 2014). Despite these calls, conservation science still cations. lags behind health, education, and development policy in (2) Spatial spillovers. While many fields can ignore the adopting best practices in impact evaluation (Banerjee & spatial component of an impact evaluation, con- Duflo 2009). Few studies meet even the basic standards servation simply cannot. Space is an essential part of an impact evaluation such as considering before and of ecological processes: water flows, pollution emis- after conditions, including control groups, accounting for sions, species migration, deforestation, and disper- confounding factors, or systematically ruling out rival hy- sal. Therefore, to assess the impact of conservation potheses (Bowler et al. 2012; Samii et al. 2014). policies, one must account for the appropriate spatial In contrast to earlier essays on this subject, we ex- scale. Yet, even when the appropriate spatial scale plore the reasons why nature conservation policy has is well known, measuring the net impacts of an in- been slow to adopt more rigorous impact evaluation de- tervention is complicated by spatial spillovers. These signs. The reasons are not trivial and the solutions are not spillovers can be a result of ecological process, but simple. We characterize the current barriers and propose can also result from behavioral responses, such as elements of a strategy that may build a systematic body when restricting access to resources in one area in- of evidence on the effectiveness of conservation initia- duces a rise in extractive activity elsewhere, in what tives. Our arguments are based on discussions from the is referred to as “leakage” (Ostwald & Henders 2014). workshop “Evaluating Forest Conservation Initiatives: New Spillovers not only affect net impacts but can also Tools and Policy Needs” organized in Barcelona, Spain in bias impact estimation when they influence nontar- December 2013. get areas that were intended to serve as control ob- servations. (3) Confounding factors. Many biophysical, behavioral, Challenges for impact evaluation in and institutional factors affect both where conser- conservation science vation initiatives take place and the outcomes we measure. Imagine that the survival of a particular Conservation programs have features that, while not species depends on forest habitat under threat by unique to conservation, translate into specific challenges logging pressures. Policy makers respond by creating for impact evaluation. a new protected area, but the location and bound- (1) Multiple outcomes and scales. Conservation interven- aries of protection are developed in consultation with tions often strive to achieve multiple objectives at local municipalities who prioritize remote areas far from human settlements. An impact evaluation that multiple scales. For instance, ensuring viable species was to compare conservation outcomes inside this populations while protecting habitat; or maintaining ecosystem integrity while increasing the provision of park with conservation outcomes outside the park ecosystem services for human populations. “Coben- might erroneously find that the park was highly efits” may be relevant in other contexts, but in con- successful if areas with low deforestation risk were servation, cobenefits are often central to program protected, while areas with easier access, closer to success. The backlash against the Reduced Emissions human settlement, and high deforestation risk were from Deforestation and Forest Degradation (REDD) left unprotected (Joppa & Pfaff 2010). Assume fur- program, for initially focusing only on carbon cap- ther that timber values increased after the park was ture as a singular metric, illustrates the distaste for created, resulting in a generalized spike in logging. single policy objectives in a multiple-output setting A before-after comparison might lead to the erro- (Corbera et al. 2010). Furthermore, ecosystems are neous conclusion that the park was unsuccessful. complex systems with nonlinear dynamics at vari- In both cases, these approaches fail to address the ous spatial and temporal scales (Fisher et al. 2009; confounding factors affecting protected area place- Koch et al. 2009). Such complexity raises practical ment and outcome. These confounding factors must hurdles. For example, the conservation of migratory be accounted for in nonexperimental evaluations. To species requires management in both breeding and do so, evaluators need to draw on expertise from Conservation Letters, January/February 2016, 9(1), 58–64 Copyright and Photocopying: 2015 The Authors. Conservation Letters published by Wiley Periodicals, Inc. 59 Mainstreaming impact evaluation K. Baylis et al. various disciplines and on-the-ground knowledge. In results in a “conservation policy research cycle,” where cases in which confounders are not easily observable, the knowledge base is continuously updated as new evaluators often use instrumental variables – vari- evidence emerges. First, to refine theories of change, ables that only affect the outcome through their ef- researchers need to cross epistemological divides and inte- fects on the probability of participating in a program grate qualitative and quantitative approaches (Margoluis et al. (e.g., weather conditions or other shocks like natural 2009; Agrawal & Chhatre 2011). Qualitative understand- disasters). Finding such variables in the conservation ing helps contextualize quantitative treatment effect es- context is difficult because they often affect conser- timates and quantitative methods can inform qualitative vation outcomes directly. research design and theory development. As an exam- (4) Randomization’s limits. Conservation science has been ple, consider the use of quantitative data to inform the slow to adopt randomized controlled trials (RCT). selection of locations for in-depth qualitative analysis, ei- Notable exceptions include Ferraro et al. (2011), ther by targeting outliers or more representative sampling Jack (2013), Samii et al. (2014), and experiments groups (Roe et al. 2013). Multidisciplinary perspectives in habitat and invasive species management studies should not only inform theories of change and related (Sutherland et al. 2004). Practical and ethical consid- intervention designs, they can also help to develop more erations often limit the successful use of RCTs. Ran- appropriate evaluation strategies (White 2009). domization is not viable with small sample sizes or The second implication relates to choosing the appropri- low replication, and research designs must be ad- ate scale of analysis. In conservation practice, the unit of justed accordingly. However, it is difficult to obtain analysis, spatial scale, and outcome variable is not al- large sample sizes or replicate if a single interven- ways readily apparent. As a starting point, the analyti- tion covers a large geographic area. For example, cal scale should be motivated by the theory of change. to randomize a program to preserve water quality And, yet the decision-making unit and the resource gov- with acceptable statistical power, one might have to ernance regime in conservation programs can be neb- treat hundreds of watersheds. RCTs also rely on the ulous: ecosystems are comanaged by private owners, “stable unit value treatment assumption,” which im- collectives, communities, or state agents. The issue is fur- plies outcomes in one observation are not affected ther complicated because the natural, social, and med- by the treatment status of another. This assumption ical sciences differ in their view on what constitutes the may not hold in the presence of spatial spillovers: “right” scale, unit of analysis, and appropriate sample size. where the outcome of one parcel affects neighboring Since social and ecological processes operate at multiple parcels. Last, it may also be politically untenable or scales, any single choice of scale will inevitably fail to cap- unethical to randomly distribute restrictive conserva- ture certain dynamics. Researchers often compare con- tion regulations. Conservation policy evaluation will servation outcomes across a landscape by dividing their thus have to rely on paired research designs and in- study areas into a uniform grid. However, uniform grids novative quasi-experimental approaches, such as re- inevitably combine multiple ecosystem types, governance gression discontinuity design and synthetic control regimes, or property owners. Thus, the choice of analyt- analysis (e.g., Abadie et al. 2014). ical unit, as well as geographical (e.g., valley, watershed, (5) Small initiatives. Large-scale and generously funded landscape, ecosystem) and administrative (e.g., commu- pilot initiatives are rare in the conservation sector, nity, municipality, county, region) scale is challenged by constraining even those firmly committed to mea- methodological constraints. The solution is not found in suring impact. Innovative program designs are often selecting fine-grained analytical units because high res- developed by small organizations that integrate mul- olutions will generate spatial correlations that bias re- tiple funding streams and gradually develop their in- sults. Conversely, coarse resolutions may fail to capture tervention design through years of experience. For local processes and inhibit the identification of appropri- these organizations, it may not be feasible to embark ate controls. Selecting the appropriate scale can reduce on impact evaluation by themselves. Either outside the unobserved confounding factors, while using an inap- support or some critical mass of similar interventions propriate scale can exacerbate the effects of unobserved is probably needed to carry out full-scale evaluation confounding factors. Where the appropriate scale is designs. unknown, impact evaluations may use hierarchical mod- els or replicate the analysis at multiple scales to evalu- ate how sensitive results are to the choice of scale (e.g., Implications for conservation policy and Avelino et al. 2015; Borner ¨ et al. 2015; Costedoat et al. science 2015). Several implications arise for the design of impact The third implication relates to incorporating spillover evaluations and the effective integration of evaluation effects into the research design, including leakage, spatial 60 Conservation Letters, January/February 2016, 9(1), 58–64 Copyright and Photocopying: 2015 The Authors. Conservation Letters published by Wiley Periodicals, Inc. K. Baylis et al. Mainstreaming impact evaluation autocorrelation, and peer effects. Several recent papers complexity of the research design. It requires even more explore the effect of conservation policies on conserva- elaborate theories and more untestable (and often less tion outcomes in neighboring areas. Some report that an credible) assumptions than are required to estimate un- increase in protection in one area displaces deforestation conditional effects. Thus, even in RCT, estimates of het- activities to other areas (Oliveira et al. 2007; Meyfroidt & erogeneous treatment effects are considered much less Lambin 2009). Others find a “halo effect” whereby ar- credible than unconditional effects unless incorporated eas adjacent to protected areas are better protected than directly in the experimental design (Ferraro & Hanauer one might expect (Honey-Roses ´ et al. 2011; Gaveau et al. 2014). 2012; Robalino & Pfaff 2012). Ideally, theories and related evaluation methods would address both potential sources of spillovers, behavioral and mechanistic, since the exis- Moving forward tence of either should be part of the estimated treatment effect. Ignoring spillover effects will bias estimates of pro- Building a body of evidence on conservation policy ef- gram impact. fectiveness will require greater collaboration between Fourth, while randomization might not be possible for researchers and conservation managers akin to the long- programs that require large, contiguous areas, some con- standing partnership between medical scholars and clini- servation instruments, such as Payments for Ecosys- cians. The evidence base we advocate for is a global public tem Services (PES) or community-based programs are good. Therefore, it is not surprising that it has been diffi- amenable to randomization, particularly if the desired en- cult to muster the resources necessary for building a solid vironmental outcomes are local. For example, incentive- evidence base. Unless practitioners are strongly encour- based contracts are being randomly allocated in the aged by donors, we will end up with an underprovision mountains of Bolivia (Asquith et al. 2008; Jones 2012) of evidence; i.e., the status quo. and Uganda (Hatanga 2014). Where feasible and ethical, Systematic reviews and systematic maps (or evi- randomizing treatment can help researchers address po- dence gap maps) are a useful tool to synthesize sci- tential confounding factors by ensuring they are not asso- entific results and identify shortfalls for policy makers ciated with treatment. Randomization may also be used (Dicks et al. 2014). The Initiative for Impact Evaluation when a program is thought to work, and program man- (www.3ieimpact.org) and the Collaboration for Envi- agers would like to test variations of the program or spe- ronmental Evidence (www.environmentalevidence.org) cific aspects of its mechanism to identify why and how have recently published systematic reviews of the effect the program produces the desired results. It also might of protected areas, payment for ecosystem services and be easier to randomize over enforcement than over the aspects of forest management on various human welfare, placement of protected areas, or it might be possible to habitat and species preservation outcomes. In all of these randomize over the type of PES contract needed to induce reviews, authors point to the limited or fragmented ev- changes in household behavior. Furthermore, if a coun- idence of the effect of these various policy instruments try is interested in introducing a nation-wide conserva- (Bowler et al. 2010; Geldmann et al. 2013; Pullin et al. tion effort, randomizing over location might be feasible. 2013; Samii et al. 2014). Protected areas have arguably re- However, any randomized intervention would require an ceived the most attention with the reviews analyzing 86 important investment in communicating its purpose and articles on habitat and species outcomes and 306 articles the targeting rationale because, as noted earlier, such ap- on perceptions of PAs and 79 on welfare impacts. Never- proaches may entail political and ethical challenges. Fur- theless, large gaps remain. Even for protected areas, our thermore, randomization needs to be part of a broader understanding of spillover and heterogeneous treatment evaluation strategy that incorporates qualitative work to effects on environmental and socioeconomic indicators is explore the causal chain. still limited (Geldman et al. 2013; Pullin et al. 2013). Em- Finally, impact evaluation in conservation should be pirical evidence is even more sparse on the effectiveness sensitive to heterogeneous outcomes (Alix-Garcia et al. 2012; and implementation costs of other large- and small-scale Pfaff & Robalino 2012; Ferraro & Miranda 2013). Con- conservation policy instruments, such as forest law en- servation policies and programs affect a variety of so- forcement, PES schemes, ecocertification, and Integrated cial actors under varying biophysical conditions. Moving Conservation and Development approaches that domi- beyond the average effects of an intervention, conser- nate public and private REDD+ initiatives (Blom et al. vation planners need to know where and for whom it 2010; Lambin et al. 2014). Furthermore, we need to worked (Deaton 2009). Estimating heterogeneous treat- know how these policies compare in their effect on hu- ment effects and uncovering causal mechanisms behind man and environmental outcomes, and how these instru- average treatment effects is difficult and can increase the ments work in policy mixes (Barton et al. 2013). Conservation Letters, January/February 2016, 9(1), 58–64 Copyright and Photocopying: 2015 The Authors. Conservation Letters published by Wiley Periodicals, Inc. 61 Mainstreaming impact evaluation K. Baylis et al. Mainstreaming impact evaluation in conservation will Xie, and S. L. Zhou. Finally, we thank three anonymous require partnerships between scientists and program im- reviewers who improved the manuscript. plementers during the design phase to: (1) clarify pro- References gram objectives, possibly with a modification in design; (2) identify a theory of change, counterfactual groups, Abadie, A., Diamond, A. & Hainmueller, J. (2014). and testable hypotheses; and (3) define performance in- Comparative politics and the synthetic control method. Am. dicators and data collection protocols. Wherever feasi- J. Polit. Sci., doi:10.1111/ajps.12116. Agrawal, A. & Chhatre, A. (2011). Strengthening causal ble, a randomized program design may reduce rather inference through qualitative analysis of regression than increase the costs of impact evaluation, particularly residuals: explaining forest governance in the Indian in subnational or single-project pilot interventions. Such Himalaya. Environ. Plann. A, 43, 328-346. partnerships should be maintained over time to facilitate Alix-Garcia, J.M., Shapiro, E.N. & Sims, K.R.E. (2012). Forest continuous feedbacks between evaluation, design proto- conservation and slippage: evidence from Mexico’s cols and criteria, and implementation practice, which all national payments for Ecosystem Services Program. Land should be flexible, adaptive, and responsive to assess- Econ., 88, 613-638. ment outcomes (Sims et al. 2014). Donors could support Asquith, N.M., Teresa, M. & Wunder, S. (2008). Selling two this process by conditioning funding, including perfor- environmental services: in-kind payments for bird habitat mance bonuses on well-designed impact evaluation, and and watershed protection in Los Negros, Bolivia. Ecol. Econ., collaborating with researchers on defining priorities for 65, 675-684. focused and carefully designed systematic reviews (CEE Avelino, A.F.T., Baylis, K. & Honey-Roses, ´ J. (2015). 2013). Most importantly, impact evaluation needs long- Goldilocks and the grid: selecting scale when evaluating term support to build a strategic global evidence base conservation polities. University of Illinois Working Paper. for conservation policies (Keene & Pullin 2011). Such Banerjee, A.V. & Duflo, E. (2009). The experimental growing collaboration and mutual understanding should approaches to development economics. Annu. Rev. Econ., 1, slowly preclude awarding funding on the basis of exag- 151-178. gerated ex ante claims of conservation potential, since Barton, D.N., Blumentrath, S. & Rusch, G. (2013). this is counterproductive to building a solid body of ev- Policyscape—a spatially explicit evaluation of voluntary idence. Conservation at the scale envisaged by interna- conservation in a policy mix for biodiversity conservation tional policy initiatives, such as REDD+, clearly stands in Norway. Soc. Natur. Resour., 26, 1185-1201. to benefit from a solid body of evidence on what works, Blom, B., Sunderland, T. & Murdiyarso, D. (2010). Getting what does not, where and why. REDD to work locally: lessons learned from integrated conservation and development projects. Environ. Sci. & Pol., 13, 164-172. Borner, ¨ J., Kis-Katos, K., Hargrave, J. & Konig, ¨ K. (2015). Acknowledgments Post-crackdown effectiveness of field-based enforcement in the Brazilian Amazon. PLoS One, 10(4): e0121544. The perspectives put forward in this article are based Bowler, D., Buyung-Ali, L., Healey, J.R., Jones, J.P.G., on discussions among scientists and practitioners who Knight, T. & Pullin, A.S. (2010). The evidence base for attended the workshop “Evaluating Forest Conserva- community forest management as a mechanism for tion Initiatives: New Tools and Policy Needs” on 10–12 supplying global environmental benefits and improving December 2013, in Barcelona, Spain. We are grateful for local welfare. Environmental Evidence: support from the European Association of Environmental www.environmentalevidence.org/SR48.html. Accessed and Resource Economists, the Robert Bosch Foundation, May 2015. the Autonomous University of Barcelona, the Center for Bowler, D.E., Buyung-Ali, L.M., Healey, J.R., Jones, J.P., International Forestry Research, and the ERA-Net Bio- Knight, T.M. & Pullin, A.S. (2012). Does community forest divERsA funded INVALUABLE project with the French management provide global environmental benefits and national funder Agence Nationale de la Recherche. The improve local welfare? Front. Ecol. Environ., 10, 29-36. INVALUABLE project generously covered publication Brower, L.P. (1995). Understanding and misunderstanding fees. We also thank the workshop participants: A. An- the migration of the monarch butterfly (Nymphalidae) in gelsen, R. Arriagada, N. Asquith, M. Bottrill, J. Bush, North America: 1857–1995. J. Lepid. Soc., 49, 304-385. E. Cisneros, S. Costedoat, P. Davies, S. Desbureaux, L. CEE. (2013). Guidelines for systematic review and evidence Glew,E.Gorriz, ´ B. Hedden-Dunkhorst, N. Hogarth, C. synthesis in environmental management. Collaboration for Loucks, A. Leonardi, B. Macura, A. Martin, M. Masozera, environmental evidence. http://www.environmental- D. Miteva, S. Pagiola, V. Rodr´ıguez Osuna, J. Schielein, evidence.org/wp-content/uploads/2014/06/Review- P. Shah, E. Sills, J.M. Torres, E. Varela, G. Le Velly, C. guidelines-version-4.2-final.pdf. Accessed May 2015. 62 Conservation Letters, January/February 2016, 9(1), 58–64 Copyright and Photocopying: 2015 The Authors. Conservation Letters published by Wiley Periodicals, Inc. K. Baylis et al. Mainstreaming impact evaluation Cook, C.N., Hockings, M. & Carter, R. (2010). Conservation in iied.org/random-trials-uganda-show-if-payments-for- the dark? The information used to support management ecosystems-services-really-work. Accessed May 2015. decisions. Front. Ecol. Environ., 8, 181-186. Honey-Roses, ´ J., Baylis, K. & Ramirez, M.I. (2011). A spatially Corbera, E., Estrada, M. & Brown, K. (2010). Reducing explicit estimate of avoided forest loss. Conserv. Biol., 25, greenhouse gas emissions from deforestation and forest 1032-1043. degradation in developing countries: revisiting the Jack, B.K. (2013). Private information and the allocation of assumptions. Climatic Change, 100, 355-388. land use subsidies in Malawi. Appl. Econ., 5, 113-135. Costedoat, S., Corbera, E., Ezzine-de-Blas, D., Baylis, K., Jones, J. (2012). Can paying for global ecosystem services Honey-Roses, ´ J. & Castillo-Santiago, M.A. (2015). How reduce poverty? Funded by Ecosystem Services for Poverty effective are biodiversity payments in Mexico? PloS ONE Alleviation Project, http://www.espa.ac.uk/projects/espa- 10(3), e0119881, doi 10.1371/journal.pone.0119881. 2012-grants. Accessed May 2015. Deaton, A.S. (2009). Instruments of development: Joppa, L. & Pfaff, A. (2010). Reassessing the forest impacts of randomization in the tropics and the search for the elusive protection: the challenge of nonrandom location and a keys to economic development. NBER Working Paper corrective method. Ann. NY Acad. Sci., 1185, 135-149. 14690. Keene, M. & Pullin, A.S. (2011). Realizing an effectiveness Dicks, L.V., Walsh, J.C. & Sutherland, W.J. (2014). Organising revolution in environmental management. J. Environ. evidence for environmental management decisions: a “4S” Manage., 92, 2130-2135. hierarchy. Trends Ecol. Evol., 29, 607-613. Koch, E.W., Barbier, E.B., Silliman, B.R., et al. (2009). Ferraro, P.J. (2009). Counterfactual thinking and impact Non-linearity in ecosystem services: temporal and spatial evaluation in environmental policy. New Directions for variability in coastal protection. Front. Ecol. Environ., 7, Evaluation, 122, 75-84. 29-37. Ferraro, P.J. & Hanauer, M.M. (2014). Advances in measuring Lambin, E.F., Meyfroidt, P., Rueda, X., et al. (2014). the environmental and social impacts of environmental Effectiveness and synergies of policy instruments for land programs. Annu. Rev. Environ. Resour., 39, 495-517. use governance in tropical regions. Global Environ. Chang., Ferraro, P.J. & Miranda, J.J. (2013). Heterogeneous treatment 28, 129-140. effects and mechanisms in information-based Margoluis, R., Stem, C., Salafsky, N. & Brown, M. (2009). environmental policies: evidence from a large-scale field Design alternatives for evaluating the impact of experiment. Resour. Energy. Econ., 35, 356-379. conservation projects. New Directions for Evaluation, 122, Ferraro, P.J. & Pattanayak, S.K. (2006). Money for nothing? 85-96. A call for empirical evaluation of biodiversity conservation Meyfroidt, P. & Lambin, E.F. (2009). Forest transition in investments. PLoS Biol., 4, e105. Vietnam and displacement of deforestation abroad. P. Natl. Ferraro, P.J., Miranda, J.J. & Price, M.K. (2011). The Acad. Sci. USA, 106, 16139-16144. persistence of treatment effects with norm-based policy Miteva, D.A., Pattanayak, S.K. & Ferraro, P.J. (2012). instruments: evidence from a randomized environmental Evaluation of biodiversity policy instruments: what works policy experiment. Am. Econ. Rev., 101, 318-322. and what doesn’t? Oxford Rev. Econ. Pol., 28, 69-92. Fisher, B., Turner, R.K. & Morling, P. (2009). Defining and Morgan, S.L. & Winship, C. (2007). Counterfactuals and causal classifying ecosystem services for decision making. Ecol. inference: methods and principles for social research.Cambridge Econ., 68, 643-653. University Press, Cambridge, UK. Fisher, B., Balmford, A., Ferraro, P.J., et al. (2014). Moving Naidoo, R., Chase, M.J., Beytell, P., et al. (2014). A newly Rio forward and avoiding 10 more years with little discovered wildlife migration in Namibia and Botswana is evidence for effective conservation policy. Conserv. Biol., 28, the longest in Africa. Oryx, 1-9, available from: 880-882. http://dx.doi.org/10.1017/S0030605314000222. Gaveau, D.L.A., Curran, L.M., Paoli, G.D., Carlson, K.M., doi:10.1017/S0030605314000222. Wells, P. & Ratnasari, D. (2012). Examining protected area Oliveira, P.J.C., Asner, G.P., Knapp, et al. (2007). Land-use effectiveness in Sumatra: importance of regulations allocation protects the Peruvian Amazon. Science, 317, governing unprotected lands. Conserv. Lett., 5, 142-148. 1233-1236. Geldmann, J., Barnes, M., Coad, L., Craigie, I., Hockings, M. Ostwald, M. & Henders, S. (2014). Making two parallel & Burgess, N. (2013). Effectiveness of terrestrial protected land-use sector debates meet: carbon leakage and indirect areas in reducing biodiversity and habitat loss. Environ. land-use change. Land Use Policy, 36, 533-542. Evid., www.environmentalevidence.org/ SR10007. Pfaff, A. & Robalino, J. (2012). Protecting forests, biodiversity, Accessed May 2015. and the climate: predicting policy impact to improve policy Hatanga, P. (2014). Random trials in Uganda to show if choice. Oxford Rev. Econ. Pol., 28, 164-179. payments for ecosystem services really work. International Pfaff, A., Robalino, J., Sanchez-Azofeifa, G.A., Andam, K.S., Institute for Environment and Development. http://www. Ferraro, P.J. (2009). Park location affects forest protection: Conservation Letters, January/February 2016, 9(1), 58–64 Copyright and Photocopying: 2015 The Authors. Conservation Letters published by Wiley Periodicals, Inc. 63 Mainstreaming impact evaluation K. Baylis et al. land characteristics cause differences in park impacts across (2014). Effects of Decentralized Forest Management (DFM) Costa Rica. BEJ Econ. Anal. Pol., 9, Article 5. on Deforestation and Poverty in Low and Middle Income Pullin, A.S. (2012). Realising the potential of environmental Countries: a systematic review. CEE 13-015a. Collabo- data: a call for systematic review and evidence synthesis in ration for Environmental Evidence. http://www. environmental management. Environ. Evid., 1:2, 1-3. environmentalevidence.org/wp-content/uploads/ Pullin, A.S., Bangpan, M., Dalrymple, S., et al. (2013). Human 2015/01/Samii DFM Review-formatted-for-CEE.pdf. well-being impacts of terrestrial protected areas. Environ. Accessed May 2015. Evid., 2:19, 1-41. Sims, K.R.E., Alix-Garcia, J.M., Shapiro-Garza, E., et al. Robalino, J.A. & Pfaff, A. (2012). Contagious development: (2014). Improving environmental and social targeting neighbor interactions in deforestation. J. Dev. Econ., 97, through adaptive management in Mexico’s payments for 427-436. hydrological services program. Conserv. Biol., 28, Roe, D., Grieg-Gran, M. & Mohammed, E.Y. (2013). 1151-1159. Assessing the social impacts of conservation Sutherland, W.J., Pullin, A.S., Dolman, P.M. & Knight, T.M. policies: rigour versus practicality. IIED Briefing (2004). The need for evidence-based conservation. Trends Papers. International Institute for Environment and Ecol. Evol., 19, 305-308. Development. White, H. (2009). Theory-based impact evaluation: principles Samii, C., Lisiecki, M., Kulkarni, P., Paler, L. & Chavis, L. and practice. J. Dev. Eff., 1, 271-284. 64 Conservation Letters, January/February 2016, 9(1), 58–64 Copyright and Photocopying: 2015 The Authors. Conservation Letters published by Wiley Periodicals, Inc. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Conservation Letters Wiley

Loading next page...
 
/lp/wiley/mainstreaming-impact-evaluation-in-nature-conservation-FJPHv8UyWD

References (54)

Publisher
Wiley
Copyright
Copyright © 2016 Wiley Periodicals, Inc.
ISSN
1755-263X
eISSN
1755-263X
DOI
10.1111/conl.12180
Publisher site
See Article on Publisher Site

Abstract

Biodiversity; conservation policy; impact An important part of conservation practice is the empirical evaluation of pro- evaluation; payment for environmental gram and policy impacts. Understanding why conservation programs succeed services; protected areas. or fail is essential for designing cost-effective initiatives and for improving the Correspondence livelihoods of natural resource users. The evidence we seek can be generated Jan Borner, ¨ Center for Development Research, with modern impact evaluation designs. Such designs measure causal effects University of Bonn, Bonn, Germany. of specific interventions by comparing outcomes with the interventions to out- Tel: +49-228-73-1873; comes in credible counterfactual scenarios. Good designs also identify the con- Fax: +49-228-73-1869. ditions under which the causal effect arises. Despite a critical need for empirical E-mail: jborner@uni-bonn.de evidence, conservation science has been slow to adopt these impact evaluation Received designs. We identify reasons for the slow rate of adoption and provide sugges- 13 September 2014 tions for mainstreaming impact evaluation in nature conservation. Accepted 10 April 2015 Editor William Sutherland doi: 10.1111/conl.12180 Impact evaluation has developed into a research disci- Introduction pline with multiple fields of application including health, Conservation science is only slowly beginning to build education, and development (White 2009). Our notion a body of evidence on the impact of conservation poli- of impact evaluation goes beyond monitoring program cies (Ferraro & Pattanayak 2006; Fisher et al. 2014). inputs, outputs, or indicators over time. It measures the Many compelling reasons motivate impact evaluations of causal effect of a specific policy, program or intervention conservation policy instruments. Organizations want to vis-a-vis ` a credible counterfactual scenario and seeks to know where to invest scarce resources, while govern- understand the conditions under which this effect arises ments and donors seek tangible outcomes. Evidence of (Ferraro & Hanauer 2014). In a comprehensive impact why conservation initiatives succeed or fail is also essen- evaluation, evaluators will rule out alternative or rival tial for designing cost-effective programs and improving explanations of program outcomes (Ferraro 2009). One the livelihoods of natural resource users (Sutherland et al. might also examine past outcomes to forecast the poten- 2004; Cook et al. 2010). In this article, we propose steps tial impact of future interventions (Pfaff et al. 2009). To toward mainstreaming and improving conservation pol- obtain these insights, impact evaluations must be more icy impact evaluation. than abstract quantitative evaluations but rather build 58 Conservation Letters, January/February 2016, 9(1), 58–64 Copyright and Photocopying: 2015 The Authors. Conservation Letters published by Wiley Periodicals, Inc. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. K. Baylis et al. Mainstreaming impact evaluation on qualitative theories of change which help identify the wintering grounds, often distributed across multi- conditions in which the desired impacts arise (Morgan & ple ecosystems and political administrations (Brower Winship 2007). 1995; Naidoo et al. 2014). In the presence of multi- We are not the first to make the above points. Sev- ple objectives at multiple scales, it also becomes more eral recent papers call for improving the quality of difficult to articulate clear theories of change and impact evaluation in nature conservation (Ferraro & empirical strategies for impact evaluation. Different Pattanayak 2006; Miteva et al. 2012; Pullin 2012; Fisher choices about scale will also inhibit comparable repli- et al. 2014). Despite these calls, conservation science still cations. lags behind health, education, and development policy in (2) Spatial spillovers. While many fields can ignore the adopting best practices in impact evaluation (Banerjee & spatial component of an impact evaluation, con- Duflo 2009). Few studies meet even the basic standards servation simply cannot. Space is an essential part of an impact evaluation such as considering before and of ecological processes: water flows, pollution emis- after conditions, including control groups, accounting for sions, species migration, deforestation, and disper- confounding factors, or systematically ruling out rival hy- sal. Therefore, to assess the impact of conservation potheses (Bowler et al. 2012; Samii et al. 2014). policies, one must account for the appropriate spatial In contrast to earlier essays on this subject, we ex- scale. Yet, even when the appropriate spatial scale plore the reasons why nature conservation policy has is well known, measuring the net impacts of an in- been slow to adopt more rigorous impact evaluation de- tervention is complicated by spatial spillovers. These signs. The reasons are not trivial and the solutions are not spillovers can be a result of ecological process, but simple. We characterize the current barriers and propose can also result from behavioral responses, such as elements of a strategy that may build a systematic body when restricting access to resources in one area in- of evidence on the effectiveness of conservation initia- duces a rise in extractive activity elsewhere, in what tives. Our arguments are based on discussions from the is referred to as “leakage” (Ostwald & Henders 2014). workshop “Evaluating Forest Conservation Initiatives: New Spillovers not only affect net impacts but can also Tools and Policy Needs” organized in Barcelona, Spain in bias impact estimation when they influence nontar- December 2013. get areas that were intended to serve as control ob- servations. (3) Confounding factors. Many biophysical, behavioral, Challenges for impact evaluation in and institutional factors affect both where conser- conservation science vation initiatives take place and the outcomes we measure. Imagine that the survival of a particular Conservation programs have features that, while not species depends on forest habitat under threat by unique to conservation, translate into specific challenges logging pressures. Policy makers respond by creating for impact evaluation. a new protected area, but the location and bound- (1) Multiple outcomes and scales. Conservation interven- aries of protection are developed in consultation with tions often strive to achieve multiple objectives at local municipalities who prioritize remote areas far from human settlements. An impact evaluation that multiple scales. For instance, ensuring viable species was to compare conservation outcomes inside this populations while protecting habitat; or maintaining ecosystem integrity while increasing the provision of park with conservation outcomes outside the park ecosystem services for human populations. “Coben- might erroneously find that the park was highly efits” may be relevant in other contexts, but in con- successful if areas with low deforestation risk were servation, cobenefits are often central to program protected, while areas with easier access, closer to success. The backlash against the Reduced Emissions human settlement, and high deforestation risk were from Deforestation and Forest Degradation (REDD) left unprotected (Joppa & Pfaff 2010). Assume fur- program, for initially focusing only on carbon cap- ther that timber values increased after the park was ture as a singular metric, illustrates the distaste for created, resulting in a generalized spike in logging. single policy objectives in a multiple-output setting A before-after comparison might lead to the erro- (Corbera et al. 2010). Furthermore, ecosystems are neous conclusion that the park was unsuccessful. complex systems with nonlinear dynamics at vari- In both cases, these approaches fail to address the ous spatial and temporal scales (Fisher et al. 2009; confounding factors affecting protected area place- Koch et al. 2009). Such complexity raises practical ment and outcome. These confounding factors must hurdles. For example, the conservation of migratory be accounted for in nonexperimental evaluations. To species requires management in both breeding and do so, evaluators need to draw on expertise from Conservation Letters, January/February 2016, 9(1), 58–64 Copyright and Photocopying: 2015 The Authors. Conservation Letters published by Wiley Periodicals, Inc. 59 Mainstreaming impact evaluation K. Baylis et al. various disciplines and on-the-ground knowledge. In results in a “conservation policy research cycle,” where cases in which confounders are not easily observable, the knowledge base is continuously updated as new evaluators often use instrumental variables – vari- evidence emerges. First, to refine theories of change, ables that only affect the outcome through their ef- researchers need to cross epistemological divides and inte- fects on the probability of participating in a program grate qualitative and quantitative approaches (Margoluis et al. (e.g., weather conditions or other shocks like natural 2009; Agrawal & Chhatre 2011). Qualitative understand- disasters). Finding such variables in the conservation ing helps contextualize quantitative treatment effect es- context is difficult because they often affect conser- timates and quantitative methods can inform qualitative vation outcomes directly. research design and theory development. As an exam- (4) Randomization’s limits. Conservation science has been ple, consider the use of quantitative data to inform the slow to adopt randomized controlled trials (RCT). selection of locations for in-depth qualitative analysis, ei- Notable exceptions include Ferraro et al. (2011), ther by targeting outliers or more representative sampling Jack (2013), Samii et al. (2014), and experiments groups (Roe et al. 2013). Multidisciplinary perspectives in habitat and invasive species management studies should not only inform theories of change and related (Sutherland et al. 2004). Practical and ethical consid- intervention designs, they can also help to develop more erations often limit the successful use of RCTs. Ran- appropriate evaluation strategies (White 2009). domization is not viable with small sample sizes or The second implication relates to choosing the appropri- low replication, and research designs must be ad- ate scale of analysis. In conservation practice, the unit of justed accordingly. However, it is difficult to obtain analysis, spatial scale, and outcome variable is not al- large sample sizes or replicate if a single interven- ways readily apparent. As a starting point, the analyti- tion covers a large geographic area. For example, cal scale should be motivated by the theory of change. to randomize a program to preserve water quality And, yet the decision-making unit and the resource gov- with acceptable statistical power, one might have to ernance regime in conservation programs can be neb- treat hundreds of watersheds. RCTs also rely on the ulous: ecosystems are comanaged by private owners, “stable unit value treatment assumption,” which im- collectives, communities, or state agents. The issue is fur- plies outcomes in one observation are not affected ther complicated because the natural, social, and med- by the treatment status of another. This assumption ical sciences differ in their view on what constitutes the may not hold in the presence of spatial spillovers: “right” scale, unit of analysis, and appropriate sample size. where the outcome of one parcel affects neighboring Since social and ecological processes operate at multiple parcels. Last, it may also be politically untenable or scales, any single choice of scale will inevitably fail to cap- unethical to randomly distribute restrictive conserva- ture certain dynamics. Researchers often compare con- tion regulations. Conservation policy evaluation will servation outcomes across a landscape by dividing their thus have to rely on paired research designs and in- study areas into a uniform grid. However, uniform grids novative quasi-experimental approaches, such as re- inevitably combine multiple ecosystem types, governance gression discontinuity design and synthetic control regimes, or property owners. Thus, the choice of analyt- analysis (e.g., Abadie et al. 2014). ical unit, as well as geographical (e.g., valley, watershed, (5) Small initiatives. Large-scale and generously funded landscape, ecosystem) and administrative (e.g., commu- pilot initiatives are rare in the conservation sector, nity, municipality, county, region) scale is challenged by constraining even those firmly committed to mea- methodological constraints. The solution is not found in suring impact. Innovative program designs are often selecting fine-grained analytical units because high res- developed by small organizations that integrate mul- olutions will generate spatial correlations that bias re- tiple funding streams and gradually develop their in- sults. Conversely, coarse resolutions may fail to capture tervention design through years of experience. For local processes and inhibit the identification of appropri- these organizations, it may not be feasible to embark ate controls. Selecting the appropriate scale can reduce on impact evaluation by themselves. Either outside the unobserved confounding factors, while using an inap- support or some critical mass of similar interventions propriate scale can exacerbate the effects of unobserved is probably needed to carry out full-scale evaluation confounding factors. Where the appropriate scale is designs. unknown, impact evaluations may use hierarchical mod- els or replicate the analysis at multiple scales to evalu- ate how sensitive results are to the choice of scale (e.g., Implications for conservation policy and Avelino et al. 2015; Borner ¨ et al. 2015; Costedoat et al. science 2015). Several implications arise for the design of impact The third implication relates to incorporating spillover evaluations and the effective integration of evaluation effects into the research design, including leakage, spatial 60 Conservation Letters, January/February 2016, 9(1), 58–64 Copyright and Photocopying: 2015 The Authors. Conservation Letters published by Wiley Periodicals, Inc. K. Baylis et al. Mainstreaming impact evaluation autocorrelation, and peer effects. Several recent papers complexity of the research design. It requires even more explore the effect of conservation policies on conserva- elaborate theories and more untestable (and often less tion outcomes in neighboring areas. Some report that an credible) assumptions than are required to estimate un- increase in protection in one area displaces deforestation conditional effects. Thus, even in RCT, estimates of het- activities to other areas (Oliveira et al. 2007; Meyfroidt & erogeneous treatment effects are considered much less Lambin 2009). Others find a “halo effect” whereby ar- credible than unconditional effects unless incorporated eas adjacent to protected areas are better protected than directly in the experimental design (Ferraro & Hanauer one might expect (Honey-Roses ´ et al. 2011; Gaveau et al. 2014). 2012; Robalino & Pfaff 2012). Ideally, theories and related evaluation methods would address both potential sources of spillovers, behavioral and mechanistic, since the exis- Moving forward tence of either should be part of the estimated treatment effect. Ignoring spillover effects will bias estimates of pro- Building a body of evidence on conservation policy ef- gram impact. fectiveness will require greater collaboration between Fourth, while randomization might not be possible for researchers and conservation managers akin to the long- programs that require large, contiguous areas, some con- standing partnership between medical scholars and clini- servation instruments, such as Payments for Ecosys- cians. The evidence base we advocate for is a global public tem Services (PES) or community-based programs are good. Therefore, it is not surprising that it has been diffi- amenable to randomization, particularly if the desired en- cult to muster the resources necessary for building a solid vironmental outcomes are local. For example, incentive- evidence base. Unless practitioners are strongly encour- based contracts are being randomly allocated in the aged by donors, we will end up with an underprovision mountains of Bolivia (Asquith et al. 2008; Jones 2012) of evidence; i.e., the status quo. and Uganda (Hatanga 2014). Where feasible and ethical, Systematic reviews and systematic maps (or evi- randomizing treatment can help researchers address po- dence gap maps) are a useful tool to synthesize sci- tential confounding factors by ensuring they are not asso- entific results and identify shortfalls for policy makers ciated with treatment. Randomization may also be used (Dicks et al. 2014). The Initiative for Impact Evaluation when a program is thought to work, and program man- (www.3ieimpact.org) and the Collaboration for Envi- agers would like to test variations of the program or spe- ronmental Evidence (www.environmentalevidence.org) cific aspects of its mechanism to identify why and how have recently published systematic reviews of the effect the program produces the desired results. It also might of protected areas, payment for ecosystem services and be easier to randomize over enforcement than over the aspects of forest management on various human welfare, placement of protected areas, or it might be possible to habitat and species preservation outcomes. In all of these randomize over the type of PES contract needed to induce reviews, authors point to the limited or fragmented ev- changes in household behavior. Furthermore, if a coun- idence of the effect of these various policy instruments try is interested in introducing a nation-wide conserva- (Bowler et al. 2010; Geldmann et al. 2013; Pullin et al. tion effort, randomizing over location might be feasible. 2013; Samii et al. 2014). Protected areas have arguably re- However, any randomized intervention would require an ceived the most attention with the reviews analyzing 86 important investment in communicating its purpose and articles on habitat and species outcomes and 306 articles the targeting rationale because, as noted earlier, such ap- on perceptions of PAs and 79 on welfare impacts. Never- proaches may entail political and ethical challenges. Fur- theless, large gaps remain. Even for protected areas, our thermore, randomization needs to be part of a broader understanding of spillover and heterogeneous treatment evaluation strategy that incorporates qualitative work to effects on environmental and socioeconomic indicators is explore the causal chain. still limited (Geldman et al. 2013; Pullin et al. 2013). Em- Finally, impact evaluation in conservation should be pirical evidence is even more sparse on the effectiveness sensitive to heterogeneous outcomes (Alix-Garcia et al. 2012; and implementation costs of other large- and small-scale Pfaff & Robalino 2012; Ferraro & Miranda 2013). Con- conservation policy instruments, such as forest law en- servation policies and programs affect a variety of so- forcement, PES schemes, ecocertification, and Integrated cial actors under varying biophysical conditions. Moving Conservation and Development approaches that domi- beyond the average effects of an intervention, conser- nate public and private REDD+ initiatives (Blom et al. vation planners need to know where and for whom it 2010; Lambin et al. 2014). Furthermore, we need to worked (Deaton 2009). Estimating heterogeneous treat- know how these policies compare in their effect on hu- ment effects and uncovering causal mechanisms behind man and environmental outcomes, and how these instru- average treatment effects is difficult and can increase the ments work in policy mixes (Barton et al. 2013). Conservation Letters, January/February 2016, 9(1), 58–64 Copyright and Photocopying: 2015 The Authors. Conservation Letters published by Wiley Periodicals, Inc. 61 Mainstreaming impact evaluation K. Baylis et al. Mainstreaming impact evaluation in conservation will Xie, and S. L. Zhou. Finally, we thank three anonymous require partnerships between scientists and program im- reviewers who improved the manuscript. plementers during the design phase to: (1) clarify pro- References gram objectives, possibly with a modification in design; (2) identify a theory of change, counterfactual groups, Abadie, A., Diamond, A. & Hainmueller, J. (2014). and testable hypotheses; and (3) define performance in- Comparative politics and the synthetic control method. Am. dicators and data collection protocols. Wherever feasi- J. Polit. Sci., doi:10.1111/ajps.12116. Agrawal, A. & Chhatre, A. (2011). Strengthening causal ble, a randomized program design may reduce rather inference through qualitative analysis of regression than increase the costs of impact evaluation, particularly residuals: explaining forest governance in the Indian in subnational or single-project pilot interventions. Such Himalaya. Environ. Plann. A, 43, 328-346. partnerships should be maintained over time to facilitate Alix-Garcia, J.M., Shapiro, E.N. & Sims, K.R.E. (2012). Forest continuous feedbacks between evaluation, design proto- conservation and slippage: evidence from Mexico’s cols and criteria, and implementation practice, which all national payments for Ecosystem Services Program. Land should be flexible, adaptive, and responsive to assess- Econ., 88, 613-638. ment outcomes (Sims et al. 2014). Donors could support Asquith, N.M., Teresa, M. & Wunder, S. (2008). Selling two this process by conditioning funding, including perfor- environmental services: in-kind payments for bird habitat mance bonuses on well-designed impact evaluation, and and watershed protection in Los Negros, Bolivia. Ecol. Econ., collaborating with researchers on defining priorities for 65, 675-684. focused and carefully designed systematic reviews (CEE Avelino, A.F.T., Baylis, K. & Honey-Roses, ´ J. (2015). 2013). Most importantly, impact evaluation needs long- Goldilocks and the grid: selecting scale when evaluating term support to build a strategic global evidence base conservation polities. University of Illinois Working Paper. for conservation policies (Keene & Pullin 2011). Such Banerjee, A.V. & Duflo, E. (2009). The experimental growing collaboration and mutual understanding should approaches to development economics. Annu. Rev. Econ., 1, slowly preclude awarding funding on the basis of exag- 151-178. gerated ex ante claims of conservation potential, since Barton, D.N., Blumentrath, S. & Rusch, G. (2013). this is counterproductive to building a solid body of ev- Policyscape—a spatially explicit evaluation of voluntary idence. Conservation at the scale envisaged by interna- conservation in a policy mix for biodiversity conservation tional policy initiatives, such as REDD+, clearly stands in Norway. Soc. Natur. Resour., 26, 1185-1201. to benefit from a solid body of evidence on what works, Blom, B., Sunderland, T. & Murdiyarso, D. (2010). Getting what does not, where and why. REDD to work locally: lessons learned from integrated conservation and development projects. Environ. Sci. & Pol., 13, 164-172. Borner, ¨ J., Kis-Katos, K., Hargrave, J. & Konig, ¨ K. (2015). Acknowledgments Post-crackdown effectiveness of field-based enforcement in the Brazilian Amazon. PLoS One, 10(4): e0121544. The perspectives put forward in this article are based Bowler, D., Buyung-Ali, L., Healey, J.R., Jones, J.P.G., on discussions among scientists and practitioners who Knight, T. & Pullin, A.S. (2010). The evidence base for attended the workshop “Evaluating Forest Conserva- community forest management as a mechanism for tion Initiatives: New Tools and Policy Needs” on 10–12 supplying global environmental benefits and improving December 2013, in Barcelona, Spain. We are grateful for local welfare. Environmental Evidence: support from the European Association of Environmental www.environmentalevidence.org/SR48.html. Accessed and Resource Economists, the Robert Bosch Foundation, May 2015. the Autonomous University of Barcelona, the Center for Bowler, D.E., Buyung-Ali, L.M., Healey, J.R., Jones, J.P., International Forestry Research, and the ERA-Net Bio- Knight, T.M. & Pullin, A.S. (2012). Does community forest divERsA funded INVALUABLE project with the French management provide global environmental benefits and national funder Agence Nationale de la Recherche. The improve local welfare? Front. Ecol. Environ., 10, 29-36. INVALUABLE project generously covered publication Brower, L.P. (1995). Understanding and misunderstanding fees. We also thank the workshop participants: A. An- the migration of the monarch butterfly (Nymphalidae) in gelsen, R. Arriagada, N. Asquith, M. Bottrill, J. Bush, North America: 1857–1995. J. Lepid. Soc., 49, 304-385. E. Cisneros, S. Costedoat, P. Davies, S. Desbureaux, L. CEE. (2013). Guidelines for systematic review and evidence Glew,E.Gorriz, ´ B. Hedden-Dunkhorst, N. Hogarth, C. synthesis in environmental management. Collaboration for Loucks, A. Leonardi, B. Macura, A. Martin, M. Masozera, environmental evidence. http://www.environmental- D. Miteva, S. Pagiola, V. Rodr´ıguez Osuna, J. Schielein, evidence.org/wp-content/uploads/2014/06/Review- P. Shah, E. Sills, J.M. Torres, E. Varela, G. Le Velly, C. guidelines-version-4.2-final.pdf. Accessed May 2015. 62 Conservation Letters, January/February 2016, 9(1), 58–64 Copyright and Photocopying: 2015 The Authors. Conservation Letters published by Wiley Periodicals, Inc. K. Baylis et al. Mainstreaming impact evaluation Cook, C.N., Hockings, M. & Carter, R. (2010). Conservation in iied.org/random-trials-uganda-show-if-payments-for- the dark? The information used to support management ecosystems-services-really-work. Accessed May 2015. decisions. Front. Ecol. Environ., 8, 181-186. Honey-Roses, ´ J., Baylis, K. & Ramirez, M.I. (2011). A spatially Corbera, E., Estrada, M. & Brown, K. (2010). Reducing explicit estimate of avoided forest loss. Conserv. Biol., 25, greenhouse gas emissions from deforestation and forest 1032-1043. degradation in developing countries: revisiting the Jack, B.K. (2013). Private information and the allocation of assumptions. Climatic Change, 100, 355-388. land use subsidies in Malawi. Appl. Econ., 5, 113-135. Costedoat, S., Corbera, E., Ezzine-de-Blas, D., Baylis, K., Jones, J. (2012). Can paying for global ecosystem services Honey-Roses, ´ J. & Castillo-Santiago, M.A. (2015). How reduce poverty? Funded by Ecosystem Services for Poverty effective are biodiversity payments in Mexico? PloS ONE Alleviation Project, http://www.espa.ac.uk/projects/espa- 10(3), e0119881, doi 10.1371/journal.pone.0119881. 2012-grants. Accessed May 2015. Deaton, A.S. (2009). Instruments of development: Joppa, L. & Pfaff, A. (2010). Reassessing the forest impacts of randomization in the tropics and the search for the elusive protection: the challenge of nonrandom location and a keys to economic development. NBER Working Paper corrective method. Ann. NY Acad. Sci., 1185, 135-149. 14690. Keene, M. & Pullin, A.S. (2011). Realizing an effectiveness Dicks, L.V., Walsh, J.C. & Sutherland, W.J. (2014). Organising revolution in environmental management. J. Environ. evidence for environmental management decisions: a “4S” Manage., 92, 2130-2135. hierarchy. Trends Ecol. Evol., 29, 607-613. Koch, E.W., Barbier, E.B., Silliman, B.R., et al. (2009). Ferraro, P.J. (2009). Counterfactual thinking and impact Non-linearity in ecosystem services: temporal and spatial evaluation in environmental policy. New Directions for variability in coastal protection. Front. Ecol. Environ., 7, Evaluation, 122, 75-84. 29-37. Ferraro, P.J. & Hanauer, M.M. (2014). Advances in measuring Lambin, E.F., Meyfroidt, P., Rueda, X., et al. (2014). the environmental and social impacts of environmental Effectiveness and synergies of policy instruments for land programs. Annu. Rev. Environ. Resour., 39, 495-517. use governance in tropical regions. Global Environ. Chang., Ferraro, P.J. & Miranda, J.J. (2013). Heterogeneous treatment 28, 129-140. effects and mechanisms in information-based Margoluis, R., Stem, C., Salafsky, N. & Brown, M. (2009). environmental policies: evidence from a large-scale field Design alternatives for evaluating the impact of experiment. Resour. Energy. Econ., 35, 356-379. conservation projects. New Directions for Evaluation, 122, Ferraro, P.J. & Pattanayak, S.K. (2006). Money for nothing? 85-96. A call for empirical evaluation of biodiversity conservation Meyfroidt, P. & Lambin, E.F. (2009). Forest transition in investments. PLoS Biol., 4, e105. Vietnam and displacement of deforestation abroad. P. Natl. Ferraro, P.J., Miranda, J.J. & Price, M.K. (2011). The Acad. Sci. USA, 106, 16139-16144. persistence of treatment effects with norm-based policy Miteva, D.A., Pattanayak, S.K. & Ferraro, P.J. (2012). instruments: evidence from a randomized environmental Evaluation of biodiversity policy instruments: what works policy experiment. Am. Econ. Rev., 101, 318-322. and what doesn’t? Oxford Rev. Econ. Pol., 28, 69-92. Fisher, B., Turner, R.K. & Morling, P. (2009). Defining and Morgan, S.L. & Winship, C. (2007). Counterfactuals and causal classifying ecosystem services for decision making. Ecol. inference: methods and principles for social research.Cambridge Econ., 68, 643-653. University Press, Cambridge, UK. Fisher, B., Balmford, A., Ferraro, P.J., et al. (2014). Moving Naidoo, R., Chase, M.J., Beytell, P., et al. (2014). A newly Rio forward and avoiding 10 more years with little discovered wildlife migration in Namibia and Botswana is evidence for effective conservation policy. Conserv. Biol., 28, the longest in Africa. Oryx, 1-9, available from: 880-882. http://dx.doi.org/10.1017/S0030605314000222. Gaveau, D.L.A., Curran, L.M., Paoli, G.D., Carlson, K.M., doi:10.1017/S0030605314000222. Wells, P. & Ratnasari, D. (2012). Examining protected area Oliveira, P.J.C., Asner, G.P., Knapp, et al. (2007). Land-use effectiveness in Sumatra: importance of regulations allocation protects the Peruvian Amazon. Science, 317, governing unprotected lands. Conserv. Lett., 5, 142-148. 1233-1236. Geldmann, J., Barnes, M., Coad, L., Craigie, I., Hockings, M. Ostwald, M. & Henders, S. (2014). Making two parallel & Burgess, N. (2013). Effectiveness of terrestrial protected land-use sector debates meet: carbon leakage and indirect areas in reducing biodiversity and habitat loss. Environ. land-use change. Land Use Policy, 36, 533-542. Evid., www.environmentalevidence.org/ SR10007. Pfaff, A. & Robalino, J. (2012). Protecting forests, biodiversity, Accessed May 2015. and the climate: predicting policy impact to improve policy Hatanga, P. (2014). Random trials in Uganda to show if choice. Oxford Rev. Econ. Pol., 28, 164-179. payments for ecosystem services really work. International Pfaff, A., Robalino, J., Sanchez-Azofeifa, G.A., Andam, K.S., Institute for Environment and Development. http://www. Ferraro, P.J. (2009). Park location affects forest protection: Conservation Letters, January/February 2016, 9(1), 58–64 Copyright and Photocopying: 2015 The Authors. Conservation Letters published by Wiley Periodicals, Inc. 63 Mainstreaming impact evaluation K. Baylis et al. land characteristics cause differences in park impacts across (2014). Effects of Decentralized Forest Management (DFM) Costa Rica. BEJ Econ. Anal. Pol., 9, Article 5. on Deforestation and Poverty in Low and Middle Income Pullin, A.S. (2012). Realising the potential of environmental Countries: a systematic review. CEE 13-015a. Collabo- data: a call for systematic review and evidence synthesis in ration for Environmental Evidence. http://www. environmental management. Environ. Evid., 1:2, 1-3. environmentalevidence.org/wp-content/uploads/ Pullin, A.S., Bangpan, M., Dalrymple, S., et al. (2013). Human 2015/01/Samii DFM Review-formatted-for-CEE.pdf. well-being impacts of terrestrial protected areas. Environ. Accessed May 2015. Evid., 2:19, 1-41. Sims, K.R.E., Alix-Garcia, J.M., Shapiro-Garza, E., et al. Robalino, J.A. & Pfaff, A. (2012). Contagious development: (2014). Improving environmental and social targeting neighbor interactions in deforestation. J. Dev. Econ., 97, through adaptive management in Mexico’s payments for 427-436. hydrological services program. Conserv. Biol., 28, Roe, D., Grieg-Gran, M. & Mohammed, E.Y. (2013). 1151-1159. Assessing the social impacts of conservation Sutherland, W.J., Pullin, A.S., Dolman, P.M. & Knight, T.M. policies: rigour versus practicality. IIED Briefing (2004). The need for evidence-based conservation. Trends Papers. International Institute for Environment and Ecol. Evol., 19, 305-308. Development. White, H. (2009). Theory-based impact evaluation: principles Samii, C., Lisiecki, M., Kulkarni, P., Paler, L. & Chavis, L. and practice. J. Dev. Eff., 1, 271-284. 64 Conservation Letters, January/February 2016, 9(1), 58–64 Copyright and Photocopying: 2015 The Authors. Conservation Letters published by Wiley Periodicals, Inc.

Journal

Conservation LettersWiley

Published: Jan 1, 2016

Keywords: ; ; ; ;

There are no references for this article.