Over the past decade or so, social funds and large-scale community driven development (CDD) programmes have become an increasingly common policy choice in conflict-affected settings (World Bank 2006a). Classified as popular forms of ‘community development’ – one of the major modalities for inducing local participation in development activities and processes – these programmes attempt to ‘bring villages, urban neighbourhoods, or other household groupings into the process of managing development resources without relying on formally constituted local government’ (Mansuri and Rao 2013: 1). Despite multiple variations in the specifics of their design and implementation (e.g., in the degree of control participants have over resources and decisions, in how participants are targeted and selected in the first place), social funds and CDD programmes typically involve financing demand-driven community-led projects at the local level, particularly in poor communities. Financed projects are diverse: while many focus on making improvements (sometimes modest, sometimes substantial) to the quality or provision of social infrastructure, such as schools and health centres, others involve transferring largely unconditional cash grants directly to recipients in order to catalyse economic productivity.
With enormous support and backing from the World Bank, social funds and large-scale CDD programmes are seen to constitute an important approach to development in conflict-affected contexts, partly because they are ‘designed to place less stress on government line agencies by optimizing the use of community actors’ (Wong 2012: iv), and partly because of their widely assumed ‘transformative potential’ to reconstruct societies through the creation of new local institutions (Fearon et al. 2009). As Barron (2011: 13) explains: ‘In areas affected by conflict, it is often claimed that CDD holds potential to encourage new forms of collaboration across conflict divides, which can improve trust and make communities less prone to fresh violence.’
Indeed, social funds and CDD projects have been framed by some as instrumental to the achievement of an ambitious range of economic, social and governance outcomes in conflict-affected situations, from gains in women’s empowerment to improvements in social cohesion and state-society relations (Barron 2011; Besley and Persson 2012). The prime example here is the National Solidarity Programme in Afghanistan, which, by design, represents a “deep” intervention into communities to shape existing socio-political hierarchies, promote democratic decision making, foster gender inclusivity, extend the power of the state, devolve authority from the national to the district level and, amongst all this, rehabilitate infrastructure and improve living conditions.
But is this transformative potential borne out by empirical evidence? This practice note takes stock of the literature on social funds and CDD in fragile and conflict-affected situations, synthesising available evidence of programme performance against three areas of impact: (i) incomes, enterprise and access to services; (ii) social cohesion, stability and violence; and (iii) state-society relations. It builds on a longer working paper (Carpenter et al. 2012), which focuses more broadly on social protection and basic services in fragile and conflict-affected situations, and draws on the findings of a systematic review into the impacts of social funds in conflict-affected situations (Carpenter et al., forthcoming). Given that the original pieces of research were completed a number of months ago, it must be emphasised up front that this note may not draw comprehensively on the most recent evidence. That said, the findings and lessons captured here remain, in our opinion, applicable and of use to those engaged in this field.
The structure of the note unfolds as follows. After describing both the methods underpinning the production of this note, as well as the kinds of interventions covered, we present evidence on the performance of social funds and CDD programmes against the first area of impact – incomes, enterprise and access to services. We then assess performance against the second area – social cohesion, stability and violence – before concentrating on the third and final impact area, state-society relations. We then present the summary findings of our systematic review, which cover all three areas of impact, before closing with some brief concluding remarks.
What the Review Process Looked Like: Notes on Methodology
As just mentioned, this note draws on the findings of rigorous, evidence focused literature review, based on both a formal systematic review approach as well as a broader, more flexible method of retrieval and analysis.
Systematic reviews are considered a ‘rigorous method to map the evidence base in an [as] unbiased way as possible, and to assess the quality of the evidence and synthesise it’ (DFID 2013) and are seen by some to offer ‘the most reliable and comprehensive statement about what works’ (Petrosino et al., in van der Knaap et al. 2008: 49). They involve following a rigid search, synthesis and analysis protocol which is determined at the outset of the review, requiring researchers to comply stringently with particular inclusion and exclusion criteria. We provide here a brief description of the social funds studies unearthed through this process (for more information, see Carpenter et al., forthcoming).2
From the 76 studies retrieved from the systematic search process, involving seven academic databases and 13 institutional websites, a total of nine studies met the criteria for inclusion in the final analysis. The included studies cover seven ‘fragile states’, with two studies on Afghanistan (Barakat 2006; Beath et al. 2010), two studies on Yemen (ESA Consultores 2003; World Bank 2006b), and the others on Ethiopia (World Bank 2005), Nepal (World Bank 2009a), Nigeria (World Bank 2009b), northern Uganda (Golooba-Mutebi and Hickey 2010) and Timor-Leste (Moxham 2005). Of these studies, two were focused on projects directed at specific sub-national situations of fragility, including three conflict-affected districts in northern Uganda and Ethiopia’s woredas (districts) most affected by the Ethio-Eritrean conflict of 1998–2000.
Although the quality of studies was mixed (with a number not providing any information on their methodology), where explanation was provided on methods, the studies scored respectably. Of the included studies, two used qualitative methods, primarily focus groups and interviews, and two used quantitative methods, including surveys. The others applied both, although this was in order to address different aspects of the fund impact and thus did not constitute a triangulated, mixed-methods approach. One major hindrance to the quality of studies was the absence of baseline data in most situations, forcing researchers to rely on, and thus replicate in their data collection tools, indicators from government-led national poverty surveys, for example in Yemen and Nepal.
Sampling strategies were articulated relatively well and were largely appropriate in around half (four) of the studies. Encouragingly, four of the nine studies discussed their data collection process, an often-neglected issue (see Hagen-Zanker et al. 2012). However, none of the studies discuss the assumptions underlying their causal claims, which may reflect the lack of theoretical frameworks to draw on in the evaluation of social funds, particularly in relation to stability-related outcomes. While eight of nine studies disaggregated their data by gender, only two conducted analysis of gendered power relations. Further, only one included a discussion of relevant ethical issues relating to the research, and none discussed the issue of reflexivity in relation to their research.
Methodological constraints of the systematic review process
While systematic reviews have some advantages, they are not problem free. As we have argued elsewhere (Hagen-Zanker et al. 2012), although using systematic review principles can help researchers improve the rigour of literature reviews, the rigidity with which they are sometimes applied can prevent the retrieval of all relevant evidence. In response to recognition of this and other shortcomings, we augmented our review with an additional mechanism which allowed us to capture some notable contributions not yielded through the original systematic review method. Given that the systematic review was undertaken in 2011, and that the nature of an evidence base is ever changing, this approach also enabled us to include more recently published studies. The findings from both ‘phases’ of the literature review have been brought together here to form a narrative description of the impacts of social funds and CDD programmes in conflict-affected situations.
Before moving on, however, it bears emphasising that this note cannot be considered a comprehensive or final statement on the effectiveness of these programmes. The ability of reviewers to identify and synthesise all available evidence on a given intervention or issue is constrained by multiple factors. These include, for example, difficult information architecture, time and resource limitations and protected primary data; furthermore, new evidence is being generated all the time. We return to this and related issues in the concluding section.
How Form and Function Vary by Programme: Notes on the Interventions
This 13 programmes discussed here are as follows: the National Solidarity Programme in Afghanistan, the Community Empowerment and Local Governance Project in Timor Leste, the Nepal Poverty Alleviation Fund, the Community-Based Reintegration Assistance for Conflict Victims (BRA-KDP) in Indonesia, the Northern Uganda Social Action Fund (NUSAF), the GoBifo project in Sierra Leone, a community driven reconstruction programme implemented by the International Rescue Committee (IRC) in Liberia, the Tuungane programme in the Democratic Republic of Congo (DRC), the Autonomous Region in Muslim Mindanao Social Fund project in the Philippines, the KALAHI Comprehensive and Integrated Delivery of Social Services project in the Philippines, the Yemen Social Fund for Development, the Ethiopian Social Rehabilitation and Development Fund, and the Nigeria Community Based Poverty Reduction Project.
Although all qualify as social funds or CDD programmes, this is not to say they look or work in the same way. Let us briefly compare, for example, two of those listed above: NUSAF and the GoBifo project in Sierra Leone. Although both were originally implemented in similar contexts – post-conflict regions characterised by high levels of poverty and fractured relationships with formal state institutions – the two programmes look quite different.
The GoBifo project was implemented in 2006 as part of a wider institutional reform agenda, and was tasked with making local government institutions more inclusive, partially in an attempt to address the perceived root causes of the civil war (Casey et al. 2012: 1758). Operating in two districts, the project consisted of two main components: (i) financial assistance in the form of block grants to fund local public goods provision and small enterprise development and (ii) attempts to establish inclusive governance structures, such as Village Development Committees, to facilitate collective action. As part of this, the project required that at least one of the three co-signatories of the community bank account be female, and several attempts were made to encourage the involvement of women and youth in the project’s processes. On average, GoBifo disbursed grants of just under US$5,000 to communities with 50 households (or 300 residents) – working out at roughly US$100 per household, or US$4.50 per capita over three and a half years. Importantly, project facilitators were required to spend one day a week in each of the participating villages, meaning that by the end of the project in 2009 each village had received roughly six months of direct facilitation.
Although similar in some respects, NUSAF was a considerably larger intervention, both in terms of cost (originally US$133.5 million) and geographical reach (initially operational in 18 districts, although later expanded to 24). The programme was framed as a massive exercise in poverty reduction, as a means of enabling the conflict-affected northern region to ‘catch up’ with the rest of the country (Gelsdorf et al. 2012; Golooba-Mutebi and Hickey 2009). Targeted particularly at vulnerable groups, NUSAF originally consisted of four components: community driven initiatives, vulnerable group support, community reconciliation and institutional capacity building. A fifth component, the Youth Opportunities Program, was introduced in 2006. This component offered young adults the opportunity to apply to their district government for large, mostly unconditional cash grants (on average US$7,497 per group, or US$382 on a per capita basis, which is approximate equivalent to a youth’s baseline annual income) (Blattman et al. 2013: 7). In contrast to the GoBifo project, facilitators were only involved with the preparation of the groups’ proposals; upon the transfer of a grant following a successful proposal, any kind of supervisory engagement was stopped.
What we already see from these two brief descriptions, then, are some important differences between the programmes. NUSAF, a longer running and significantly larger programme, has historically and primarily been tasked with addressing geographical wealth disparities, whereas the much smaller GoBifo project appears to have originally been more concerned with institutional change, albeit partly through material development progress. Moreover, variations in the degree of support and facilitation provided to groups are just one example of the way in which the specifics of programme design might influence outcomes (for example, through the creation of selection bias). Such details are important but sometimes overlooked.
Thus, there are often important differences in the design of programmes and in the way in which they are implemented – differences which may be explained by variations in context, programme objectives and bureaucratic or administrative factors. A proper understanding of impact depends on an appreciation of the particulars of the intervention in question, something which often calls for additional (and substantial) analysis and interpretation. The example which emerges when comparing NUSAF and GoBifo is just one of many across programmes, but provides an indication of the multiple ways in which programmes are implemented – and not just the type of programme – can affect their impact.
Incomes, Enterprise and Access to Services
The impacts of social funds and CDD on the economic and social welfare of beneficiaries are mixed, although a number of recent studies offer promising results.
At one end of the spectrum are the cases of Afghanistan and Timor-Leste. Beath et al.’s (2010: 66) recent mid-term evaluation of the National Solidarity Programme in Afghanistan states that:
the programme does not result in any change in levels of household income flows or the incidence of poverty or in the regularity of income sources and also has no effect on levels of consumption expenditures, the composition of household consumption, or on the extent to which the food needs of households are met.
That said, a follow-up study by the authors found a strong positive impact of the Programme on subjective economic outcomes for both male and female respondents, with participants: (i) more likely to report that the economic situation in their household had improved from the previous year; and (ii) more likely to feel positively about their village’s economic prospects for the following year (Beath et al. 2012b: 16). The authors find that, overall, the proportion of respondents perceiving their economic situation in a positive light was roughly five percentage points higher in villages participating in the National Solidarity Programme.
In Timor-Leste, it has been suggested that unviable, artificial and implanted enterprise projects were the reason underlying the failure of the enterprise development component of the country’s Community Empowerment and Local Governance Project (CEP), a US$18 million community reconstruction programme which provided funds to more than 400 newly created local development councils (Moxham 2005). The same study also highlights the programme’s failure to build on existing capacities and sources of resilience.
More recent evaluation work from eastern DRC finds that the economic wellbeing impacts of Tuungane, a large community driven reconstruction programme with a targeted beneficiary population of almost 2 million people, are ‘surprisingly negative’ (Humphreys et al. 2012). The authors, who also argue that the ‘null findings are broadly in line with results from other studies that have also failed to find evidence for the strong claims made on behalf of the [Community Driven Reconstruction] model’, suggest that the weak effects may be attributed to low per capita investments (in terms of both financing and training) and the short timeline between treatment and measurement.
At the other end of the spectrum are experiences from Nepal, Indonesia and Uganda. Drawing on data from two survey rounds, Parajuli et al. (2012) find a statistically significant causal impact of the Nepal Poverty Alleviation Fund on key welfare outcomes, including a 19 per cent growth in real per capita consumption, a 19 per cent decline in the incidence of food insecurity and a 15 per cent increase in the school enrolment rate among those aged between six and 15.3 Although the Nepalese context is quite different from some of those discussed above – particularly in terms of levels of insecurity, institutional capacity and donor support – the findings are nonetheless useful for informing programme design in ‘fragile and difficult political environments’ (ibid.: 3). Lending general support to these findings is a study by Barron et al. (2009) from Indonesia. Evaluating the BRA-KDP – a US$21.7 million programme designed to support the welfare of conflict victims, build social cohesion and improve trust in the state among an estimated 530,000 beneficiaries in more than 1,700 villages in Aceh – the authors report that the programme is ‘associated with a strong set of welfare gains and improvements in perceptions of well-being’ (ibid.: iii). Finally on this issue, mid-term results from Blattman et al.’s (2011) experimental mid-term evaluation of NUSAF in northern Uganda suggest that the programme’s economic impacts are large: cash earnings for both men and women increase by more than 40 per cent compared with the control group, and the authors estimate the transfer provided by NUSAF yields a real annual return on capital of 35 per cent on average. Longer run evidence from the same experiment confirms these ‘huge economic effects’ (Blattman et al. 2013).
Social funds can affect basic services, either directly where programme activities contribute to improved services, or indirectly where beneficiaries become sufficiently empowered to demand more or better services. In this case, there is a wider body of evidence on which to draw. The Social Fund for Development (SFD) in Yemen has seen statistically significant increases in the enrolment rate for girls (41.7 per cent to 58.3 per cent), the proportion of sick individuals who managed to receive health care for their illness (54.9 per cent to 68.4 per cent) and access to household taps providing clean drinking water (35 per cent increase) (ESA Consultores 2003). Taniguchi (2012) also reports positive impacts of a social fund in Mindanao on participants’ access to services. In fact, of the four studies included in our original systematic review which measured educational effects, all found positive impacts on enrolment (ESA Consultores International 2003 and World Bank 2006b on Yemen; World Bank 2005 on Ethiopia; World Bank 2009b on Nigeria).
Further, NUSAF in northern Uganda has created community assets, such as boreholes, that have had tangible benefits in terms of reducing the burden on women and children with regard to carrying water, and new classrooms and accommodation for teachers are associated with higher levels of attendance, both of children and staff, at schools (Golooba-Mutebi and Hickey 2010). Similarly, increased access to improved water sources has been associated with the CEP in Timor-Leste (Moxham 2005). But for the National Solidarity Programme in Afghanistan, there is no clear picture from the evaluations conducted to date on its impact on access to health and education services (Beath et al. 2010).
Social Cohesion, Stability and Violence
The story is again mixed on social cohesion, with studies on social funds and CDD projects showing both positive and negative impacts. A randomised field experiment conducted by Fearon et al. (2009) on an IRC project found that the introduction of community development committees (CDCs), and exposure to their operation, enhanced the ability of community members to act collectively for mutual gain.4 The study reported a significant effect on community cohesion, measured through the amount of funding the community raised for a collective project through anonymous play in a public goods game, with a 9 per cent increase seen in the treatment group. The authors assert that their study shows that improvements in social cohesion can occur in a short space of time, in response to outside intervention, and can develop without fundamental changes to the structure of economic or macro-level relations.
Taniguchi (2012) studies the effects of the Autonomous Region in Muslim Mindanao Social Fund for Peace and Development in conflict-affected areas. Drawing on focus group and structured questionnaire data collected from 50 villages, the author reports a range of ‘positive physical, psychological and governance outcomes’ (ibid.: 11) of the project (although it must be pointed out that no control group was used). These include increases in: (i) the rate of participation in village assemblies and an increased opportunity for group work; (ii) knowledge of village affairs; (iii) the level of trust in regional government and/or local government units; and (iv) access to public services. Taniguchi (2012: 10) also identified improvements in participants’ perceptions of the security situation in their villages (attributed to ‘an internal sense of security […] due to social inclusion or integration, active social relationships, positive interactions and an increased level of trust’).
Conversely, a randomised field experiment by Casey et al. (2011) in Sierra Leone found no evidence that the GoBifo project led to fundamental changes in the ‘software of collective action – namely, local fundraising capacity, decision-making processes, or even social attitudes and norms’ (ibid.: 5). The authors speculate that this may owe to the fact that attempts to create new institutions and norms where formal structures have broken down – such as in Liberia at the time of the Fearon et al. (2009) study – may encounter less resistance than similar efforts in situations where it is necessary to transform the behaviour of existing actors and groups – such as chiefs in the case of Sierra Leone. In a slightly more recent contribution, Casey and colleagues (2012) suggest it may be worth questioning the assumption that the civil war in Sierra Leone left communities ‘highly compromised in terms of social cohesion and their ability to work together’ (ibid.: 16), with baseline survey data indicating already high levels of social cohesion prior to the CDD intervention. Separate research by Bellows and Miguel (2009) further suggests that individuals who experience war-related violence often become even more politically and socially active than they were before.5 The central implication here is that ‘[s]uch initially high levels of cohesion throw into doubt the necessity of the social facilitation aspect of CDD in the context’ (Casey et al. 2012: 17).
Moreover, in northern Uganda, the Vulnerable Group Support scheme (one component of NUSAF) has been found by one qualitative study to have had a negative impact on social cohesion (Golooba-Mutebi and Hickey 2010). In nearly 60 per cent of the authors’ 72 interviews in Nebbi district, respondents stated that the projects in their area had generally failed. In terms of social cohesion, many respondents stated that the distribution of assets within beneficiary groups had regularly led to the outbreak of violent conflict, with some even citing family breakdown and fatalities following the resulting violence. That said, Blattman et al.’s (2011) mid-term evaluation of the Youth Opportunities Program – a separate component of NUSAF – finds ‘mild’ improvements in social cohesion and community support (of between five per cent and ten per cent) and a more impressive 50 per cent decline in interpersonal aggression and disputes among males. Findings were not universally positive, however, with a 50 per cent rise in such disputes among females, a pattern, it must be noted, which disappeared after four years (Blattman et al. 2013: 30). On a broadly similar note, the recent evaluation of the Tuungane programme in eastern DRC found no effects on measures of both within and between village cohesion (Humphreys et al. 2012).
The relationship between CDD and stability – proxied by levels of violence – is far from clear, but, as suggested above, in some cases projects have been found to inadvertently accentuate violence, especially in areas where there is competition over project resources (see Barron 2011). A recent study by Crost et al. (2012b) into the effect of KALAHI-CIDSS – the Philippines’ flagship anti-poverty project between 2003 and 2008 – on casualties in armed civil conflict finds that the programme exacerbated violent conflict in eligible municipalities. The effect is found to be large and statistically significant and, claim the authors, cannot be explained by ‘differences in pre-program violence or other observable and fixed unobservable characteristics’ (ibid.: 4). Further, the increase in violence lasted only for the duration of the programme and was stronger for municipalities that received larger amounts of aid.
What about the frequently and enthusiastically discussed National Solidarity Programme in Afghanistan? Generally speaking, evaluations have found mixed evidence in terms of impacts on social cohesion. The mid-term evaluation by Barakat (2006) found enhanced perceptions of community solidarity achieved through processes of community governance, with 86 per cent of respondents (90 per cent of women) volunteering such a description of the impact of the local CDC on their lives.6 But the mid-term randomised evaluation by Beath et al. (2010: 52) notes that the National Solidarity Programme had ‘no impact on specific measures of community trust or solidarity or on the outbreak of village disputes or tribal feuds’. (It should be pointed out here that the both studies took place prior to the completion of many projects, meaning the impacts identified must be treated as intermediate.) Regarding stability, more recent evaluation results suggest that although the National Solidarity Programme has improved villagers’ perceptions of security, there is little convincing evidence that it has had any significant effect on actual security incidents in or around villages (Beath et al. 2012b: 17–18).
There appears to be considerable variation in the effects of social funds and CDD on levels of violence and patterns of social cohesion. What might explain this? Two central factors emerge from the literature.
First, programme design appears important. While Crost et al. (2012a) have suggested that high ‘visibility’ of a transfer may increase levels of violence and conflict through the creation of ‘high-profile targets’ for insurgents, Taniguchi (2012: 11) attributes the formation of social capital among participants in Mindanao to the programme’s ‘demand-driven, inclusive, simple, and transparent approaches’. The implication here is that how something is designed and implemented may be just as, if not more important, than what it is that is being implemented in the first place. The results of recent work on service delivery in conflict-affected situations seem to reflect this (see, for example: McLoughlin 2013; Wild and Mason 2012).
Second, and perhaps unsurprisingly, context also appears to be a strong determinant (World Bank 2006a). Outcomes may be dependent to a large degree on, among other things, pre-existing “levels” of social capital within communities, the extent to which community relations were damaged by conflict, and the motives and strategies of insurgent groups. In reality, therefore, the impacts or CDD and social fund programming are likely to be mediated not only by the particulars of their design, but also by the context in which they are implemented. Future work would do well, therefore, to investigate more rigorously the aspects of programme design and implementation context that appear to be associated with positive impacts on stability and violence reduction.
The literature on CDDs also contains some discussion of their ability to strengthen state–citizen relations and contribute to state-building outcomes. However, empirical evidence on this relationship is scarce and the data that do exist do not offer much in the way of a reliable basis for conclusions. That said, promising ongoing work by Beath and colleagues in Afghanistan (see Beath et al. 2012a) will hopefully lead to a strengthened evidence base around this important issue.
In terms of what currently exists, there is mixed qualitative evidence from northern Uganda. Drawing on the case of NUSAF, Manor (2007) argues that social funds can undermine government legitimacy when their well-funded committees operate alongside elected and local bodies with insufficient money to fulfil their mandates. Certainly, there has been recognition among the donor community that NUSAF did excessively circumvent local government and institutions. As a result, recent initiatives, including the DFID Northern Uganda Post-conflict Recovery Programme and the US Agency for International Development (USAID) Northern Uganda Development of Enhanced Local Government Infrastructure and Livelihoods programme, have been set up to directly work with and fund local government.
However, other research from northern Uganda suggests a different picture, as the response of a senior politician in Gulu district regarding the impact of NUSAF on people’s civic awareness illustrates (Golooba-Mutebi and Hickey 2010: 1229):
It enabled people to learn how to be demanding as a society. It somehow strengthened the demand side. In a situation where the state had almost collapsed here because of the war, citizens somehow started seeing the state just as a security agency without any ability to respond to their economic and welfare needs. So through NUSAF people learnt that you can actually demand some interventions.
Testimonies of beneficiaries of NUSAF reinforced this view. By enabling them to go to local government offices to check up on their applications and of dealing with public servants, the sub-projects reinforced the sense that it was beneficiaries’ right to act in this way (Golooba-Mutebi and Hickey 2010). This was not, however, a universal state of affairs: some experienced a deepened sense of resignation that their views and demands were not being listened to or acted on, and the high rate of project failure within NUSAF has lent support to those elites that question the capacity of the poor to manage development resources effectively (ibid.). The most recent Blattman et al. (2013) evaluation suggests that involvement in NUSAF – or, more specifically, the Fund’s Youth Opportunities Programme – has neither strengthened nor undermined beneficiaries’ political participation, with no or only weak effects observed on voting behaviour, support for the government and protest attitudes and behaviour.
Yet, while the findings of Barron et al.’s (2009) Indonesian study suggest a similar state of affairs – ‘there is only minimal evidence that exposure to BRA-KDP resulted in higher levels of trust in village and higher level governments’ (ibid.: iii) – a more positive picture emerges from Afghanistan and Sierra Leone. For example, Beath et al. (2012b) report that the National Solidarity Programme has improved attitudes towards government figures at almost all levels, with the strong caveat that ‘positive effects on attitudes are not observed in areas with high levels of initial violence’ (ibid.: 17), and Casey et al. (2012) find that although a CDD intervention in Sierra Leone failed to alter social and institutional dynamics at the community level, it was nonetheless able to create ‘meaningful links between villagers and the lowest tiers of elected government’ (ibid.: 16).
How Far Does This Evidence Take Us?
This practice note has provided a stocktake of the impacts of social funds and large-scale community driven development (CDD) programmes in fragile and conflict-affected situations, with a particular focus on their assumed transformative effects. However, the note has also opened up an opportunity for a critical discussion of the nature of the empirical research in this area – an opportunity to ask: how far does this evidence take us?
On the basis of the studies summarised here, we can conclude that many of the impacts of social funds and CDD programmes in conflict-affected situations are encouragingly positive. Participation in these interventions can lead to substantial and sustained increases in economic wellbeing at the individual and household level. It can also improve access to basic services, such as education, with plenty of evidence of increases in enrolment across contexts. We also find that participation in social funds and CDD programmes can sometimes lead to improvements in various measures of local-level stability, social cohesion and state-society relations. However, the evidence in this area is much more mixed and far less convincing than when we consider the effects on more basic measures of material wellbeing. The central message emerging here is that any bold claims regarding the ability of social funds and CDD programmes to pacify societies and cement social contracts – to generate, in other words, ‘transformative effects’ – must be treated with caution.
Still, some important questions about how far this evidence takes us remain. Whilst we might know something, on the basis of the studies reviewed here, about what the impacts of social funds and CDD programmes look like – about whether or not they are capable of producing particular effects – we are left with an incomplete understanding of how and why such changes are occurring. Without these kinds of knowledge, researchers are not in a good position to advise others on ‘what works’.
There are a couple of reasons that help explain why our understanding currently falls short. First, and as already discussed, there is sometimes a failure to take seriously the importance of programmatic and contextual dynamics. Indeed, it is idiosyncrasies such as these that are central to understanding why a particular intervention is successful in a particular place at a particular time. As Pritchett and Sandefur (2013: i) have recently argued, there is a ‘need to evaluate programs in context, and avoid…putting most (or all) weight on the most “rigorous” evidence with no allowance for context’. In other words, context is not simply a potential confounding factor that needs to be controlled for, but is rather a core part of any explanation of impact. Taking this seriously in an impact evaluation is one thing; doing justice to it in a review of multiple studies is quite another.
Second, the relatively small number of rigorous studies available, their spread across highly variable contexts, and the diverse nature of many CDD programmes (meaning that studies often examine distinct components of the same or different interventions) makes drawing generalisable conclusions about the effectiveness of social funds and CDD programmes in conflict-affected situations profoundly problematic. When faced with a heterogeneous body of studies – studies which may be looking at quite different aspects of social funds from one another, and which have to deal with different contextual circumstances – how do we then ‘aggregate up’ to construct grander statements on the effectiveness of these instruments? This is not a trivial question – and it is not clear what the answer is.
And third, while it is certainly clear that our understanding of social funds and CDD programmes would benefit greatly from a greater number of high quality, timely and longer-term evaluations (Wong 2012: 52), it is also important that we are able to adequately theorise and explain the mechanisms that lead to change (and to identify whether these look qualitatively different in places affected by conflict). If we are interested in understanding causality, then simply saying that intervention x produces (or not) effect y is insufficient. It is only through a careful examination of mechanisms – and the conditions and contexts which enable them – that we will be able to truly understand impact.