Efficiency, Legitimacy, and Impacts of Targeting Methods: Evidence from an Experiment in Niger

Abstract

1. Introduction

The targeting of social programs is a topic of active policy debates and research. Recent work has discussed when targeted programs or universal approaches may be preferable (Brown, Ravallion, and van de Walle 2018; Hanna and Olken 2018; Banerjee, Niehaus, and Suri 2019; Gentilini et al. 2020).¹ The performance of methods to select beneficiaries is a key consideration. Although achieving full coverage of eligible populations is a widespread aspiration, many countries have limited budgets for social spending. Even after applying geographical targeting criteria, methods to select beneficiary households are commonly required.

Analyses of targeting performance often focus on efficiency, that is, whether the pre-program profile of the selected population matches intended target groups (e.g., Coady and Skoufias 2004; Coady, Grosh, and Hoddinott 2004a; Coady, Grosh, and Hoddinott 2004b; Del Ninno and Mills 2015; Klasen and Lange 2016; McBride and Nichols 2016; Stoeffler, Mills, and del Ninno 2016; Bah et al. 2018; Brown, Ravallion, and van de Walle 2018). For instance, “inclusion errors” quantify the likelihood that selected households have welfare above a given threshold before program roll-out.

Few studies directly compare alternative targeting methods based on their implementation in real-life settings. Experiments testing different targeting methods are particularly rare: Alatas et al. (2012) compare proxy-means testing (PMT) and community-based targeting (CBT), while Alatas et al. (2016b) compare PMT and self-selection in Indonesia. Some comparative studies are based on the implementation of one targeting method and the simulation of another, such as PMT and CBT in Honduras and Peru (Karlan and Thuysbaert 2019), or PMT and selection by community leaders in Malawi (Basurto, Dupas, and Robinson 2019).

This study presents the results of a large-scale targeting experiment embedded in a national cash transfer program. It provides a comprehensive assessment of targeting performance by analyzing targeting efficiency, targeting legitimacy and the impact of targeting choice on program effectiveness.

The experiment compares two of the most popular methods, PMT and CBT, as well as another method to identify temporarily food-insecure households (FCS). Specifically, 318 villages eligible for a cash transfer program in Niger were randomly assigned to have beneficiary households selected by CBT (104 villages), PMT (105 villages), or FCS (109 villages).

For each method, the study measures targeting efficiency by comparing pre-program welfare indicators between selected and nonselected households. It then assesses legitimacy and estimates how program welfare impacts vary across targeting methods using data collected after program roll-out.² It also considers a range of mechanisms that can explain differences in the relative efficiency and legitimacy of targeting methods, in particular how imperfect information, manipulation and local inequality affect CBT.

PMT, FCS, and CBT can theoretically lead to the selection of beneficiaries with different profiles. When assessing targeting efficiency, a relevant welfare benchmark needs to be identified. Through CBT, local populations select households based on their own conception of poverty, which may not match standard welfare benchmarks such as consumption or food security. PMT seeks to target households with low average consumption per capita, which captures a relatively stable measure of welfare. In contrast, FCS aims to target temporarily food-insecure households by proxying the food consumption score³ measured during the lean season. This approach is akin to the Household Economy Analysis (HEA), which is widely used by humanitarian actors.⁴ Food security fluctuates and is more volatile than consumption. As such, the FCS method attempts to account for transient factors related to shocks or seasonality, which PMT may not capture (Del Ninno and Mills 2015). Differences in the profile of households selected by each method can affect targeting efficiency, targeting legitimacy, and program effectiveness.

Considering the legitimacy of targeting methods is relevant for several reasons. Measures of legitimacy obtained after program roll-out can complement subjective well-being indicators to assess which targeting methods best match definitions of poverty held by local populations. Yet targeting legitimacy may depend on the targeting process, not solely on the targeting results. Populations and policy makers may particularly value selection processes that are transparent, participatory, and free of manipulation. CBT has the distinct feature of relying on a participatory process. Formula-based targeting methods, such as PMT and FCS are sometimes considered as lacking transparency (Hanna and Olken 2018). Ultimately, targeting legitimacy (or illegitimacy) can affect social cohesion at the local level and influence broader political support for government programs.

The choice of targeting methods may also determine program effectiveness, that is, the magnitude of program impacts on outcomes related to program objectives. Targeting is only a means to effectively achieve program objectives, such as reducing poverty or improving food security. Ravallion (2009) highlights the limits of targeting efficiency measures, which have little correlation with the poverty impacts of a large social program in China. Basurto, Dupas, and Robinson (2019) and Banerjee, Niehaus, and Suri (2019) even suggest trade-offs between targeting efficiency and program effectiveness. Basurto, Dupas, and Robinson (2019) find that local leaders are less efficient at allocating subsidies to the poor but are able to identify individuals for whom returns are higher. The present study tests whether the impacts of a cash transfer program vary across villages assigned to different targeting methods.⁵ It uses a difference-in-differences strategy that compares changes in baseline and post-program outcomes related to (chronic) food security, consumption, and poverty between beneficiaries selected by each method.

Results show that PMT is more efficient than other methods in identifying households with higher poverty rates (by 5–8 percentages points) and lower consumption per capita (by 10–13 percent). The targeting methods have similar efficiency with respect to other welfare measures. CBT and PMT are closely related to subjective well-being indicators that proxy perceptions of poverty, but FCS less so. Findings on legitimacy show that nonbeneficiaries consider the use of formula-based methods (PMT and FCS) more legitimate. Nonbeneficiaries are 9–14 percentage points more likely to want the PMT and FCS approaches to be repeated in the future, relative to CBT. An analysis of mechanisms explains why CBT is not necessarily considered the most legitimate by local populations. Some committee members attempt to manipulate the CBT process, and individual committees generate substantial exclusion errors due to imperfect local knowledge. Households are aware of manipulation risks, with almost half of respondents reporting that committee members try to benefit themselves. Finally, program impacts tend to be larger among households selected by PMT than CBT. For instance, impacts on food stock are larger by 20 percent and impacts on poverty reduction larger by 9 percent for cash transfer beneficiaries in PMT villages compared to CBT villages.

The findings highlight challenges with targeting in low-income settings where safety net coverage is constrained by limited funding. Variations in performance across targeting methods are not large in the study setting, but the results point to some gains when using PMT. Trade-offs between targeting efficiency, legitimacy, and program effectiveness are not found. The results call for particular caution when considering CBT due to information imperfection and risks of manipulation. One caveat of the study is that baseline measures of food security were collected slightly after the end of the lean season. This limits the assessment of the performance of FCS, including its efficiency in reaching temporarily food-insecure households. As such, conclusions on the FCS approach are less clear-cut.

2. Study Design and Implementation

Context and Niger Safety Nets Program

The challenge of selecting beneficiaries for social programs is particularly salient in low-income settings, including in the Sahel. Niger is among the poorest countries in the world, with a poverty incidence of 51.4 percent in rural areas (World Bank 2017) and widespread food insecurity (Schnitzer 2019). Social protection programs in Niger are thin and fragmented. A large share of resources for social programs is allocated to emergency responses implemented with relatively weak government coordination. The total amount of funding for social programs can cover only a small share of the population in need of support. There are recurring questions on how best to select safety net beneficiaries. In this context, a better understanding of the performance of alternative targeting methods can facilitate coordination as part of a national social protection system.

In 2011, the Government of Niger, with support from the World Bank, set up a national safety nets project. It included a cash transfer program providing small, regular transfers of 10,000 FCFA per month (or approximately $17.2),⁶ for 24 months, to poor households. The duration of the program contrasts with temporary humanitarian interventions providing cash transfers for three to four months during the lean season. By offering small regular cash transfers over a longer period of time, the program seeks to improve food security in the short term, while providing consumption support, reducing poverty, and facilitating investments in human capital and income-generating activities over the medium term. The cash transfer program was designed based on a pilot initiated in 2010 (Stoeffler, Mills, and Premand 2020), and has expanded over time to reach 100,000 households by 2018.

The present study is a large-scale randomized experiment that tests alternative targeting methods. The experiment was embedded in the second phase of the cash transfer program implemented from 2016 to 2018 through a collaboration with the government of Niger. This phase took place in five regions that host 95 percent of the country's poor population (World Bank 2011).⁷ Geographical targeting identified 38 rural communes with the largest poor population in these five regions.⁸ Within these communes, the number of villages to be served by the program was determined to cover on average 40 percent of households in participating villages. All villages were considered eligible, and, to ensure transparency, beneficiary villages were selected through public lotteries (Premand and Barry 2020).⁹ The geographical targeting process to select communes and villages was applied everywhere and is not the focus of this paper.

Targeting Experiment Design and Implementation

The targeting experiment was implemented in 18 communes,¹⁰ where 356 villages were selected to participate in the program; 318 villages were drawn to be part of the targeting experiment.¹¹ In all villages, a registry census was collected door to door among all households. Private firms were hired to collect the census based on a brief two-page questionnaire. The questionnaire contained information to compute PMT and FCS scores for each household, as further described below.

The registry census was used to determine a quota of beneficiary households per village. This was done based on (1) a quota of beneficiaries per commune depending on the estimated poor population in the commune, and (2) the application of a PMT formula to the registry census to allocate beneficiaries in the commune between villages.¹² As a result of this process, program implementers knew how many beneficiaries needed to be selected in each village.

The 318 study villages were then randomly assigned to have beneficiary households selected through one of three methods: community-based targeting (CBT, 104 villages), a proxy-means test formula designed to identify the poor (PMT, 105 villages), or an alternative formula designed to identify temporary food-insecure households (FCS, 109 villages). The randomization was performed in Stata and stratified by commune. To ensure balance, within each commune, randomization was further stratified using a PCA index based on village population, village average PMT score, and Gini coefficient.¹³ These stratification variables were obtained from the registry census.¹⁴ As a result of this process, program implementers knew which method had to be used to select beneficiaries in each village. Each village was informed one by one about the targeting steps that would be used in their specific case. This was done when it was time for the next step in the targeting protocol to be implemented.¹⁵

Implementation of Community-Based Targeting (CBT)

The CBT method relies on village members to select beneficiaries (Coady, Grosh, and Hoddinott 2004a; McCord 2013). In the context of the study, it was implemented through a participatory wealth ranking process. Villagers were asked to rank households according to their poverty level based on well-defined protocols. In each village randomized to have beneficiary households selected by CBT, a village-wide assembly was organized by trained facilitators to provide information about the process. To ensure that information was consistently explained throughout, scripts with key messages were developed in local languages. They aimed to (1) recall the objectives and key features of the cash transfer program; (2) explain the objectives and process to identify program beneficiaries; and (3) guide the community in selecting three committees to perform CBT. Protocols were put in place to maximize participation during the assembly. In the assembly, and with guidance from the facilitator, all villagers were asked to jointly form three committees of five members each. Committee members were chosen on the basis that they were well known and trusted. The three committees include a group of leaders (with at least two women), a group of nonleader women, and a mixed group of nonleaders (with at least two women).¹⁶

Once the three committees were formed, each committee conducted its ranking independently with the support of a trained facilitator.¹⁷ The discussion started by asking members to define the usual characteristics of households that have a low, medium, and high capacity to satisfy basic needs during the entire year.¹⁸ Next, the committees were given a set of pre-printed cards. Each card included the name of the household head (and adult household members) of one household visited during the registry census.¹⁹ The order of the cards was randomized: each card included a random number, and the cards were sorted from the smallest to the largest number. The name on each card was read aloud one by one in the pre-defined random sequence. After each card was read, committee members were asked to place it on a long board containing three different colors (red, yellow, and green), capturing the perceived capacity of each household to satisfy needs during the entire year. When disagreements were observed, the facilitator would recall the characteristics of “poor households” defined during the discussion to help members find a consensus.

Once each committee completed its ranking, the information was compiled by the national safety nets unit. An average ranking was calculated for each household.²⁰ Based on this process, within each village, households were ranked according to their average score. This ranking led to the identification of a list of beneficiary households to be registered in the program, that is, households with average ranks lower or equal to the number of beneficiaries to be selected based on the quota for each village.

Implementation of Targeting by Proxy Means Testing (PMT)

The Proxy Means Testing method relies on a formula approximating household consumption based on a limited set of household characteristics (Grosh and Baker 1995). The characteristics include variables such as household demographics, dwelling characteristics, land ownership, and assets.²¹ The PMT formula was developed based on a national household survey (LSMS) collected in 2011.²² The survey included two rounds of data collection, the first during the lean season, and the second post-harvest. The formula was derived from a stepwise regression starting from a broad set of variables including household demographics, dwelling characteristics, land ownership, livestock, and assets. It identified the variables that best predict average consumption²³ between the two rounds.²⁴ The PMT formula was then applied to the registry census data to calculate a PMT score for each household. Within each village, households were ranked according to their PMT score, which together with the village quotas determined a list of households to be registered in the program.

Implementation of Targeting by Food Insecurity Proxy Formula (FCS)

Many humanitarian actors use a method called Household Economy Analysis (HEA), which aims to identify households that are temporarily food insecure. Schnitzer (2019) shows that the formula on which the HEA method is based can be further improved to identify households at risk of temporary food insecurity.²⁵ This formula proxies the food consumption score indicator during the lean season and constitutes the third targeting method tested in the experiment (FCS). The food consumption score captures dietary diversity based on a set of questions asking about the consumption frequency of different food groups during the past seven days (World Food Programme 2008).²⁶ This indicator is widely used by agencies responding to food crises and aims to reflect both the quantity and quality of food consumed.

The FCS proxy formula aims to predict temporary food security in the lean season based on a limited number of easily verifiable variables. As for the PMT method, the formula was obtained from the nationally representative 2011 LSMS survey. The formula was derived from a step-wise regression model starting from a broad set of variables including household demographics, dwelling characteristics, land ownership, and assets. The formula contains the variables that best predict households at risk of temporary food insecurity based on the food consumption score measured in the lean season. The formula was then applied to data from the registry census to obtain a proxy FCS score for each household.²⁷ Together with the village quotas, the ranking of households based on this score produced a list of households to be registered in the program.

Registration of Selected Beneficiaries

After the safety nets unit produced a list of beneficiaries to enroll in the program based on the targeting method assigned to each village, implementation teams went back to each village to communicate the results and register beneficiaries. The registration phase started with a village assembly, when the list of selected households was read to the community. During this assembly, community members could publicly object to the inclusion of households, though in practice very few did.²⁸ Based on the list of beneficiaries validated by the community, selected households were registered.

Timeline and Data

The study took place between August 2015 and May 2017. Figure 1 illustrates the timeline. The first source of data for the experiment comes from the registry census collected in all households in villages that were selected to participate in the cash transfer program. It included data on a limited set of variables needed to apply both the PMT and FCS formulas. In total, 54,051 households were surveyed between August and November 2015. The registry census was collected between the last month of the lean season and the first month of the post-harvest season.

Figure 1.

Study Timeline

Source: Authors’ presentation of study timeline. Rows contain: year, month, season, targeting steps, data collection and program implementation steps.

Open in new tabDownload slide

A baseline survey was collected shortly after the registry census and before any beneficiary selection process was implemented. It included a sample of 12 households per village randomly drawn from the registry census: 3,816 households were sampled and 3,496 interviewed.²⁹ The baseline survey contains detailed information on several welfare dimensions (discussed further below in section 3), including consumption, food insecurity, anthropometrics (for children under the age of five), assets, household demographics, education and health status of household members. Note that the baseline survey took place between September and December 2015. The survey was collected during the harvest and post-harvest season, which is a limitation of the study since temporary food insecurity during the lean season is not observed. The registry census and baseline surveys were collected before the randomization of targeting methods, which prevents differential response bias across villages assigned to each method.

Following the baseline survey, targeting protocols were implemented. The CBT process was implemented between December 2015 and January 2016. The PMT and FCS formulas were applied in parallel. The communication of results took place during registration assemblies (March to April 2016). Out of the 3,496 baseline households, 1,439 were selected to receive cash transfers for 24 months. The cash transfer payments started in June and July 2016.

Three to five months into the cash transfer program (between October 2016 and January 2017), a first follow-up survey was collected to analyze the legitimacy of the various targeting methods, as well as to assess short-term impacts of the choice of targeting methods on a limited set of outcomes related to food security. The survey took place in a random subsample including 162 villages, or approximately half the villages in the baseline sample.³⁰ Eight households were randomly drawn from the baseline sample in each village, including four beneficiary and four nonbeneficiary households.³¹ The first follow-up survey sample included 1,296 households: 96.5 percent were successfully interviewed, including 611 beneficiary households and 639 nonbeneficiary households.

Lastly, a second follow-up survey was collected approximately one year after the cash transfers started (in April–May 2017). While the second follow-up survey was not collected specifically for this study,³² it contains additional information on household welfare that is helpful to assess whether program impacts on consumption per capita and poverty vary across targeting methods. The sample for the second follow-up survey included 1,280 beneficiary households randomly drawn from the baseline.

Administrative data on costs, and household-level information on registration and payment status, as well as monitoring data from CBT implementation complement the survey data.

Baseline Balance

Results for the full baseline sample are presented in table S3.1 in the supplementary online appendix, available with this article at The World Bank Economic Review website. Beneficiary households represent 40 percent of the sample. The balance tests show that the randomization worked as intended. There is only a small number of statistically significant differences across villages assigned to different targeting methods.³³ The baseline data also highlight that the sample is overwhelmingly poor, with 85 percent of households below the national poverty line. Food insecurity affected 12 percent of households during the harvest or post-harvest season. Households reported that their own agricultural production covered food needs for an average of 3.4 months during the year. While the overall level of inequality is low (with an average Gini of 0.24), there is variation across villages, with village-level Gini coefficients ranging from 0.1 to 0.4.³⁴

3. Efficiency of Targeting Methods

This section documents the relative efficiency of the three targeting methods. The objective is to understand if there are differences in the pre-program profile of beneficiaries selected by each method, and if so, along which welfare dimension. The analysis of efficiency can also provide information about the implicit welfare benchmarks used by local populations when they select beneficiaries through CBT, and whether they differ from other targeting methods.

Equation (2) is estimated for a range of welfare measures collected at baseline, including household-level indicators (panel A), children indicators (panel B), and subjective well-being indicators (panel C). The  annex contains a detailed definition of each measure. Household-level welfare measures include consumption per capita, consumption per adult equivalent and consumption with adjustment for economies of scale, as well as a poverty rate calculated based on consumption per capita. Consumption includes annualized expenditures and self-consumption for food, nonfood (including health and schooling) and durables.³⁶ Other household-level welfare benchmarks include the food consumption score, asset and livestock indices,³⁷ income (monetary earnings) per capita and food stock coverage.³⁸ Children indicators in panel B include the incidence of severe stunting, wasting, and underweight for children under the age of five. Subjective well-being indicators include dummies capturing if a household (1) is perceived as very poor by other community members³⁹, and (2) has a low self-reported ability to satisfy basic needs.⁴⁰

Results show that PMT is more efficient than FCS and CBT in identifying households with higher poverty rates and lower consumption per capita.⁴¹ Households selected by PMT have a poverty rate higher by 8 percentage points than households selected by CBT, and higher by 5 percentage points than households selected by FCS. Similarly, households selected by PMT consume 15 KFCFA (or 13 percent) less than households selected by CBT, and 11 KFCFA (or 10 percent) less than households selected by FCS.⁴² It shows that, even if the population is largely poor, there are meaningful welfare differences between households selected by alternative methods. The FCS and CBT methods perform similarly relative to poverty and consumption welfare benchmarks.

Figure 2 plots the likelihood of a household being selected as a beneficiary by each method along the baseline consumption distribution. The figure illustrates that PMT is relatively more efficient at including the poorest and excluding the better-off, based on consumption per capita. In contrast, the performance of CBT and FCS is rather flat along the consumption distribution.

Figure 2.

Targeting Efficiency along the Consumption Distribution

Source: Baseline Survey.Note: The graph displays the probability of households to be selected as beneficiary (y axis) depending on their ranking along the distribution of baseline consumption per capita (percentiles in x axis). This is plotted separately for each targeting method: community-based targeting (CBT), proxy means testing (PMT) or a formula to proxy temporary food insecurity (FCS). The graph is based on a semiparametric regression model (Yatchew 1998) where commune and village fixed effects enter linearly in the functional form.

Open in new tabDownload slide

When using welfare measures other than consumption, no large difference is found in the efficiency of the three targeting methods. Table 1 shows that there are fewer differences in average characteristics of beneficiary households selected by each method across other welfare benchmarks such as (chronic) food insecurity, assets, livestock, income per capita, food stock coverage, as well as stunting, wasting and underweight of children under the age of five.⁴⁶

The finding that methods perform similarly based on (chronic) food insecurity may be explained by the fact that the FCS method was designed to predict the food consumption score during the lean season, while the baseline survey measures the food consumption score during the harvest season. (This was due to delays in the baseline data collection resulting from implementation delays in the registry census). This finding is consistent with Schnitzer (2019), who finds that HEA performs much better than PMT in proxying (temporary) food insecurity during the lean season, but does less well in proxying (chronic) food insecurity in the harvest season. This is related to the limited correlation between households experiencing (temporary) food insecurity in the lean season and (chronic) food insecurity in the harvest season (which is less prevalent).

It is noteworthy that no significant difference in subjective well-being indicators is found between PMT and CBT, although FCS is less aligned with subjective well-being. For instance, among beneficiary households selected by CBT, 62 percent are perceived as very poor by other community members, and 64 percent have a low self-reported ability to satisfy basic needs. These indicators are not significantly different for households selected by PMT but are lower among households selected by FCS. These results contrast with those from Indonesia, where CBT was found to perform considerably better than PMT in matching community perceptions of poverty (Alatas et al. 2012). Determinants of targeting efficiency are further analyzed in section 5 below.

The PMT method leads to the selection of households with higher poverty and lower consumption, as well as lower income per adult equivalent and food stock. Overall, the mean consumption per person of PMT-selected beneficiaries is 49 cents a day, compared to 63 cents for those not selected. While in absolute terms the difference may seem small, it represents almost 30 percent of beneficiaries’ consumption. PMT also selects households with a slightly higher share of underweight children than nonbeneficiaries.⁴⁷ The CBT process leads to the selection of households with fewer assets, fewer livestock, lower income per adult equivalent, and lower food stock. The FCS method leads to the selection of households with fewer household assets.⁴⁸ CBT, PMT and to a lesser extent FCS select households with lower subjective well-being indicators. For instance, beneficiary households selected by CBT, PMT, respectively FCS are 28, 25, respectively 19 percentage points more likely to be considered very poor by other community members than nonbeneficiary households.

4. Legitimacy of Targeting Methods

Overall, legitimacy appears relatively high across targeting methods (table 3):⁵² 81 percent of sample households are very satisfied with the targeting process, with no statistical difference across methods; 80 percent of respondents state that they would like the same approach to be used again in the future, with few significant differences across methods, except for a small preference for FCS over CBT. Inclusion and exclusion issues are reported, but again with little differences across targeting methods. On average, 28 percent of respondents report that at least one nonpoor household has been selected. Exclusion issues are more prevalent, with 78 percent of respondents reporting that at least one poor household has been excluded.⁵³ The only significant difference is that more inclusion issues are reported with PMT than CBT.

However, legitimacy is much higher among beneficiaries (table 4) than among nonbeneficiaries (table 5): 99 percent of beneficiaries are “very satisfied,” compared to only 65 percent of nonbeneficiaries; 95 percent of beneficiaries would like the same targeting approach to be used in the future, but only 65 percent of nonbeneficiaries. Perceptions of accuracy differ less, but nonbeneficiaries are also slightly more likely to report that there are poor, respectively nonpoor households wrongly targeted.

Importantly, there are sharper differences in legitimacy and perceived accuracy of targeting methods among nonbeneficiaries (table 5). Nonbeneficiaries are particularly unlikely to want the CBT approach to be repeated: they are 9.5 and 13.9 percentage points more likely to want to repeat the PMT, respectively the FCS approach.⁵⁴ This holds even though perceptions of accuracy favor CBT over PMT: in PMT (respectively FCS) villages, nonbeneficiaries are 16 percentage points (respectively 9 percentage points) more likely to say that there was a nonpoor household selected than nonbeneficiaries in villages where the CBT approach was used. This suggests that higher legitimacy is related to the targeting process used with formula-based methods, rather than the precision of targeting results. No significant difference is found across methods in terms of legitimacy and perceived accuracy among beneficiaries (table 4).⁵⁵

5. Determinants of Efficiency and Legitimacy

This section considers mechanisms that can explain differences in the relative efficiency and legitimacy of targeting methods. First, it tests whether imperfect information and manipulation affect CBT performance. The results of CBT are sometimes considered as revealing local populations’ own welfare rankings. The ability of CBT to consider local knowledge is widely considered a key advantage. However, information failures may arise when local populations do not have full information on other households’ welfare levels.⁵⁶Manipulation (of which elite capture is a form) is a common concern with CBT (Conning and Kevane 2002), though evidence is mixed (McCord 2013).⁵⁷ This section shows that these factors can explain why CBT is not necessarily perceived as the most legitimate process in the study context. The section then explores whether targeting legitimacy varies by local levels of inequality. On the one hand, Bardhan and Mookherjee (2000) show that elite capture is more likely in high-inequality settings. On the other hand, CBT might not be able to identify households with different welfare levels in low-inequality settings where there are few observable welfare differences across households. Lastly, a range of other potential determinants of targeting performance are considered, including fatigue in CBT, the ability of methods to take into account the effects of shocks, as well as implementation issues in formula-based methods.

Imperfect Information

Manipulation

Another concern with CBT is that committee members may manipulate the selection process and try to benefit themselves. The ranking provided by each of the three independent committees can be analyzed to assess whether manipulation occurs. This would arise if committee members’ households have a higher chance to be selected as beneficiaries. Estimation controls for baseline household welfare measures.⁵⁹

Table 7 shows that some committee members attempt to manipulate the process. After controlling for baseline welfare measures, the probability that committee members’ households are selected is higher than for other households. It is statistically significant for two of the three committees: households of the mixed nonleader and leader group have a 24.1 and 16.4 percentage point higher chance to be selected as beneficiaries, respectively. No significant manipulation is observed in the women committee. Triangulating rankings across several committees only partly mitigates manipulation. Based on the average ranking, committee members’ households have an 11.7 percentage point higher chance to be selected as beneficiaries (significant at the 1 percent level).⁶⁰

These results can explain why CBT was not considered the most legitimate process by nonbeneficiaries. Results are also consistent with respondents’ perceptions: since 48 percent declare that committee members try to benefit themselves. Households displaying low trust (by believing committee members try to benefit themselves) are also 12 percentage points less likely to want to repeat the CBT approach.

Variation in Targeting Performance by Local Inequality Levels

Empirical studies are rarely able to analyze whether local levels of inequality affect targeting performance. This section assesses variations in targeting efficiency and legitimacy between low and high-inequality villages. This is done by splitting the sample at the median of the village-level Gini coefficient from the registry census.

Figure 3 illustrates results for inclusion errors. The efficiency of formula-based methods (particularly PMT) remains similar at low and high levels of inequality. In contrast, the results of CBT vary strongly by levels of inequality: inclusion errors are 14 percentage points higher in low-inequality than in high-inequality villages. CBT is as efficient as formula-based methods in high-inequality villages, but more inefficient in low-inequality villages.

Figure 3.

Inclusion Errors (by Village Inequality Levels)

Source: Baseline Survey.Note: This figure displays inclusion errors for each targeting method (CBT: community-based targeting, PMT: proxy means testing or FCS: formula to proxy temporary food insecurity) between low- and high-inequality villages. Low-inequality villages have Gini coefficient below the median. Gini coefficients are estimated from the registry census.

Open in new tabDownload slide

Tables 8 and 9 show how legitimacy varies between low- and high-inequality villages. The higher legitimacy of formula-based methods may be partly driven by low-inequality villages, where households assigned to formula-based methods are 5 (marginally insignificant) and 7.8 percentage points more likely to prefer repeating PMT, respectively FCS compared to households assigned to the CBT approach. In contrast, legitimacy is found to be similar across methods in high-inequality villages. Relative to CBT, perceived inclusion and exclusion errors also tend to be more consistently significant for PMT and FCS in high-inequality villages.

Results can be interpreted as consistent with CBT committee members being better able to identify worse-off households in higher-inequality settings. They are not consistent with higher elite capture in high inequality settings (as in Bardhan and Mookherjee 2000).

Other Factors: Fatigue, Shocks, and Implementation Issues in Formula-Based Methods

A range of other factors can affect targeting performance. Alternative explanations are explored in the supplementary online appendix. First, no statistically significant effects of fatigue are found in CBT. Second, some targeting methods may in principle better account for transient poverty factors. However, no differences are found across methods in the probability of excluding households exposed to shocks in the study setting. Finally, potential implementation issues with formula-based methods are found to have little effects on targeting efficiency.

6. Impacts of Targeting Methods on Program Effectiveness

This section complements the analysis of targeting efficiency and legitimacy by testing whether the choice of targeting method affects program effectiveness. Specifically, it analyzes whether the impacts of the cash transfer program vary across villages assigned to different targeting methods. The focus is on key welfare indicators related to core program objectives, including proxies for food security in the short term, as well as consumption and poverty. This section does not aim to estimate program impacts directly,⁶¹ but rather to highlight if there are differences in program impacts across targeting methods.

The impacts of targeting choice on program effectiveness are measured for outcomes collected in the first and second follow-up surveys. The first follow-up survey took place a few months after the start of the cash transfer program. It captures four proxy measures of food security: a (chronic) food insecurity dummy, the food consumption score, the number of months household food stock could last (food coverage), and a hunger scale (see  annex for detailed description of each indicator). It also includes an indicator measuring whether any conflict was reported between the household and another community member. This is to assess potential unintended effects of targeting choice. The second follow-up survey was collected one year after the start of the program and included outcomes such as consumption per capita and poverty.

The impacts of the cash transfer program may vary depending on the profile of selected households. If there is no trade-off between targeting efficiency and program effectiveness, larger program impacts on welfare would be expected for targeting methods identifying poorer populations.

Table 10 presents the results. No difference is found in the food consumption score, hunger and conflict indicators. However, larger cash transfer program impacts are found on food stock in villages where PMT targeting is used.⁶³ Food stocks are on average 16 percent higher across all households in villages randomized to PMT targeting compared to villages randomized to CBT targeting, and 21 percent higher compared to villages assigned to FCS.