Closing the gaps in experimental and observational crop response estimates: a bayesian approach

Abstract

A stylized fact of African agriculture is that crop responses to inorganic fertilizer application derived from experimental studies are often substantially greater than those from observational studies (e.g. surveys and administrative data). Recent debates on relative costs and benefits of expensive farm input subsidy programs in Africa, have raised the importance of reconciling these estimates. Beyond mean response differences, this paper argues for including parameter uncertainty and heterogeneity arising from variations in soil types, environmental conditions, and management practices. We use a Bayesian approach that combines information from experimental and observational data to model uncertainty and heterogeneity in crop yield responses. Using nationally representative experimental, survey, and administrative datasets from Malawi, we find that: (1) crop responses are low in observational data, (2) there are large spatial heterogeneities, and (3) based on sensitivity analysis, ignoring parameter uncertainty and spatial heterogeneity in crop responses can lead to questionable policy prescriptions.

The best fertilizer on any farm is the footsteps of the owner.

(Taken from Scott 1998, p. 284, attributed to Confucius)

1. Introduction

A stylized fact of African agriculture is that experimentally derived crop responses to inorganic fertilizer application are often substantially greater than those obtained from observational studies (e.g. using farm survey or official administrative data). There is also a long history of description of this yield gap, which can be reduced to the presence or absence of positive or negative confounding factors such as biologically optimal crop management by researchers versus biologically sub-optimal (albeit possibly optimal bio-economic) management by farmers; smaller, more uniform, plot sizes used in experiments versus larger, and heterogeneous plot sizes used by farmers; biases or spatial inconsistencies in site or sample selection of scientific versus farmer plots; and observer bias (see, e.g. Snapp et al. 2014, Coe et al. 2016; Benson et al. 2021). In addition to these factors, sample sizes (or the number of replicates) also differ, with farm surveys that can span thousands of households versus experiments that often include a few hundred sites at most. According to Bullock and Bullock (2000: 97), ‘…the simple fact is that most agronomic experiments are not run for enough years and enough locations to obtain many different observations of weather and possible field characteristics’. Because a field experiment at a few locations cannot capture all this nuance in variation, the representativeness of agronomic experiments is often questioned because crop response estimates and recommendations derived from them are different from what is experienced under farmers’ conditions. The aim of this study is to apply a Bayesian approach that combines experimental and observational evidence thereby providing estimates for making recommendations on fertilizer use that take into account multiple sources of information.

Economists have had long-standing debates on both the causes and solutions related to yield gaps.¹ Some of the early work on this topic includes Davidson and Martin (1965), Davidson et al. (1967), Anderson and Dillon (1968), and Anderson (1992). Such debates have continued in contemporary agricultural policy considerations (see Benson et al. 2021, Benson et al. 2024). This study contributes to this prior literature by using experimental and observational evidence in Malawi to characterize the crop response gaps and applies Bayesian linear and hierarchical models to combine the estimates from observational and experimental studies.

The discrepancy in experimental versus commercial yield response can have profound policy implications. For example, a study by Jayne et al. (2015) using observational crop responses of the social benefits versus costs of the Malawian farm input subsidy program—one of the largest targeted national farm input subsidy programs in Africa—found it to be unduly costly.² In direct contrast, using experimental crop response data, Chirwa & Dorward (2013) found the program to be economically beneficial relative to its costs. According to Jayne et al. (2015), the use efficiency of the nitrogen applied to maize is perhaps the most important factor determining the benefits of the Malawi farm input subsidy program. The crux of their case largely hinges on the following yield response relativities:

‘These (crop response) estimates (3.4–9.9 kg of maize output per unit of fertilizer applied per ha) are based on farm survey data and not researcher-influenced plots, and they reflect the range of management practices and production constraints found within Malawi's smallholder farm sector… Unfortunately, Dorward and Chirwa (2015) maize response estimates of 16–18 kg are derived from researcher-influenced farm trials undertaken in the late 1990s with participants who were largely master farmers’ (Jayne et al., 2015: 746)

While noting the challenges of reconciling these crop responses, Arndt et al. (2016) evaluated the subsidy program using crop responses ranging from 11.8 to 18.5 kg of maize per ha per kg of nitrogen fertilizer. They settled on this range of responses, more or less arbitrarily, to cover reported rates from the observational and experimental evidence they reviewed³. Arndt and co-authors further comment that reconciling the experimental and observational crop responses remains an important and unresolved problem. The difficulty of fully reconciling the estimates from these multiple sources of data, which were collected at different time periods in different locations, with different varieties and using different research methods, is exacerbated by the reliance on mean response comparisons that completely ignore the substantive spatial and temporal heterogeneity in these responses. These concerns remain especially in recent literature which questions the effectiveness of fertilizer subsidies given the low yields, persisting food insecurity, and non-compensatory price ratios (e.g. Benson et al. 2024).

Given these challenges, policy making is usually left to guesswork regarding the true crop responses and a reliance on arbitrary approaches to re-adjusting experimental yield responses to better reflect farmer conditions. In this paper, we apply a simple and replicable method of bringing all these subjective judgements into a formal estimation framework. The approach is based on the Bayesian paradigm of combining prior information and observational data. We use a Bayesian hierarchical model to incorporate both parameter uncertainty and heterogeneity in crop response functions and fertilizer recommendations. The Bayesian approach of combining different evidence on the same phenomenon has recently been used by Fessler & Kasy (2018) to combine predictions of labor demand and wage inequality derived from economic theory and empirically derived estimates, by Meager (2019) to combine results from various randomized control trials of micro-credit interventions across countries, and by Rosas et al. (2018) to impose duality theory restrictions based on experimental trial data to assess market level crop yield responses to prices in the USA. In macroeconomic forecasting, the idea of combining evidence using the Bayesian paradigm has been implemented in Dynamic Stochastic General Equilibrium Modeling as well as New Keynesian Macroeconomics. It has also been used in the economics of education literature to combine teacher value added measures that are precise but biased with alternative measures based on admission lotteries for students that are unbiased but imprecise (Angrist et al. 2017). Other studies focus on using machine learning and quasi-experimental approaches to adjust the estimates from either observational or experimental studies (Bernard et al. 2024).

This paper contributes to the crop responses literature by exploring the possibility of improving soil fertility recommendations through the careful combination of experimental and observational crop response evidence, while also taking into account parameter uncertainty and heterogeneity. Specifically, the study analyzes the effect of observational crop responses when conditioned on prior (experimental) crop responses. The applications from this modeling approach are many, especially given a lack of directly comparable experimental data over time due to changes in experimental designs. Using the Bayesian approach, researchers can simply use previous estimates as priors in a new analysis. Similarly, in many agronomic research projects scientists are asked to conduct household surveys prior to or while conducting experiments as part of learning the environment. With this approach, they can formally use the household survey estimates as priors in their experimental analysis. The key rationale is that Bayesian estimates weight the estimate from the present and prior data using an inverse of the variance parameters so that the uncertainty of the parameters determine whether the prior or present data dominate. We specifically incorporate parameter uncertainty in single output and multi-output crop response function estimation, which provides a more complete description of the crop response parameters. Specifically, we contribute to the on-going debates on the use of experimental versus observational mean crop responses by showing that using the mean response function in combination with arbitrary adjustments may result in suboptimal policy prescriptions in most cases because the inherent unobserved heterogeneity within and across farms requires site-specific optimization. Instead, researchers are likely better off using the entire distribution of the parameters (as this distribution contains more information than the mean), which entails comparing the distributions of benefit-cost ratios and profits obtained from the different alternatives being studied.

Although endogeneity issues from measurement error, simultaneity, and omitted variables require close consideration when estimating crop response functions (and such concerns bedevil all prior crop response assessments that use observational data), the use of district-level fixed effects allows comparisons of within district differences of the sources of evidence, while accounting for uncertainty of parameter estimates, and heterogeneity of crop response estimates. Furthermore, this paper follows a partial identification strategy to test if the crop response parameter is observationally equivalent under various prior specifications.

Debates on whether crop response estimates are low or high in Malawi and other African countries are difficult if not impossible to resolve when uncertainty and heterogeneity of the estimates across time and space is ignored. Most importantly, the results we obtain below show that even with an extremely high prior mean yield response (e.g. 30 kg/ha of maize output per kg of nitrogen (N) fertilizer applied) and level of precision (e.g. a value of 10, which is equivalence to a variance of 0.1), the posterior crop response estimates using observational data can only go as high as 20 kg of maize output for a unit of nitrogen (N) fertilizer applied. In addition, the lowest is around 2 kg of maize output per kg of fertilizer. This implies that there is a 95 per cent probability that the mean crop responses are between 2 and 20 kg of maize output per kilogram of N fertilizer applied.⁴ Further analysis in the paper shows that there is huge spatial heterogeneity in the crop responses, which should be of importance in policy design because some of the districts are non-responsive to fertilizer application.

This evidence therefore suggests that resolving policy debates that depend on crop responses should consider variances and heterogeneity in these responses. In summary, the results illustrate that Malawian maize yield responses are generally low and highly variable (over time and space). This underscores the need for evidence-based targeting of locations and beneficiaries if farm input subsidy programs such as that presently operating in Malawi are to constitute a cost-effective public policy and be profitable for smallholder farmers.

The rest of the paper is structured as follows. We present next the model focusing on the theoretical profitability analysis model and the empirical econometric approach that uses Bayesian analysis. In Section 3, we present the data sources and descriptive statistics. In Section 4, we present the results and discussion. We finally conclude and provide a discussion of limitations and future research in Section 5.

2. Model

2.1 Theoretical model

The standard neoclassical approach to production economics on crop response to inputs like fertilizer is a primal approach based on deterministic profit maximization. Following Hartley (1983), the deterministic conditional neoclassical model of fertilizer usage and output response, given that a positive area of land has been allocated to crop j, assumes that farmers maximize profits with respect to all variable input levels associated with the area of land |${{a}_{ij}}$| which is usually normalized to a unit hectare. The profit associated with crop |${{j}}$| in each plot i is defined as

where |${{{\rm{\pi }}}_{{{ij}}}}$| is the profit per unit (hectare) for each plot i and crop j. |${{p}_{ij}}$| and |${{w}_{ij}}$| are prices of crop outputs and fertilizer respectively. |${{y}_{ij}}$| is the crop specific yield (kg/ha) and |${{y}_{ij}} = {{f}_{ij}}( {{{x}_{ij}},{{a}_{ij}},{{z}_{ij}};{\rm{\ }}{{{\rm{\theta }}}_{ij}}} )$| describes the production technology where |${{x}_{ij}}$| is the quantity of fertilizer applied (kg/ha), |${{a}_{ij}}$| represents area under crop j in plot i, |${{z}_{ij}}$| represents the quantity of other inputs like labor, and |${{{\rm{\theta }}}_{ij}}$| represents the set of relevant response parameters that are usually estimated from the data. |$F{{C}_{ij}}$| represents fixed costs.

Under assumptions of twice continuously differentiability, convexity of the production possibilities set, strict concavity of the objective function, the economic condition for optimality is

Using the implicit function theorem or assuming conventional functional forms for |${{f}_{ij}}( {{{x}_{ij}}^*,{{a}_{ij}},{{z}_{ij}};\ {{\theta }_{ij}}} )$|⁠, it is easy to find the optimal |${{x}_{ij}}^*$| and this approach has been used extensively in practice to make fertilizer use recommendations.

There are fundamental flaws using the neoclassical production model. First, the parameter |${{\theta }_{ij}}$|⁠, which essentially drives the optimality as well as heterogeneity across farms, is assumed to be known and certain such that it is usually not included in the optimization. However, these parameters are rarely if ever known (either in an agronomical or statistical sense), which implies that economic decisions made on the basis of this assumption are suspect. The fundamental problem in agricultural settings is that the crop is typically grown on soils with an ‘inherent soil fertility gradient,’ which implies that yields even under heavily controlled environments will be uncertain because of the unobserved heterogeneity in the soil even a few centimeters apart (Zingore et al. 2007).⁵

In a statistical sense, |${{\theta }_{ij}}$| is usually a set of unknown parameters, about which farmers may have some prior information based on the performance of the same or different crops under similar or different input regimes. In conventional theory, there is no provision to incorporate this prior. Second, this conventional approach does not provide any direction as to what type of data would be required to estimate the production relation |${{f}_{ij}}( {{{x}_{ij}}^*,{{a}_{ij}},{{z}_{ij}};\ {{\theta }_{ij}}} )$|⁠. A researcher can conduct experiments to decipher some |${{x}_{ij}}^*$|⁠, but no known experimental design can comprehensively investigate the effect of each of the |${{x}_{ij}}$| on yield, while also controlling for all other effects in |${{x}_{ij}}$| and |${{z}_{ij}}$|⁠. Occasionally, multi-factorial experiments are conducted to (partially) address this challenge. Another line of research uses farm surveys to analyze the observable determinants of yield. Under this approach, the farmer has already made a set of input and crop management choices depending on their observed and unobserved circumstances. Using these different approaches result in different estimates of |${{\theta }_{ij}}$|⁠. This study applies an extensive Bayesian analysis to investigate combined estimates of |${{\theta }_{ij}}$| that are consistent with theory and the practical challenges (and the relative prices) faced by farmers.

To make comparisons across different scenarios we use first order stochastic dominance, in particular ‘posterior stochastic dominance.’ Thus, different information sources are being combined probabilistically and stochastic dominance is being used to compare among them. Stochastic dominance is normally defined with respect to stochastic outcomes, which in the case of this study are profits. The study therefore concentrates on whether the fertilizer response parameters dominate each other across the entire measured range of fertilizer use when the prior is updated with additional information. Definition 1 below provides a description of posterior stochastic dominance.

 

Using the definition of stochastic dominance and interpretation of posterior parameter estimates as consisting of a prior and a data-based likelihood, two important claims follow when interpreting the prior scenarios. The first claim, based in the mean responses, is that if the mean for a prior is greater than the mean of the likelihood holding the variance or precision parameter constant, then the resulting posterior parameter is greater than the mean of the likelihood. Second, if the variance for a prior is greater (i.e. has lower precision) than the variance of the likelihood assuming the same mean, then the variance for the posterior is less than the variance of the likelihood.

The stochastic dominance ordering is therefore an empirical question that depends on the relative magnitudes of the prior mean and precision versus the mean and the uncertainty of the likelihood. To illustrate the concept of stochastic dominance in comparing the prior scenarios, Fig. 1 demonstrates three hypothetical cumulative distribution functions; F(.), G(.) and Q(.). In the figure, F(.) first order stochastically dominates G(.) since F(.) < G(.) across the entire measured range of profits. Higher order levels of stochastic dominance can be used to compare F(.) and Q(.) or G(.) and Q(.) (see Levy 2016).

2.2. Empirical models

To incorporate the facets of the theory above, we use two estimation strategies, namely a Bayesian linear model and a Bayesian hierarchical model. These models allow estimation of the crop response parameter which is key to the stochastic dominance comparisons in the theory. All the models are quadratic in the crop response parameter.⁶ The choice was made so that the results of the paper are comparable with most of the experimental and observational estimates hitherto reported for Malawi, which have used this functional form (see, e.g. Harou et al. 2017).

2.2.1 Estimating equation: Bayesian linear model

A Bayesian linear model is used to estimate the ray production function (see Online Appendix B). The Bayesian linear model is equivalent to the ordinary least squares (OLSs) regression model when a non-informative prior (e.g. zero mean and an arbitrarily large variance such as 10,000) is used. The Bayesian linear model for the ray production function approach is

where |${{\tilde{y}}_i}$| is the output norm (hereafter referred to as total output index) defined as |$\tilde{y} = {{[ { \sum_{j = 1}^p y_j^2} ]}^{0.5}}$|⁠, and |${{y}_j}$| is the yield (kg/ha) of crop j. When mono-cropped maize is considered, the total output index is equivalent to maize yield|$;{{x}_{ij}},x_{ij}^2,{\rm{\ }}{{z}_{ij}}{\rm{\ and\ }}\lambda $| are vectors of nitrogen (N) fertilizer use (kg/ha), squared N fertilizer use, other explanatory variables (like seed use (kg/ha), rainfall etc.), and angular crop output coordinates (representing crop mix), respectively (see Online Appendix B for details). The corresponding parameters are; |${{\beta }_1},\ {{\beta }_2},\ {\rm{\alpha }},\ {\rm{and\ }}\xi $|⁠. Without loss of generality, we use the matrix notation for |$\beta $| to represent all the parameters and X the design matrix for all variables in the model in the derivations that follow. The disturbance term, |$\epsilon $| has a multivariate normal distribution with mean |$0$| and covariance matrix |${{\sigma }^2}I$|⁠, where I is an identity matrix, i.e. |$\epsilon {{ \sim }^{iid}}\ N( {0,\ {{\sigma }^2}I} )$|⁠. In Bayesian econometric terminology, the variances |${{\sigma }^2}$| can be written as precision estimates, h where |$h = {{\sigma }^{ - 2}}$| (Carlin and Louis 2009). ⁷

Though exact sampling from the posterior is possible, the model was estimated using Markov Chain Monte Carlo using Gibbs Sampling. All the models were run with 11,000 MCMC iterations with 1,000 used as burn in and the remaining 10,000 for posterior analysis. Non-informative priors (0 prior mean and 0.001 prior precision) for the parameter estimates were assumed in the set of models except where stated for the sensitivity analyses. The trace plots for key variables (fertilizer use) in both linear and hierarchical linear models showed convergence at small number of runs as expected for linear models (see Online Appendix Fig. A2 for some of the trace plots. The rest are available upon request).

2.2.2. Heterogeneity in crop responses: Bayesian hierarchical model

The enormous heterogeneity in Malawi's smallholder farming systems implies that even crop response parameters that capture the uncertainty in associated model parameters may not be sufficient to characterize the different biophysical and socioeconomic circumstances faced by farmers. To address heterogeneity in the crop responses we deployed a Bayesian hierarchical modeling approach. According to Carlin and Louis (2009), a hierarchical modeling approach allows for a more explicit assessment of the heterogeneity both within and between groups. This modeling approach has been used extensively in statistics and economics literature to model heterogeneity among individuals. For example, Cabrini et al. (2010) uses the Bayesian hierarchical approach to estimate market performance expectations (e.g. prospective prices) of individuals working in agricultural market advisory services. Chib & Carlin (1999) and Allenby & Rossi (1998) show how the hierarchical model can help in generating consumer and household specific parameters that are useful for marketers of consumer products.

Following Chib & Carlin (1999), consider the normal hierarchical model in matrix notation,

where each group i has |${{k}_i}$| observations. The term ‘group’ is being used generally here so that any type of heterogeneity may be considered. For instance, a group may constitute a location (region/district/village/agroecological zone), household, soil type or poverty status. |${{X}_i}$| is |${{k}_i}\ \times p$| design matrix of p covariates. |$\beta $| is a corresponding |$p \times 1\ $|vector of fixed effects. |${{W}_i}$| is |${{k}_i} \times q$| design matrix. |${{b}_i}$| is |$q \times 1$| vector of subject-specific means and enable the model to capture marginal dependence among the observations on the |$^{}$|ith group. The group-specific random effects follow: |${{b}_i} \sim {{N}_q}\ ( {0,\ {{V}_b}} )$|⁠. And the errors: |${{\epsilon }_i} \sim N( {0,\ {{\sigma }^2}{{I}_{{{k}_i}}}} )$|⁠. Assuming standard conjugate priors, |$\beta \sim {{N}_p}\ ( {{{\mu }_\beta },\ {{V}_\beta }} )$| and |${{\sigma }^2} \sim \textit{InvGamma}\ ( {nu,\frac{1}{\delta }} )$| and |${{V}_b} \sim \textit{InvWishart}\ ( {r,\ rR} )$| where r is set to the number of parameters in the model and R is a diagonal matrix with values along the diagonal equal to the number of parameters (Chapman and Feit 2015). In the estimation, we have used the MCMChregress function which implements the Gibbs sampling algorithm based on algorithm 2 in Chib and Carlin (1999).

3. Data sources and descriptive statistics

3.1 Data sources

The study uses both experimental and surveyed fertilizer response data for maize. In particular, the paper uses evidence from a) the fertilizer verification experimental data collected and analyzed by the Malawi Maize Productivity Task Force in 1995/6–1997/8,⁸ and b) the nationally representative Third Integrated Household Survey data, which were collected between 2010 and 2011 (and reflect production decisions for the 2008 and 2009 farming seasons) across all National Statistical Organization enumeration areas in Malawi.

3.1.1 Experimental data

The study uses geo-referenced on-farm experimental data for the 1995/96 and 1997/8 growing seasons. The trials were carried out as experiments run on farmers’ fields under the auspices of the Malawi Maize Productivity Task Force consisting of national and international experts. More than 1,500 trials were successfully implemented to evaluate six different inorganic fertilizer packages for hybrid maize grown by smallholders across the whole country (Government of Malawi 1997). The distribution of successful trials was unbalanced across the sites/regions and seasons due to statistical and administrative reasons. As reported in Table 1, all six treatments (A, B, C, D, E, and F) were tested in the 1995/96 trials, while four (A, C, D, and E) were tested in the 1997/98 trials. The structuring of treatments in the fertilizer trials suggests that the crop yield consequences of nitrogen and phosphorus may be confounding. That noted, agronomic studies on fertilizer use in Malawi (e.g. Government of Malawi 1997) have argued that nitrogen is the most limiting macro-nutrient, and as such we focus on nitrogen responses.

In each of the two seasons, two hybrid maize varieties were planted; Malawi Hybrid 17 (MH17) was planted in upland sites with historically good rainfall conditions, and MH18 was supplied for trials in lowland areas and at those upland sites in rain-shadow areas. A few sites also tested composite varieties. The soil texture was recorded for each plot for each treatment plot per year, and a standard protocol was followed across all locations to ensure timely weeding, pest management, and other agronomic management activities. According to Benson (1999: 12), one notable feature of the standardized protocol was to conduct the trials on farmer's field that had not received fertilizer or been planted to legumes in the previous two years. The plot size was 6.3 m by 9 m, consisting of seven ridges spaced 90 cm apart. The net harvest plot size was five full ridge lengths, or 1/247 ha (0.00405 ha).

Table 2 includes descriptive statistics for the fertilizer trials in the two seasons. For each of the treatments, yields in 1995/96 were relatively higher than those obtained in 1997/98, reflecting less favorable weather during the 1997/98 season. In terms of treatments, the average yields were highest in treatment E, while the nil N treatment (Treatment A) had the lowest mean yield, which is expected considering that nitrogen fertilization is considered yield increasing, at least when moving from little or no N.

A notable feature of the data summarized in this table, is the large variation in yield responses across each of the treatments. For both seasons, the coefficient of variation for the nil fertilizer treatment (A) are highest, with the lowest variation observed in treatments with the highest amount of nitrogen fertilizer applied (E and F). The variability observed can be attributed to interactions between fertilizer application and many other observed and unobserved factors including location, weather and topography. In this study, we explore the importance of understanding variability attributable to location effects.

3.1.2 Household survey data

The household survey was conducted by the Malawi National Statistical Office (NSO) in collaboration with the World Bank's Living Standards Measurement Survey (LSMS) in 2010. It is a nationally representative sample survey covering about 10,000 households.⁹ The data are analyzed at the crop-plot level to distinguish between input crop responses in single crop versus multi-crop farming systems. The observations pertain to rainy season plots that were owned and/or cultivated by the farm household and that were subject to Global Positioning System (GPS)-based land area measurement. The data files were merged first using the available plot geo-codes and then using household geo-variables (e.g. longitude, latitude, and distance to road). The merging was done in a way that made sure that all the households in the final sample had consistent and identifiable household geo-coordinates. The geo-referenced data allow for the analysis of both agronomic and farmer behavioral responses. The use of this spatially explicit plot level data therefore implies that it is possible to estimate a structural model of multi-crop production enterprises (Fezzi and Bateman 2011). All plots not grown with either maize or a legume were excluded from the analysis. In the final data used for the analysis, there are 19,692 plot-crop observations for five key crops: maize, groundnuts, beans, pigeon peas, and soybeans. This represents 70 per cent of the plot crop observations in the data. These are the major crops for Malawi (accounting for 70 per cent of the country's total cropped area in 2009–2013, Johnson 2016) that are also featured in the integrated soil fertility management literature.

3.1.3 Administrative data

Administrative data were compiled from annual production estimates included in the Ministry of Agriculture and Food Security annual statistical bulletin for the period 1983–2015. These data are reported at the district level and consist of the total hectarage and production and average yield for each crop (i.e. maize, groundnuts, beans, pigeon peas, and soybeans). This source does not report any fertilizer use data by district, and thus these administrative data were only used in calculating cross-district differentials in crop yield performance, a spatial dimension of yield gaps.¹⁰ For maize, the data has varietal (local, composite, and hybrid) specific yield, hectarage and production information.

3.2 Descriptive statistics

Table 3 presents selected descriptive statistics for the various variables characterizing the farm households and plots.

Table 3 shows that almost 90 per cent of the plots in the sample were planted with maize followed by groundnuts (35 per cent). The maize yields are within the range reported in most microeconomic studies. Inorganic fertilizer was used on almost 70 per cent of the plots, while only 12 per cent of the plots received organic fertilizers. The average fertilizer use is about 162 kg/ha (corresponding to 51.37 kg N/ha), which is around the application rate reported for Malawi in other microeconomic studies.¹¹ This figure is higher than in other sub-Saharan African countries possibly because farmers are cultivating very small plots on especially small farms in the context of a generous farm input subsidy program.¹² Additional descriptive statistics are presented in Online Appendix Table A1 in the appendices. On average, the plots are 0.79 km away from the homestead, though with a huge variation across the sample (ranging from 0 to 10 km). The average plot size is 0.44 ha. Most farmers perceive that their plots are either good (45 per cent) or fair (43 per cent) in response to a question about the perceived soil quality. Most of the plots (59 per cent) have soils that are loam (i.e. between sand and clay) which are considered good soils for crop cultivation.

The majority of the households (75 per cent) are male-headed with an average household size of 4.8 people. About 76 per cent of the household heads have had no formal education. Almost 46 per cent of these households are classified as poor, with average household incomes less than MK 37,002 per person per year based on the formal definition of the Malawi NSO. Most of the households live in remote rural areas, about 9 km from a main road and 37 km from the nearest trading center. Online Appendix Fig. A1 in the appendices shows the number of plots planted with each of the crops. Most of the plots are planted with a pure stand of maize followed by a pigeon pea-maize intercrop.

3.3 Experimental and observational yield gaps

The challenge of combining observational and experimental evidence is that it is unlikely that one will find directly comparable treatments, that is, yield responses obtained using similar amounts (and types) of fertilizer grown in the same weather events and similar soil types. The nationally representative datasets available are almost 20 years apart (experiments in the 1990s and surveys in the 2010s). To demonstrate that experimental-observational yield gaps existed in the 1990s when the experiments were being conducted, we compared the district averages from the plot level experimental data with the corresponding hybrid varietal-specific administrative data for each of the two seasons, 1995/96 and 1997/98. Figure 2 shows scatterplots of district level averages of experimental hybrid maize yields for each of the fertilizer treatments (see Table 2) and the corresponding district averages of hybrid maize yields from administrative data in each of the respective seasons.¹³

There are six plots for the 1995/96 agricultural season and four plots for the 1997/98 agricultural season, with each of the plots representing the fertilizer treatments in the experimental evidence.¹⁴ The rays indicate the ratio of experimental to observational yields.

Across all the treatments, experimental yields are more than two times higher than the corresponding farm yields reported in the administrative database. As expected, the experimental-observational yield gaps increase as the amount of nitrogen applied in the experimental data increases. According to the Government of Malawi (1999), the 1997/98 agricultural season was a bad maize-growing year in that some districts experienced drought. This is especially evident in Fig. 2b where the yield gaps for the no-fertilizer treatment are much lower. This highlights that the gap between observational and experimental maize yields are affected by environmental and climate conditions.

4. Results and discussion

4.1 Overview of the existing maize crop response literature for Malawi

The research on crop responses in Malawi dates back to at least the 1960s. Blackie M. et al (1998) reported estimates of experimental maize responses in studies conducted from the 1960s to 1998 ranging from 23.1 to 34 kg of maize per unit of additional applied nitrogen. Table 4 below taken from Arndt et al. (2016: supplemental material) reports the microeconomic evidence on the marginal returns to fertilizer use for selected types of maize seed. The mean maize responses range from 2.8 to 15 kg/ha for observational studies, much lower than the 23–34 kg/ha range reported in the experimental research. Informed by this prior evidence, below we use maize yield responses in the range of 0–30 kg/ha of maize for an additional kg of applied nitrogen as priors by which to anchor the assessment of estimates in prior studies.

In all the prior published assessments of both the experimental and observational maize yield response estimates for Malawi and sub-Saharan Africa, only mean responses were reported, absent any measures of the associated variation or uncertainty in these reported responses.¹⁵ However, to compare across studies and to make sense of these mean crop response parameter estimates, one cannot ignore the associated measures of precision.

4.2 Experimental, observational and Bayesian crop responses

In this section, we present the results from a Bayesian linear model (with results that are the same as using an OLSs on equation 3). The set of results (see details in Online  Appendix Tables A2 and A3) show the ray production functions for maize intercropped with either groundnuts, beans, pigeon peas, or soybeans. The variables of interest in the production functions include N fertilizer, N fertilizer squared and coordinate angles, the latter representing the crop output mix. The coefficients for the polar coordinate angles are negative for all maize-legume combinations (see Table A3 in the appendices). This implies that an increase in the output mix reduces the total output index, meaning that the total output index is lower when maize is intercropped with a particular legume. We estimate that the mono-cropped maize response to N fertilizer application is about 10.56 kg/ha per kg of applied nitrogen, with a 95 per cent credible interval of 9.78–11.36 kg/ha (see Table 5 and Online Appendix Table A2). The experimental maize responses are about two times higher at 20.58 kg/ha per kg of applied nitrogen, consistent with finding of Anderson (1992) who observed that

‘There is a systematic overstatement of the extent of responsiveness of crops to applied fertilizer in Africa, relative to what is achievable under most farm conditions. The extent of overstatement is of the order of a factor of, say, two in terms of incremental response ratios.’ Anderson (1992: 393).

Given these results, we can combine the experimental coefficient and the observational coefficient by simply using the experimental estimate (25.41) and its standard deviation (0.41) as the prior in a regression using the observation data. Figure A3 shows that the resulting posterior distribution of the N coefficient (i.e. median: 12.01, 95 per cent credible interval: 11.53, 12.50) is still closer to the distribution of N responses derived from the observational estimates (i.e. median: 11.09, 95 per cent credible interval: 10.21, 11.99) that the distribution derived from the experimental results.

4.3 Bayesian analysis with sensitivity testing

The foregoing analysis documents the crop response gaps and the hybrid crop responses when using particular experimental and observational data. It is justifiable to question the use of experimental data that were collected almost two decades before the observational data. A lot of biophysical factors (including varieties and soil quality) may have changed. Therefore, the following set of results uses a range of alternative priors that span the plausible range drawing on evidence gleaned from the prior published literature.

In particular, the Bayesian sensitivity results indicate changes in Bayesian estimates of the maize yield responses given changes in the prior distributions of crop responses at 55 kg of nitrogen per ha. The sensitivity checks are in the changes to the prior on N fertilizer use on the mono-cropped maize response function. We considered a range of experimental crop response estimates reported for Malawi as summarized by Arndt et al. (2016) and Snapp et al. (2014) to assess if incorporating these priors leads to revisions in the crop responses that would warrant a change in the recommendations.¹⁶ We considered six mean prior levels of the crop response coefficient; specifically values of 0, 6, 12, 18, 24, and 30.

A directed search for variance parameters across prior literature revealed that the estimates vary as well. For example, using different econometric specifications of a quadratic response function as we do, the standard error for the maize response to nitrogen ranges from about 0.3 to 0.5 in Harou et al. (2017). Using a quadratic production function, Darko (2016: 92) estimated standard errors of the crop responses ranging from 1.6 to 3.2. In this study, we therefore consider three precision levels: 0.1, 1, and 10 corresponding to variances of 10, 1, and 0.1, respectively. Table 6 shows crop response quantiles for 18 different models for the various plausible mean and variance priors for the parameter corresponding to N fertilizer use. It is important to note that these are based on calculating marginal effects (which we also call crop response) not the direct coefficient of the N fertilizer. Marginal effects are calculated as |${{\beta }_1} + {{\beta }_2}\bar{N}$| where |${{\beta }_1}$| and |${{\beta }_2}$| are coefficients for N and N squared terms, and |$\bar{N}$| is the average nitrogen fertilizer rate at which the effect is evaluated at (i.e. 55 kg N/ha).

The computed crop responses (Table 6: row 2–row 13) are largely invariant to changes in the prior mean when the prior precision falls in the 0.1 and 1 range, but are indeed sensitive to the choice of prior means when a higher precision (10) is assumed. This is revealed by the extent of the overlapping 95 per cent credible intervals when the different prior means are compared across the same low prior precision level (e.g. compare 5.92 upper quantile for 0 mean prior and 0.1 prior precision with 4.97 lower quantile for 30 mean prior and 0.1 prior precision). In terms of the effect of differences in the precision of the prior estimates, the table shows that when crop response posteriors at different prior precision are compared across the same prior mean, the pattern of crop response posterior results is fuzzy. The reason for this is that the prior precision of one parameter also affects the off-diagonal terms in the variance-covariance matrix, such that depending on the associations across all parameters in the regression, the Bayesian crop responses may not be an intuitive weighted average. In Bayesian terminology, this is called the effect of nuisance parameters. This also partly explains why the posterior estimates in the baseline case (with 0 prior mean and 0.001 for all parameters) are different from the sensitivity analyses (0 prior mean and 0.1 prior precision for all other variables except N and N squared terms).

4.3.1 Mean crop response scenarios

The scenarios in Table 6 are illustrated graphically in Fig. 3 below using cumulative distribution functions (cdfs) of the crop response parameters subject to different priors. The comparisons of the cdfs can be interpreted as the posterior stochastic dominance. This is done by first holding the precision constant while varying the mean of the prior for the nitrogen coefficient (Fig. 3a and b). In Fig. 3a, as expected, holding precision constant at the same low level, the model with the higher prior mean (30 maize kg/ha per kg of N) stochastically dominates all the other models. However, the differences between the posterior distributions of mean responses are small.

At a higher precision level, the ordering of the mean response distributions is maintained, but now the differences between the posterior distributions are quite pronounced (Fig. 3b). This implies that achieving lower variance (high precision) may be necessary in the development of recommendations as also suggested by agronomic research (Vanlauwe et al. 2016 and Coe et al. 2016). These results show that comparing mean experimental and observational estimates without considering the variation in responses around the mean can result in final combined recommendations that are a reflection of the scenario in Fig. 3a or that in Fig. 3b. Either way, the posterior responses are less than the experimental mean crop responses that are used to develop fertilizer application recommendations in Malawi.

4.3.2 Precision of crop response scenarios

In Fig. 4a and b, we show the cases where the precision priors are varied, while the prior mean is fixed at some value. As expected from stochastic dominance, the higher the precision assumed for the prior, the more likely it is that the posterior will be similar to the prior distribution. Since the prior mean is 0 in Fig. 4a, the posterior parameter estimates for the model with a highly informative prior (i.e. a high precision prior = 10) is stochastically dominated by the ones with a weakly informative or effectively non informative priors (1 or 0.1). A contrasting case is presented in Fig. 4b where a prior mean of 30 is assumed. Here, the model with a high precise prior mean stochastically dominates the models with low(er) precision.

4.3.3 Combined mean and precision scenario

In the preceding stochastic dominance results, a clear and consistent ordering of either the prior mean or prior precision was assumed. However, what of the case that has a higher prior mean but a lower precision relative to the opposite case (i.e. lower prior mean higher precision)? This is where Bayesian stochastic dominance becomes useful. Let's consider a case where the prior means and variances are different. In Fig. 5, a low mean-high variance prior leads to posterior parameter estimates that stochastically dominate the high mean-low variance results. This implies that the debate between Jayne et al. (2015) and Dorward & Chirwa (2015) on whether a lower or higher mean crop yield response is appropriate for assessing the economic veracity of Malawi's farm input subsidy program is problematic when the precision (or variance) of the mean estimates are ignored.

When uncertainty is incorporated it is the case that for assumed precision levels of 0.1 to 1—which are typical of the precision levels in observational research—, the posterior crop response estimates that are likely relevant for commercial agriculture (observational) range from 4 to 9 kg of maize output for a unit of fertilizer per ha when the prior mean responses levels for experimental trials range from 0 to 30. While for precision levels of 1–10 which are prevalent in experimental research, the posterior crop estimates range from 9 to 19 for prior mean levels from 0 to 30. Based on our evidence, the likely fertilizer crop responses for Malawian agriculture are low and highly variable. Thus, any claims of substantial crop responses to fertilizer application in Malawian maize production are questionable. Therefore, when evaluating the efficacy of policies that depend on empirical estimates of crop responses, it would be advisable to err on the conservative side (and draw on all the plausible evidence about the mean responses and variations around this mean).

4.4 Heterogeneity in crop responses: Bayesian hierarchical model results

Beyond the question of variations around the mean crop responses, the crop response gaps may also be due to differences in locations where each of the studies were conducted within the country. We therefore need to understand the heterogeneity in the crop responses across locations. There are two extremes in the way heterogeneity is typically handled in econometric analysis. Most studies pool all the data and generate a single response parameter, assuming a homogenous response for the whole sample. At the other end of the spectrum, one may consider estimating the response parameters with specific (additive and multiplicative) fixed effects for individual cohorts of the data (e.g. individual districts), but this is generally inefficient due to data limitations. A Bayesian hierarchical modeling framework is an efficient (i.e. in terms of degrees of freedom) middle ground, which allows estimation of individual specific parameters as random parameters.

Figure 6 shows the density plots of parameter uncertainty (panel A) and heterogeneity (panel B). Panel A is based on crop response parameter for the draws from the Bayesian linear model, and illustrates the parameter uncertainty under the maintained assumption of a spatially invariant response function. Panel B is based on a random parameter specification of the district-specific crop response parameter in a hierarchical Bayesian model, and represents the district-level heterogeneity in crop responses.

The results indicate that the model-based parameter uncertainty (ranging from about 8 to 14 additional kgs of maize for additional unit of fertilizer) plotted in Panel A is smaller than the district-to-district heterogeneity plotted in Panel B (−40–30 additional kgs of maize for additional unit of fertilizer). The negative responses imply that soils are not conditionally responsive to fertilizer application in these locations. While it is uncommon for agricultural economists and agronomists concerned with average responses to report negative responses, this can occur when the fertilizer applied scorches the seed, especially in relatively dry conditions (Vanlauwe et al. 2011). The findings on district-to-district heterogeneity may seem unrealistic when compared with previous analyses that assumed spatial homogeneity in responses. This result is nonetheless consistent with agronomic research that addresses individual plot heterogeneity. For example, a study by Vanlauwe et al. (2016) compared empirical distributions (heterogeneity) to model based distributions (measuring uncertainty) of crop responses from agronomic trial data related to maize in Western Kenya and beans in Eastern Rwanda. They concluded that model based distributions provide better precision in the extremes than empirical curves, but that model based distributions depend on the assumption that the model is unbiased. In terms of developing targeted crop response support, the heterogeneous model may be more appropriate as it can help identify districts and plots that are non-responsive.

Based on the observational data used in our analysis, the districts of Machinga, Nsanje, and Chikwawa, for example, appear to be non-responsiveness to fertilizer application (see also Fig. 7). This is in line with experimental evidence (Government of Malawi 1997) that reports lower crop responses in the shire valley districts (Nsanje and Chikwawa). This suggests a future research strategy that proceeds by answering two questions: (1) will maize in a given field respond to fertilizer; (2) if so, what is the optimum fertilizer rate? Answering question 2 is more difficult than answering question 1. At a minimum, being able to answer question 1 is really impactful. The hierarchical Bayesian model allows one to answer both questions in that we can identify unresponsive districts and the magnitude of the response for the responsive districts.

Figure 7 shows the scatterplot of the district level linear crop response parameters in a hierarchical model for experimental and survey data. For almost seven districts (specifically, Chikwawa, Mulanje, Machinga, Rumphi, Mangochi, Phalombe, and Mulanje), the soils are not responsive to fertilizer application based on the observational evidence but are responsive in the experimental evidence.

As a research matter, this implies that there still more we need to learn about the biophysical and socio-economic aspects that distinguish these districts (i.e. water holding capacity or timing of fertilizer application). In terms of policy, it implies that well targeted extension services are required so that farmers do not waste fertilizer on unresponsive soils.

5. Limitations, future research, and conclusion

5.1. Limitations and future research

There are still several remaining limitations in addressing modeling challenges of parameter uncertainty, heterogeneity and disparate information sources in the estimation of crop responses to fertilizer application. The first limitation is that given the weaknesses of both experimental and observational studies, it is difficult to measure the quality of the subsequent posterior evidence. Recent advances in Bayesian optimization and machine learning, especially mult-fidelity methods (see Frazier 2018) hold promise in combining the estimates and evaluating the quality of such combined estimates. These advanced methods are beyond the scope of this paper.

The second limitation is that the scenarios on the effect of changes to the prior on the assumed posterior parameter estimates is based on the pooled model not the hierarchical model, which entails that heterogeneity is being treated separately from partial identification of the distribution of the parameters. This is inevitably the case because estimates treating fertilizer use parameters as random parameters across groups are not available. Other shrinkage models like empirical Bayes modeling and machine learning using ridge regression and multi-fidelity methods are potential candidates for future research. In addition, endogeneity concerns across both the experimental (due to self-selected master farmers) and observational (management bias and substantial measurement errors) evidence are areas of valid concern that future research could systematically address using quasi-experimental methods.

Finally, the models are not directly linked to any policy parameter like whether to subsidize fertilizer, which not only depend on the uncertainty and heterogeneity of crop response parameters, but also on other parameters (e.g. relative profitability of other crops) and the associated political economy considerations. While recent literature has demonstrated interactive effects of crop management, including weed management (e.g. Burke et al., 2020, Burke et al., 2022), and inherent soil quality and degradation (e.g. Berazneva et al. 2023), none of these studies attempt to provide whether experimental or observational evidence should be the basis of such conclusions across heterogeneous environments. Future research should consider the effect of incorporating multiple sources of information, uncertainty and heterogeneity on a policy decision and the analytical tools used in the paper are appropriate.

5.2. Conclusion

This paper has incorporated three aspects that are often ignored in the crop response literature, namely parameter uncertainty, multiple sources of information, and (spatial) heterogeneity in the response to fertilizer use. A Bayesian approach is employed to address each of these themes and close the measured gaps in the responses. This is an important goal for agricultural research because of the fairly constant trends of crop output/fertilizer price ratios across sub-Saharan Africa, which are indicative of the proposition that long-term trends in fertilizer profitability require improvements in farmer crop response rates. The analysis has shown that using prior knowledge of crop response estimates adds insights to the assessment of crop responses using observational data. In particular, we find that ignoring the precision parameter when using crop response estimates may lead to inconclusive policy prescriptions.

The debates on whether crop responses to fertilizer application are high or low are therefore questionable when uncertainty that appears to measurably affect the stochastic dominance ordering of crop response estimates is ignored. Unless uncertainty is considered, the arguments for or against the use of experimental and observation crop response estimates are inconclusive, thereby leading to questionable policy prescriptions. Moreover, while the debates have centered on means of crop responses, this paper has shown that both the means and variances matter in these policy discussions. The results of incorporating heterogeneity in the estimation by way of using a hierarchical Bayesian modeling approach are quite revealing. We find that the degree of spatial heterogeneity in fertilizer responses varies markedly, with some districts being effectively non-responsive to the application of fertilizer (e.g. Chikwawa), while other districts are highly responsive (e.g. Dedza). These spatial differences may be more important to explore for fertilizer policy targeting as they are as wide as national level mean differences between experimental and observational estimates. To make progress on reconciling the disparate estimates, we advocate for including uncertainty measures (e.g. standard errors) and specifics on climatic, soil and environmental conditions under which estimates being compared were generated. Methodologically, building on the Bayesian approach we have used, advances in Bayesian optimization and machine learning including multi-fidelity methods hold promise in guiding future experiments and surveys for generating spatially specific and robust recommendations.

Acknowledgments

This paper was prepared with support from the McKnight Foundation and Excellence in Agronomy (EiA) Initiative of the CGIAR funded by Bill and Melinda Gates Foundation (BMGF). We also acknowledge additional support from the University of Minnesota GEMS Informatics Center, Minnesota Agricultural Experiment Station project MIN-14–171 and BMGF funded Cereal Systems Initiative for South Asia (CSISA). The author(s) thank Jeffrey Apland and Jeffrey Coulter for extremely helpful comments on this paper. The content of this paper solely reflects the opinions and findings of the author(s).

Data availability statement

The main data and code are provided on GitHub: https://github.com/MaxwellMkondiwa/closing_gaps_bayes. Any other additional data beyond what has been shared will be shared on reasonable request to the corresponding author.

Conflict of interest

The authors declare no conflict of interest.

Footnotes

References