Digital Training for Micro-Entrepreneurs: Experimental Evidence from Guatemala

Abstract

Previous literature shows minor impacts of in-person business training in developing countries, but few papers study the effectiveness of digital training. A research partnership with a multinational company operating in the food sector of Guatemala enables the randomized evaluation of a digital training program involving the franchise store owners of one of its retail chains. The training program combined a mobile app offering access to reproducible video capsules and virtual one-on-one consulting meetings. The results of the randomized evaluation reveal significant impacts on knowledge, business practices, sales, and profits. An examination of the mechanisms underlying these results reveals that consulting meetings are crucial in inducing engagement with the app’s content. Program flexibility, internet access, and initial sales are also crucial determinants of training effectiveness.

1. Introduction

Management matters for firms and entire economies. Firms with better management have higher productivity, profits, output growth, exports, R&D expenditures, and patents (Bloom et al. 2019). Further, differences in management practices account for about 30 percent of the total factor productivity gap between the United States and other countries (Bloom, Sadun, and Van Reenen 2016). Therefore, designing cost-effective policies to address management gaps is crucial for international development policy. While the effectiveness of formal in-person business training—the most prominent direct policy to target managerial practices (Scur et al. 2021)—has been extensively documented in developing countries (McKenzie and Woodruff 2014; McKenzie 2021), the effectiveness of digital training programs to improve management remains largely understudied.

Digital training may succeed where in-person delivery fails. For starters, digital delivery offers significant reductions in implementation costs for individual firms. According to Chang (2016), e-learning, which includes digital training, saves up to 60 percent of instruction expenses, including travel, facility rental, supplies, administrative costs, and salaries. Second, digital delivery offers greater flexibility than in-person training programs to trainees. This can help overcome the geographical limitations that hamper the delivery of educational programs in hard-to-reach locations in developing countries and facilitate participation among women, who often face greater time constraints than men (Bandiera and Zipfel 2019). Flexibility may also increase training effectiveness directly by raising trainee effort and productivity, or it could indirectly make training more effective by reducing commuting times.¹ Third, digital training materials are easily stored in cloud space, which lowers their reproduction cost and increases material readiness for trainees. Zero reproduction costs make digital delivery ideal for re-training programs, including standardized programs for massive audiences and company-specific onboarding modules. Increased material readiness enables the formation of content libraries from which trainees can choose content to customize training to their specific needs.² Finally, digital training programs do not require physical proximity, a crucial feature made evident by the onset of the COVID-19 pandemic.

However, the impacts of digital training on knowledge, business-practice adoption, and profitability are far from obvious. First, persistence and completion rates of e-learning programs have been shown to be very low in the education literature.³ Such low persistence rates may be explained by behavioral barriers particularly present in online instruction.⁴ Second, while the decline in prices of ICT technologies over the last 20 years drastically improved internet access rates and smart device ownership worldwide, only 15 percent of the world’s population can afford access to broadband internet (World Bank 2016). Moreover, developing countries suffer from a large gender gap in smart device ownership (International Telecommunication Union 2016). Because the implementation of digital training programs necessitates access to broadband internet and smart device ownership, limitations at the country level and within countries may severely limit take-up. Digital training programs may also increase inequality by disproportionately favoring the digitally literate.⁵

This paper studies the causal impact of digital business training on knowledge, business-practice adoption, sales, and profitability of micro-enterprises in developing countries. The empirical strategy of the study consists of a randomized field experiment conducted in partnership with a multinational company operating in the chicken retail sector of Guatemala, involving 498 franchise store owners across the country, most of which are women. The experiment randomizes the delivery of a digital training program at the store-owner level. Training consists of a series of 28 short video capsules on 24 business management topics, which are progressively released on a weekly basis and last 67 minutes in total. These capsules are accompanied by knowledge quizzes and soft copies of training booklets containing exercises with an estimated total completion time of 6.5 hours. Trainees also receive three one-on-one calls of 30 minutes each with professional business consultants. The one-on-one calls are designed to address the needs of the franchise owners and facilitate take-up of the modules. The training program curriculum was custom-made for the franchise owners by a consulting company and covers a selection of topics like the franchisor’s business model and the best business management practices in the food retail sector. Treatment is administered through a smartphone application that records all the participants’ interactions with the training program materials, allowing us to measure treatment take-up accurately.⁶

The study leverages survey data, administrative records from the multinational company, and log file data from the mobile training app to measure impacts. Survey data include a baseline survey administered the month before treatment and a follow-up survey administered six months after treatment. Both surveys contain a business knowledge exam designed to test key technical lessons from the training program topics and a battery of questions designed to measure the self-reported business practices and profits of study participants. Administrative records from the multinational company include monthly sales measured without error at the store level in pounds of product and USD, as well as store market entry and exit dates. These records are used to measure impacts on sales and market exit 12 months after treatment. Finally, the study records the interactions of each trainee with the digital content of the mobile training app to analyze engagement trends during treatment.⁷

The empirical analysis of the paper proceeds in three steps. First, the paper reports the causal impacts of digital training on micro-entrepreneur knowledge, business practices, store sales, and self-reported profits from the field experiment. Digital business training increases the mean value of an overall knowledge and practices index by 7 percent relative to its mean for the control group at baseline.⁸ This effect is explained by a statistically significant improvement in overall business knowledge by 5.2 percent, as tested by the exam, a betterment of marketing practice by 11 percent, and an improvement of 6.4 percent in finance and inventory management practices. The paper then investigates impacts on sales using administrative records. It reports that the experimental treatment results in a marginally significant increase in sales above their pre-trend, amounting to 6 to 12.7 percent. Finally, the paper examines the impacts of digital training on self-reported profits, reporting a positive and statistically significant impact of 16.4 to 21.8 percent, which is explained by the increase in sales, and a negative, albeit non-significant, effect on operating expenses.

An exhaustive examination of impact heterogeneity for all the study outcomes reveals evidence of heterogeneity in take-up and business practices. Take-up is higher for women than for men, indicating no gendered penalty for women in the effects of training.⁹ However, digital delivery introduces a new type of penalty in take-up: broadband internet access. Specifically, take-up is positive only for trainees that own a smartphone with a data plan or have broadband internet access at home. For business practices, the treatment effects of the intervention are stronger for trainees without previous experience and high entrepreneurial ability, consistent with previous literature.¹⁰

Second, the paper examines the mechanisms giving rise to these findings by leveraging engagement data from the log file of the mobile app. These data are used to investigate the impact of holding a virtual meeting with a professional business consultant on the probability of watching video capsules on a given date. To uncover causal effects, the paper utilizes an instrumental variables (IV) strategy that relies on the calendar availability of the instructors as a source of exogenous variation in the timing of the consulting meetings. Holding a business consulting meeting raises the probability of watching video capsules on the same date by 13.8 percentage points, indicating that business consulting meetings are crucial in inducing digital engagement.

Finally, the paper models the cost-effectiveness (CE) of the digital training program under several alternative assumptions. The most conservative of these assumptions implies a cost-effectiveness ratio of 2.3, without including time and transport savings for trainees and consultants. This finding constitutes evidence that digital training programs have the potential to be cost-effective.

This study contributes to the literature on formal business training in developing countries, which is too vast to describe in detail in this article, but has been previously summarized in McKenzie and Woodruff (2014) and McKenzie (2021). While several randomized evaluations have measured the impact of different modalities of in-person training before, fewer studies have been conducted to measure the impact of digital business training programs. The only other randomized evaluation of a digital training program in a developing country is the study by Jin and Sun (2021), who find that a training program for new sellers provided by an e-commerce platform in China has a positive effect on sales. This paper’s experiment is the first to measure the impact of digital training on physical businesses rather than e-commerce trade.

This study also contributes to the literature on formal business training in developing countries by offering highly accurate measures of sales, business practices, and knowledge. First, instead of relying on survey responses, this study uses administrative records on sales at the store level from the company distributing chicken and all other foods to the stores participating in the experiment, allowing it to measure sales without measurement error. A ubiquitous problem with previous studies in the business training literature is that impacts are measured using self-reported information from experimental surveys that are prone to measurement error, with the well-known implication that the resulting profitability measures depend on the type of questionnaire used to elicit information (De Mel, McKenzie, and Woodruff 2009). Second, the monthly administrative records on sales span a five-year period prior to the randomized experiment, greatly improving the statistical power of the study regressions.¹¹ In a well-known article, McKenzie (2012) highlights the potential value of multiple measurements of experimental outcomes at relatively short intervals to reduce noise and improve power. Third, the intervention comprises franchise stores of relatively similar sizes operating within the same economic sector under a uniform business model. Thus, the study’s local context parses out some of the heterogeneity in unobservable determinants of business practices and profitability of micro-enterprises, which plagues the training literature, given that most studies consider interventions that target micro-enterprises of vastly different sizes, economic sectors, and business models.

Finally, this paper joins the literature documenting the effectiveness of methods other than standard in-person training in improving the profitability of small and medium-sized enterprises (SMEs), including movies (Barsoum et al. 2022), mentorship (Brooks, Donovan, and Johnson 2018), handbooks of local practices (Dalton et al. 2021), meetings among firm owners (Lafortune, Riutort, and Tessada 2018), and consulting, outsourcing, and insourcing (Anderson and McKenzie 2022). This study adds to this literature by documenting the effectiveness of a bundled approach consisting of the digital delivery of a combination of training and consulting.

2. Contextual Information

The field experiment results from a collaboration agreement signed in 2018 between IDB Invest, which is the private-sector arm of the Inter-American Development Bank Group, and Corporación Multi Inversiones (CMI), a multinational corporation based in Guatemala operating in the food, real estate, finance, infrastructure, and telecommunications industries of 15 Latin-American countries. The partnership between IDB Invest and CMI aims to create economic opportunity for entrepreneurs and their families while also increasing access to fresh, safe food in the surrounding communities. Improving access to affordable and nutritious foods is of particular importance for Guatemala, given that it is ranked 106 out of 120 in the list of countries with the lowest rates of chronic malnutrition and that 47 percent of all Guatemalan children under the age of five suffer from stunting (Sanchez, Scott, and Lopez 2016).

Casas de Pollo Rey (CDPR) or “Houses of the King of Chicken” is one of the retail chains of the food sector branch of CMI, specializing in cooked and uncooked chicken and pork products since 2014. The CDPR retail chain operates across the country under a franchise business model in which local entrepreneurs own and operate small-scale franchise stores. In November 2020, CDPR operated 752 franchise stores, out of which 317 were located in the central region, 305 in the west zone, and 130 in the east, as shown in supplementary online appendix fig. S1.1. Only 87 franchise stores were located in Guatemala City’s municipality, within the country’s central region. The model store is a small shop furnished with refrigeration equipment and promotional decoration, as shown in supplementary online appendix fig. S1.2. The list of products sold by CDPR includes 70 cooked and uncooked chicken and pork products, out of which 19 must be offered in all stores and 51 are optional. Supplementary online appendix table S1.1 presents the complete list of products.

To become franchise owners, interested entrepreneurs fill out an application on CDPR’s website or apply by telephone. The applicant must declare 12–17 thousand Quetzales (1,558.4–2,207.8 USD) in unencumbered funds and propose a store location.¹² Human resources personnel conduct an initial interview with the applicant to further explain CDPR’s business model. If the applicant remains interested, the franchisor’s operation team evaluates the proposed store location in terms of pedestrian traffic, distance to the closest CDPR neighbor, and safety. Corporate managers prefer safe locations in busy commercial and residential areas and try to ensure a minimum distance of 1.5 km to the nearest neighboring store to prevent business stealing. If a location proposal is approved, the applicant signs a franchising contract with CDPR. The terms of the agreement include a CDPR products’ exclusivity clause, a hygiene clause ensuring franchisees keep food products properly refrigerated, a clause by which the franchisee commits to registering as a federal taxpayer and obtaining all necessary sanitary permits for operation, and a clause by which the franchisee commits to opening the store at least six days per week. If the agreement is signed, CDPR leases all store equipment to the franchisee, including freezers and fridges, provides exterior painting services, and gives the franchisee instructions on how to reorder food inventories, while the franchisee is responsible for purchasing food inventories and kitchen utensils. Finally, the franchising relationship continues for an unlimited term, unless the franchisee voluntarily terminates it, or the franchisor revokes the franchise for violations of contractual clauses.

Women from low-income households make up the majority of the franchise owners and decision-makers. A survey involving 196 randomly selected franchise owners, conducted by IDB Invest in December 2018 (supplementary online appendix table S3.1 contains a list of summary statistics from this survey), revealed that women comprise 75 percent of the group of store owners. They are, on average, 37 years old and have two children. Furthermore, owners are typically members of low-income households, as their average self-reported monthly household income was 6,519 Quetzales or 847 USD. This income level is below the cost of a basic goods basket, which was 8,219 Quetzales in December 2018, according to the national institute of statistics or Instituto Nacional de Estadística.

In contrast to supermarkets and other large vendors, CDPR franchise stores almost exclusively serve clients from nearby neighborhoods. A survey administered to 108 female CDPR clients selected at random from the stores across the country in November 2018 reveals that they travel short distances and pay frequent visits to their nearest store. Specifically, each client visits her closest CDPR store 12 times per month on average and buys 3.4 pounds of chicken per visit. Furthermore, 56.5 percent of the clients travel by foot, averaging 12 minutes per journey each way, while 36.1 percent use some form of public transport, averaging 34 minutes each way. Only 7.4 percent travel by car or taxi. Further summary statistics from this survey are contained in supplementary online appendix table S3.2.

CDPR franchises are small businesses. A survey administered to all 752 franchise stores operating in November 2020 (see supplementary online appendix table S3.3 for this survey’s summary statistics) shows that 36.3 percent of the franchises did not have any paid employees, 51.7 percent only had one full-time¹³ employee, and the remaining 12 percent had 2–4 full-time employees, with autonomy and responsibilities typically limited to daily operations, such as customer service, food handling, processing payment, and cleaning. Additionally, family members of the owner worked without pay for at least 20 hours per week in 29.4 percent of the franchises. In the same month, store-level administrative records show that sales were 2,248 USD on average, with substantial variation in sales across franchise stores. Figure 1 presents the sales distribution, which is skewed to the right, with a median of 1,587 USD, an interquartile range of 2,067 USD, a minimum of 174 USD, and a maximum of 10,281 USD. Additionally, this figure shows that the entire distribution of sales has shifted leftward each year since 2017.

CDPR store owners and operators need business training for three reasons. First, store owners typically do not have any substantial business experience and do not receive any substantial training when opening their first store. According to the same survey administered to 196 randomly selected owners in December 2018, 62 percent of the store owners had not previously owned a business, and 55.6 percent had not received any business training before opening their store. Second, store turnover and exit levels are exceptionally high. While the total number of franchise stores increased from 552 in January 2017 to 752 in November 2020, over the same period, there were 28 store openings and 21 exits on average each month. Third, there was a declining trend in monthly mean sales per store before the implementation of the training program. From January 2017 to 2020, aggregate sales of all CDPR franchise stores remained stagnant at around 2.1 million USD, despite a 70 percent increase in the number of active stores.

3. The Digital Training Program

The digital training program is designed to upgrade the business skills of the franchise store owners of CDPR that are working with CMI. Intending to tailor the program to the needs of the franchise store owners, IDB Invest hired Fundes, a consulting company headquartered in Costa Rica, to conduct an initial diagnosis of the CDPR stores’ business practices. The initial diagnosis consisted of 9 interviews with CMI corporate executives, 2 online focus groups with 5 randomly selected franchise owners each, and 50 telephonic surveys applied to a randomly selected group of franchise owners. Based on the diagnosis results, the consulting company designed the training program.

The final training program lasts 7 weeks and consists of 28 short video capsules, a workbook with additional training exercises, and 3 one-on-one video meetings of 30 minutes with a professional business consultant. The training topics covered by the training materials include branding, operations, patrimonial security, marketing, equipment maintenance, hygiene and food safety, financial management, gender empowerment, inventory management, digital payment options, and customer satisfaction.

Video capsules are released gradually every week, and participants are free to watch them at their preferred time from the moment they are released. They are between 1 and 7 minutes long, and their total duration is 67 minutes. As shown in supplementary online appendix fig. S2.1, they focus on a combination of formal business administration concepts and simple heuristic guidelines and rules of thumb.¹⁴  Supplementary online appendix table S2.2 contains a detailed description of the objective and content of each video capsule.

The workbook contains practical exercises, which take 6.5 hours to complete, according to a breakdown elaborated by the consulting company. The workbook also includes templates of the necessary materials to adopt the practices covered in the video capsules (e.g., a template for a monthly cash flow).

In the one-on-one video meetings, a professional business consultant provides personalized advice and strategies to improve the store owner’s business practices. In their first meeting, the business consultant conducts an initial diagnosis, provides individualized advice, and sets concrete next steps for the implementation of the business practices covered in the video capsules. These next steps aim exclusively to commit trainees to advance in the implementation of specific business practices, rather than committing trainees to the formulation of personal goals about business outcomes like sales or profitability. In the second and third meetings, the consultant provides technical assistance and feedback for the implementation of their advice from the first meeting. Training participants are free to schedule the video meetings at their preferred time and date, although this is subject to the availability of the business consultants.

As mentioned in the introduction, training take-up is an issue of particular concern for the digital delivery of education in developing countries due to (a) relatively low smartphone ownership, broadband internet penetration, and digital literacy rates, as well as expensive internet data prices, and (b) behavioral biases that affect online instruction by lowering course persistence. To address smartphone ownership and digital literacy, the multinational lent a tablet to store owners that did not own a smartphone, and the consulting company provided step-by-step instructions to download the app and create log-in credentials during an online welcome workshop and through phone calls to individual trainees. To ensure course persistence, the three strategies described in detail next were adopted.^15,16

First, the consulting company sent standardized weekly WhatsApp reminders to all training participants when new video capsules were released, personalized reminders of their scheduled one-on-one mentoring sessions, and personalized weekly text messages comparing their individual training completion rates against the rates of other store owners in the training group.^17,18 Notably, all training content was strictly excluded from these messages to prevent them from directly impacting specialized knowledge, as measured by the score in the knowledge exam, which was explicitly developed to assess the respondent’s command over the training program’s most technical concepts rather than its general content. In contrast, suggestive evidence presented in supplementary online appendix S7 indicates that reminders successfully increased take-up, as video capsule completion times are strongly correlated with the timing of these reminders.

Second, the mobile application required users to answer a knowledge question after finishing each video capsule and before moving on to the next training module. Panel A of supplementary online appendix fig. S2.3 shows that mobile app users could only watch unlocked video capsules, signaled with a check mark. App log file data reveal that trainees unlocked training content in their first attempt only 65.7 percent of the time. Furthermore, watch time increased on average by 35.7 percent after each additional attempt, thus indicating that trainees exerted effort to learn the correct answer and did not simply guess after a failed attempt. Nonetheless, there is evidence of content skipping, as trainees completed the video capsules faster than their actual duration on 10.9 percent of the occasions in which their first attempt was successful.

Third, the CMI corporation rewarded training participants with digital money for each training module completed, redeemable for chicken and pork inventories. Panel B of fig. S2.3 presents the screen message participants saw, detailing the products they could purchase using their digital money after completing a training module. Supplementary online appendix table S2.1 lists the monetary value of the in-kind rewards that training participants could redeem, which ascended to a maximum of 70.2 USD for participants that completed all training modules. The section on treatment effect heterogeneity tests whether these in-kind rewards induced heterogeneous effects on sales. It finds that they had no additional impact on sales.

In addition to the above measures aimed at maximizing training take-up, the study also adopted two measures to guard against treatment group contamination. First, the mobile app required user authentication to access training content. As shown in supplementary online appendix fig. S2.2, the app required all users to register and verify their cell phone number, agree to terms and conditions, and input their personal information and their store code. This code is an identification number used by the CMI corporation to keep track of each store’s sales, as well as by the store owners to retrieve their financial information for fiscal purposes. Thus, franchise store owners had a strong incentive not to share their code with anyone. Second, data from the log file of the mobile application was used to track the number of times each participant watched a video, the number of modules completed, and the time spent watching each video. The IP addresses of all mobile application users were also tracked to detect and shut down any suspicious activity.

Finally, the mobile app did not allow for social interactions between trainees through a forum or WhatsApp group, limiting the scope for information sharing through digital means. Moreover, training did not take place in a physical venue nor include any recurrent event that would allow store owners to meet each other. Furthermore, the geographical distance between nearest neighbors was 3.6 km on average in the experimental sample, making in-person information sharing unlikely. However, franchisees may have shared information about the training program with their personal networks, which may include other CDPR owners. This type of information sharing could have led control group members to improve their business practices in response to the information conveyed to them by their peers, potentially biasing the treatment effect estimates of the study downward.

4. Experimental Design

To uncover the causal impact of digital training on business outcomes, this study uses a field experiment involving 498 out of the 582 CDPR store owners operating in Guatemala in September 2021.¹⁹ The store owners in the experimental sample managed 539 stores in the same month. The remaining franchise owners were unwilling to participate in the study, primarily reporting general distrust of telephone surveys, particularly given the high extortion rates prevailing in Guatemala.²⁰ Recent work by Brown et al. (2021) and Estefan et al. (2022) indicates that extortion is higher for high-value businesses and that extortion rates vary by location. A formal test for observable differences in average sales and store location between the stores managed by the owners in the experimental sample and the stores managed by non-participating CDPR owners is presented in supplementary online appendix table S4.1. While the existence of unobservable correlates of the decision to participate cannot be ruled out, this test finds no evidence of any statistically significant difference across the two groups in any of these variables.

The field experiment used in this study randomized assignment into a single treatment consisting of the digital training program described in the previous section. As mentioned in the introduction, randomization was stratified by gender, year of opening of the owner’s first franchise store, initial sales, and region to ensure a balanced sample. This stratification strategy yielded the 35 strata shown in supplementary online appendix table S4.2. Armed with this stratification strategy, random assignment was then conducted relying on a computer program to ensure replicability by setting the program’s seed to the randomly chosen number 327195. This number was specified before obtaining the final list of training participants, as attested by the trial plan available online in the AEA RCT registry. Finally, randomization was clustered at the store-owner level. This level of analysis was chosen because the sample includes 35 multi-store owners who managed a total of 76 franchise stores in September 2021, so treatment impacts in any of these stores may spill over to all stores managed by the same owner.

Randomization resulted in 251 out of 498 store owners assigned to the treatment group and the remaining 247 owners assigned to the control group. In terms of stores, randomization resulted in 273 out of 539 stores assigned to the treatment group and the remaining 266 stores assigned to the control group.

The timeline for the implementation of the intervention is as follows. A surveying company administered the baseline survey by telephone between the first week of September and the first week of October 2021. After levying this baseline information, owners were randomized into the treatment or the control groups in the second week of October 2021. The training program was administered by the consulting company hired by IDB Invest between mid-October and mid-December 2021, according to the program schedule detailed in supplementary online appendix table S4.3.²¹ The surveying company levied a follow-up survey by telephone between the last week of May and the last week of June 2022.

The main experimental hypothesis of this study is that digital business training leads to an improvement in business outcomes. While the ultimate outcome of interest is business profitability, the study tests for impacts on each of the intermediate outcomes in a well-defined theory of change. The rationale for expecting a digital business training program to impact business outcomes is similar to that of many previous studies that examine the impact of in-person business training. In particular, the theory of change is (a) treatment assignment increases the amount of time spent watching online capsules and interacting with business consultants, (b) the materials of the training program improve knowledge of the CDPR franchising model and the best business practices in the food retail sector, (c) franchise managers adopt the business practices that they learn, and (d) business revenue increases, but expenses remain unchanged or drop, so profits increase.

For each step in the theory of change, indicators are constructed to measure the training program’s impacts. First, data from the log file of the mobile application are used to construct two indicators of treatment take-up: time spent watching video capsules, and number of training modules and mentoring sessions completed. These indicators enable the construction of measures of full and partial treatment compliance. Additionally, video capsule completion dummies with daily frequency are constructed for all training participants to measure the influence of the timing of the consulting meetings and the reminders on course persistence. Second, to measure actual knowledge and self-reported business practices, a knowledge and business-practice index is constructed by averaging the answers of each store owner in the experimental sample to a short knowledge test and a battery of business-practice questions administered as part of the baseline and follow-up telephonic surveys. Third, data on each store of the experimental sample are extracted from the CDPR’s administrative records to construct monthly sales. Finally, self-reported estimates of mean monthly revenue, operating expenses, and profits, also from the baseline and follow-up surveys, are used to examine the training program’s impact on two alternative measures of profits: self-reported profits and self-reported revenue minus expenses.

While the relevance of the theory of change follows from previous studies, including robust empirical evidence indicating that business practices explain a large share of variation in business outcomes in developing countries (McKenzie and Woodruff 2017), the experimental design does not allow the study to verify that digital training affects profitability exclusively through the channels in the causal pathway of the theory of change. To address this concern, survey data are used to test and rule out alternative mechanisms through which digital training may impact business outcomes, including increased entrepreneurial effort, as reflected in the store opening hours, the number of employees in each store, and business stealing.²²

5. Data

The study draws on several data sources to measure treatment take-up and training impacts on business knowledge, business practices, sales, and profits. The paragraphs below describe the data sources used to construct the key variables of the study.

Mobile application log file. Given the potentially low rates of online education persistence previously highlighted in the literature (Banerjee and Duflo 2014), a key outcome of the study is treatment completion. To measure this outcome, the study leverages individual-level data from the mobile application log file on the 251 training participants’ behavior, including their dates of access to the application and time spent watching each video capsule. Armed with these data, it first constructs 28 variables with a daily frequency capturing the dates of capsule completion for each participant, defined as the event that the participant finishes watching the video capsule and afterward responds correctly to the questions of a short knowledge quiz. The study uses these data to investigate the effect of the business consulting meetings and the unexpected release dates of personalized WhatsApp and phone call reminders on daily completion rates.

Baseline and follow-up survey. According to the theory of change, digital training increases business profitability by improving business knowledge, business practices, and store sales, while potentially lowering production costs. To construct separate indicators of these outcomes, the study uses data from the baseline and follow-up surveys, levied by phone given the COVID-19 contingency that started in March 2020.²³ First, the study uses the study participants’ responses to a short business knowledge exam consisting of 11 true/false questions included in both survey waves to construct an objective measure of business knowledge, defined as the number of correct answers divided by the number of questions in the knowledge exam. Next, to measure self-reported business practices, it uses the participants’ responses to a survey module consisting of 26 yes/no questions on self-reported practices in the business areas of branding, operations, patrimonial security, marketing, equipment maintenance, hygiene and food safety, financial management, gender empowerment, inventory management, digital payments, and customer satisfaction. The answers to these questions are used to construct four practices indexes for (a) marketing, (b) financial planning and inventories, (c) operations and training, and (d) time management, defined as the share of practices followed by the participant in each of these categories. The items included in each index are detailed in supplementary online appendix table S6.1. Furthermore, an overall knowledge and business-practice index, defined as the simple average of the test score and the number of best practices followed by the participant, is constructed. Finally, to measure self-reported business outcomes, the study uses the study participants’ responses to three questions that ask them to report each of their stores’ mean monthly profits, operating expenses, and total revenue. Supplementary online appendix fig. S5.1 shows the transcript of the baseline and follow-up survey in English.

Sales administrative records. To measure the primary outcome of the intervention, the study relies on CMI’s administrative records of CDPR sales. These records contain monthly store-level information on the total value of sales in USD and their weight in pounds, as well as the value of sales for each of three broad product categories: raw chicken, raw pork, and prepared foods. These administrative records run from January 2017 and include the universe of CDPR stores that ever operated in Guatemala. In addition to providing an accurate sales measure for the experiment, these records document the reasons for each store’s exit from the market, which include definite store closures, shifts to a different line of business in the same store location, changes in the name of the legal owner to avoid personal income taxation, and store code changes due to store relocation or for fiscal purposes, such as avoiding corporate and payroll taxation.

6. Empirical Strategy

The empirical strategy of the paper is presented in this section. First, the regression model used to measure the causal impacts of digital training on treatment take-up, business knowledge, business practices, and self-reported business outcomes is discussed, along with the strategy followed to prevent false positives. Such a strategy is crucial given that the paper’s empirical analysis investigates impacts on several related practices and outcomes. Second, the regression framework used to estimate the impact of training on store sales is explained. Finally, the strategy followed to estimate the heterogeneous impacts of the intervention is presented.

6.1. Take-Up, Business Knowledge, and Business Practices and Outcomes

Ordinary least squares (OLS) is used to estimate the intention-to-treat (ITT) effect of the digital training program on the post-treatment business outcome Y_i of the franchise store owner i, given by β in the following linear regression model:

where T_i is a treatment dummy that takes the value of 1 if the franchise owner is a member of the treatment group and 0 otherwise, and X_i represents a vector of strata dummies. Standard errors are robust to heteroskedasticity of unknown form.

The study tests multiple hypotheses pertaining to the impact of a digital training program on several related business concepts, practices, and outcomes, which could lead to false discoveries if the analysis ignores that the probability of rejecting at least one true null hypothesis increases with the number of tests. Thus, for each of two families of hypotheses, pertaining respectively to (a) knowledge and business practices and (b) business outcomes, the procedure described in Anderson (2008) is used by the paper to compute sharpened q-values, which control for the false discovery rate (FDR), or the proportion of rejections that are “false discoveries” or type I errors, in addition to reporting ordinary p-values.²⁴

6.2. Store Sales

A different strategy is used to measure the impact of digital business training on sales, given the availability of five years of monthly CDPR administrative records. In particular, the study measures the impact of digital training on store-level sales relative to their pre-trend instead of measuring impacts at the owner level. The following panel data model is estimated:

where Y_it are the sales of store i in month t, T_it is a treatment dummy that takes the value of 1 if the franchise owner of store i is a member of the treatment group and t is a post-treatment month, and 0 otherwise. The parameter γ_i is a store fixed effect, which controls for the time-invariant characteristics at the store level that correlate with sales, including the store location. The regression specification controls for the time-specific determinants of sales that affect all stores equally, such as the COVID-19 lockdowns, by including the δ_t time dummies. Furthermore, it controls for time trends that depend on initial store sales and municipality-specific trends, included in the |$\mathbf {X}_{i,t_0}$| vector. Including these controls is necessary to correct for differential sales trends arising by chance prior to treatment, as discussed in detail in supplementary online appendix S8.3. Standard errors are clustered at the store-owner level and are robust to heteroskedasticity of unknown form.

Additionally, to gain power in estimation,²⁵ the following analysis of covariance (ANCOVA) specification is estimated using 12 rounds of monthly sales data post treatment:²⁶

where Y_it are the sales of store i in month t, T_i is a treatment dummy that takes the value of 1 if the owner of store i is a member of the treatment group and 0 otherwise, X_i is a vector of strata dummies and dummies for missing data for 12 rounds of pre-invention sales, δ_t are time dummies, and Y_{i, 0} is a vector of 12 rounds of monthly pre-intervention sales and sales trends. Standard errors are clustered at the store-owner level and are robust to heteroskedasticity of unknown form.

6.3. Heterogeneous Effects

To test for heterogeneous effects within the OLS regression framework described above, the treatment indicator is interacted with one dichotomous variable of interest at a time, as follows:

where β denotes the effect of treatment, η denotes the direct effect of Z_i, and θ denotes the differential effect of treatment on the group with Z_i = 1. This regression model is estimated via OLS, and standard errors are clustered at the store-owner level and are robust to heteroskedasticity of unknown form.

Finally, to estimate heterogeneous impacts on sales within the differences-in-differences framework, Z_i is interacted with T_it, as follows:

where θ denotes the differential effect of treatment on the group with Z_i = 1. Standard errors are clustered at the store-owner level and are robust to heteroskedasticity of unknown form.

7. Impacts on Knowledge, Business Practices, Store Sales, and Profits

This section presents the study’s estimates of the causal impacts of digital business training. First, a balance table of the outcome and control variables in the baseline survey and store sales in the pre-treatment months is provided to demonstrate the quality of the randomization procedure. Second, results from the attrition analysis are presented. Third, the impact estimates of digital training on take-up, knowledge, business practices, store sales, and profits are discussed. Finally, the results from the analysis of heterogeneous effects are shown.

7.1. Checking Pre-treatment Balance of Outcome and Control Variables

Table 1 presents evidence of pre-treatment balance for crucial outcomes and control variables. Panel A compares the outcome means between the 251 store owners in the treatment group and the 247 store owners in the control group for business knowledge, practice indexes, and demographics, including entrepreneurial performance, as captured by a three-tier system designed by the franchisor (i.e., owner classes A, B, and C, in descending order of performance). Panel B compares the outcome means between the 273 stores in the treatment group and the 247 stores in the control group, including store sales and self-reported revenue, costs, and profits. For each variable, columns (1), (2), and (3) present the number of observations and the mean and standard error for the control group, whereas columns (4), (5), and (6) show the respective number of observations, mean, and standard error for the treatment group. Column (7) presents the p-value for a two-sided t-test of difference in means between the treatment and the control groups for each variable. These tests fail to reject the null hypothesis of no difference in means for all variables in the table at conventional significance levels.

7.2. Handling Missing Values

Between the baseline and the follow-up survey, 48 store owners attrited from the experiment. These store owners represent 9.6 percent of the list of 498 store owners in the baseline survey. Since the administrative records track all store exits, they can be used to infer, through cross-examination of the survey responses and the administrative records, the reason why stores drop from the sample. This cross-examination exercise reveals that 31 store owners established different businesses in the same locations, and 17 dropped out because their stores permanently closed.

In terms of stores, attrition rates were higher, which is in line with the high rates of turnover documented in the contextual information section. From September 2021 to June 2022, a period that corresponds to the span of time between the baseline and the follow-up surveys, 92 stores attrited from the experimental sample. Examination of CDPR’s administrative records reveals that 17 stores permanently closed, 33 established a different business in the same location, 14 changed the name of the legal owner to avoid personal income taxation, and 28 changed their store code due to store relocation or for fiscal purposes, such as avoiding corporate or payroll taxation.

Given attrition, the empirical analysis that follows rigorously tests whether treatment impacted the odds of dropping from the experimental sample. Table 2 presents the differential attrition analysis results. It begins by testing the effect of treatment assignment on store-owner attrition at follow-up in panel A. Column (1) shows the coefficient from an OLS regression of an attrition indicator on the treatment assignment dummy, including strata dummies as controls. The estimated effect is non-significant at conventional levels and is also close to zero in magnitude. The table then decomposes the attrition indicator into two mutually exclusive categories: store closure and establishment of a different business in the same location. Column (2) shows no statistically significant effect of treatment assignment on owner attrition due to store closure, while column (3) also shows no statistically significant effect on the probability the owner attrites the sample by establishing a different business in the same location. Panel B reports similar findings for store attrition by June 2022, with additional results reported in columns (4) and (5), of no statistically significant effects of treatment on the probability of changing the name of the legal owner or changing the store code, respectively. Thus, in what follows, regression estimates are not adjusted for differential attrition, as there is no evidence of this issue in the data.²⁷

7.3. Take-Up, Knowledge, and Business Practices

The empirical analysis then turns to investigate the effects of treatment assignment on take-up, knowledge, and business practices in table 3. This table presents ITT estimates obtained by regressing each outcome variable on a dummy for treatment assignment in the follow-up survey. Column (1) shows that treatment assignment increases the probability of treatment take-up, as measured by a dummy for graduation, by 50.4 percentage points (t-statistic = 14.8). The criteria used by the consulting company for graduation consist of completing all three personalized business consulting meetings and watching at least 70 percent of the video capsules in the mobile app. Alternative take-up definitions requiring different video capsule completion rates yield similar impact estimates of treatment assignment on the probability of take-up, as shown in supplementary online appendix table S8.8.

The table then shows the effects of treatment on knowledge and business practices in columns (2) through (7). Column (2) reveals a statistically significant increase of 0.029 points (t-statistic = 2.4) in the business knowledge test, which is equivalent to a 5.2 percent increase relative to the mean exam score for the control group in the baseline survey. In column (3), the table presents a strongly significant effect of 0.069 points (t-statistic = 3.3) on the marketing index, which is equivalent to an 11 percent increase relative to the control mean at baseline. Next, in column (4), the table shows a statistically significant effect of 0.047 points (t-statistic = 2.1) on the finance and inventories index, equivalent to a 6.4 percent increase relative to the control mean at baseline. There is no significant impact on the business operations index, which includes employee training and onboarding practices, or the time management index, as shown in columns (5) and (6), respectively. Finally, column (7) presents a strongly significant effect of 0.05 points (t-statistic = 3.3) on the overall knowledge and business-practice index, equivalent to a 7 percent improvement relative to the control mean at baseline. All the statistically significant effects resist Anderson’s sharpened q-value correction for multiple hypothesis testing.

7.4. Store Sales

The paper reports impacts on store sales relative to their pre-trend, which are measured without error using CMI’s administrative records, in table 4. Panel A reports the ITT on store sales in USD from the differences-in-differences specification. It presents results for three estimating samples: the full experimental sample, the sub-sample of stores that remain open throughout the study period, and the sub-sample that excludes stores run by multi-store owners. The second sample comprises only time variation in mean monthly store sales that is not influenced by entry or exit decisions, while the third sample comprises the stores directly operated by trainees. Column (1) reports a marginally significant increase in sales amounting to 158.7 USD above their pre-trend (t-statistic = 1.7) for the estimating sample that includes all stores in the experimental sample, equivalent to a 6 percent increase in sales relative to the mean for the control group in October 2021. Column (2) also reports a marginally significant impact of 220.7 USD (t-statistic = 1.9), or 7.2 percent relative to the control mean in October 2021, for the estimating sample that only includes stores that remained in operation throughout the study period. Finally, column (3) reports a statistically significant impact at conventional levels of 231 USD (t-statistic = 2.2), or 9.6 percent relative to the control mean in October 2021, for the estimating sample that excludes the stores of multi-store owners. These findings indicate that effects on sales are larger conditional on remaining in operation and are stronger for stores operated directly by trainees. The latter of these findings is consistent with the absence of a significant effect on employee training practices reported in the previous section.

Panel B of the table reports the ITT on store sales in USD from the ANCOVA specification. For the sample that includes all stores in the sample that had not exited by January 2022, which is the first month of the post-treatment period, column (1) reports a marginally significant impact of 207.2 USD (t-statistic = 1.8), equivalent to an 8 percent increase relative to the control mean in October 2021. Column (2) reports a statistically significant effect of 280.7 USD (t-statistic = 2.1) for the sample of stores that remained in operation throughout the study period, or 9.4 percent relative to the control mean in October 2021. Finally, for the sample that excludes stores owned by multi-store owners, column (3) shows an impact of 304.2 USD (t-statistic = 2.3), or 12.7 percent relative to the control mean in October 2021, which is also statistically significant at conventional levels. These findings are consistent with the results from the differences-in-differences specification in panel A.

Thus, there is evidence for a 6 to 12.7 percent increase in sales resulting from digital training, depending on the empirical strategy and estimating sample of operating stores. Supplementary online appendix S8.4 shows that the positive impacts of digital training on sales are not significant when the estimating sample is expanded to include stores after they exit by coding their sales as zero in the months following their exit. This result is consistent with the previous finding that treatment has no significant impact on the probability of store attrition, reported in table 2.

7.5. Profits

Self-reported information on store-level business outcomes from the follow-up survey is used to analyze impacts on profits. As in other developing countries, CDPR franchisees often do not keep financial records, making profit impact estimation reliant on recall. As mentioned in the data section, the survey asks franchisees to directly report each of their stores’ mean monthly revenue, operating expenses, and profits. The question regarding operating expenses explicitly asks franchisees to include store rent, employee wages, loan payments, taxes, utility fees, and other services, such as cleaning, in their estimation. These data enable the comparison of two alternative profit measures: reported profits and reported revenue minus expenses.

While the paper reports impacts on both profit measures, the former measure is preferred for three empirical reasons, which align closely with the reasons cited in De Mel, McKenzie, and Woodruff (2009) to utilize self-reported profits rather than reported revenue minus expenses as a reasonable measure of actual profits. First, the question regarding operating expenses in the follow-up survey does not explicitly ask franchisees to include the cost of food inventories. This cost is difficult to calculate from self-reported revenue because the markups implied in the final consumer pricing guidelines, regularly issued by the franchisor,²⁸ differ across food product categories (i.e., raw chicken, raw pork, and prepared foods), thereby making food inventory cost dependent on the mix of products that each store sells. Second, there are substantive differences across the two measures. In particular, the correlation coefficient between reported profits and reported revenue minus expenses is low, between 0.38 and 0.47. Furthermore, 7.8 percent of the stores report negative revenue minus expenses in the baseline survey, while none report negative profits, as shown in supplementary online appendix table S8.6. Third, while there is a close correlation between reported revenue and the best available estimate of actual revenue, constructed by multiplying each store’s actual sales by 1 plus the markup implied by the average sales mix across all stores,²⁹ there is also evidence of substantial underreporting of revenue. A close examination of the distribution of reported and actual revenues in supplementary online appendix table S8.7 reveals that store owners underreport sales by 19.2 percent on average in the baseline survey.³⁰

Table 5 presents the regression results. As shown in column (1), treatment increases reported profits by 102 USD (t-statistic = 2.5), or 23.3 percent relative to the mean for the control group in the baseline survey.³¹ This result resists Anderson’s sharpened q-value correction. Columns (2) through (4) report treatment effects that run in the expected directions but are not statistically significant for reported revenue, expenses, and revenue minus expenses, respectively. Importantly, the point estimate for the effect of the training program on operating expenses is negative, implying that the increase in sales did not heighten operating expenses.

The results in this and the previous section, which point to a significant increase in sales relative to their pre-trend coupled with a non-significant drop in expenses, are consistent with what is found when dissecting the training program’s impacts on marketing and finance in supplementary online appendix S8.7. Table S8.9 shows that treatment increased the probability of implementing low-cost marketing tactics, such as running discount sales, calling clients by name, building client contact lists, making flyers, and taking client orders on WhatsApp. Table S8.10 shows that treatment improved cost overseeing, by increasing the probability of keeping a monthly cash flow and inventory control.

Moreover, additional analysis conducted using survey data confirms that the most likely mechanism giving rise to this increase in self-reported profits is the one embedded in the theory of change. Alternative explanations are soundly ruled out, including an increase in weekly opening days and hours in table S8.11 and an increase in the number of store employees or the decision to sell homemade complementary products like salsas in table S8.12.

7.6. Treatment Effect Heterogeneity

Table 6 presents the results of the heterogeneity analysis, which focuses on the most relevant heterogeneity dimensions for each outcome. Beginning with training take-up, the table inspects heterogeneity by gender, educational attainment, age, and access to broadband internet. Column (1) shows that the take-up rate for women is 12.3 percentage points higher than the corresponding rate for men (t-statistic = 1.66). Column (2) reports a difference of 14.8 percentage points between the take-up rate of high-school-educated individuals and the corresponding rate for individuals with lower educational attainment (t-statistic = 2). Column (3) reports a negative albeit non-significant difference of 14.8 percentage points in take-up rates between trainees aged 50 and older and younger trainees (t-statistic = 1.5). Column (4) reveals a strongly significant difference of 53.3 percentage points in training take-up between individuals with broadband internet access and individuals without internet access (t-statistic = 6.1).³² Finally, column (5) confirms the findings reported in the previous columns by simultaneously including the interactions of treatment with gender, high-school education, broadband internet access, and age within a single regression.

Next, columns (6) through (8) report heterogeneity in the effects of digital training on knowledge and the adoption of business practices. The table tests heterogeneous impacts by entrepreneurial ability first, using the three-tier system designed by CDPR to classify store owners.³³ Column (6) reports a negative difference of 0.066 index points in the effect of training on the index of knowledge and practice adoption between the low-performing tier and other groups (t-statistic = 2.1).³⁴ Next, the table tests impact heterogeneity by previous business experience, as captured by a dummy indicating that the individual recently opened the CDPR franchise store, had never received previous formal business training, and had never operated another business. Column (7) shows a positive albeit non-significant difference of 0.059 index points in the effect of treatment on the index of knowledge and practice adoption between inexperienced and seasoned entrepreneurs (t-statistic = 1.59). Column (8) confirms the findings from the previous two columns.

Finally, the table turns to test heterogeneity in sales and profits. Columns (9) and (10) test, respectively, whether the ability to realize higher sales and profits depends on receiving in-kind rewards. Both columns report no evidence of a heterogeneous gain for training participants who redeemed the in-kind rewards over the gains observed by other trainees.

8. One-on-One Interactions’ Impact on Digital Engagement

This section investigates the effect of virtual business consulting meetings on trainee engagement with reproducible digital content.³⁵ This analysis leverages data on the timing of digital engagement of training participants from the log file of the mobile app and data from the log books of consultants on the timing of the meetings. These data are used to investigate the impact of holding a virtual meeting with a professional business consultant on the probability of watching video capsules on a given date. To uncover causal effects, the paper proposes an instrumental variables (IV) strategy that utilizes time variation in the calendar availability of consultants as a source of exogenous variation in the timing of the consulting meetings to uncover their causal impact on digital content engagement.³⁶ This section presents the results from implementing the IV strategy, which indicate that one-on-one interactions with business consultants have a strongly significant impact on engagement with the digital app content.³⁷

8.1. Empirical Strategy

To quantify the effect of business consulting meetings on an indicator for the decision of individual i to watch the video capsule c in the smartphone application at date t, denoted by W_ict, the parameter β from the following linear regression model is estimated via OLS:

where M_it is an indicator for participant i holding a business consulting meeting with a professional business consultant at t; X_it is an indicator for the event that the consulting company sent a text reminder to encourage completion at t; δ_{i, dow(t)} are individual-specific weekday fixed effects, which control for day-of-week effects that may affect engagement differently across individuals, such as weekend effects; λ_{i, woy(t)} are week fixed effects, which control for week-of-year effects that may affect individuals differently, including holiday breaks; and θ_ct are capsule × date dummies, which control for capsule release dates and other time effects that may affect engagement differently across capsules over time. Standard errors are robust to heteroskedasticity of unknown form and are clustered at the store-owner level.

A key concern when estimating equation (1) via OLS is omitted variable bias. If trainees face binding idiosyncratic shocks to busyness or time availability, they will be less likely to complete video capsules on the same dates as they hold business consulting meetings. The reason is that the opportunity cost of time will increase with the amount of time spent on the training program in a single day. Omitting these idiosyncratic shocks from the regression will bias the OLS estimate of β downward.³⁸ To address this concern, the paper uses an IV strategy which relies on time variation in the availability for business meetings from the business consultants side. In particular, an instrument Z_{b(i), t} indicating whether the business consultant b assigned to trainee i has an open schedule for meetings at t is constructed. The identifying assumption of this empirical strategy is that consultant availability affects the probability of watching a video capsule c only through its impact on the probability of scheduling a business meeting at t.

8.2. Results

Table 7 presents the results from estimating equation (1). Panel A shows the results from estimating the effect of holding a business consulting meeting on the probability of watching a video capsule via OLS. The estimate in column (1) reveals a significant effect of 4.5 percentage points (t-statistic = 22.5), which represents a more than four-fold increase relative to the probability of watching a video capsule on any given date, were such a probability uniformly distributed across the 7 days of the 14 calendar weeks of the study period. Including a dummy for receiving a text message, individual × weekday and individual × week fixed effects, and date × capsule dummies in columns (2) through (5), respectively, leaves the order of magnitude of the coefficient and significance levels largely unchanged.

Panel B presents the 2SLS estimates from the IV strategy described above. The estimate in column (1) shows a significant effect of 13.8 percentage points (t-statistic = 34.5) of holding a business consulting meeting on the probability of watching a video capsule on the same date, revealing that the OLS estimate is biased downward. This finding constitutes evidence that participants’ busyness is evenly distributed across the calendar. Columns (2) through (5) show that the coefficient size and significance remain unaltered with the inclusion of a dummy for the reception of text reminders, individual × weekday and individual × week fixed effects, and capsule × date dummies, respectively.

Panels C and D present the first-stage and reduced-form equations. Column (1) of panel C reveals a strongly significant first-stage estimate of 0.0796 ($F$-statistic = 8, 075.9), ruling out any potential concerns regarding the weak instruments problem. This estimate means that availability of the business consultant for a meeting on a given date increases the probability of holding a business meeting by 7.96 percentage points, an almost eight-fold increase relative to the mean probability that would be observed under a uniform distribution. This estimate remains large and significant after progressively including controls in columns (2) through (5). Likewise, the reduced-form estimates, presented in panel D, show that the availability of the assigned business consultant increases the probability of watching a video capsule by 1.1 percentage points on any given date, which is equivalent to a one-fold increase in the probability of watching a video capsule, were this probability uniformly distributed over time. These estimates remain roughly unchanged in size and significance after the inclusion of covariates in columns (2) through (5).³⁹

9. Cost-Effectiveness Analysis

A key potential advantage of delivering training programs digitally over doing so within a traditional classroom context is the promise of steep cost reductions. This section quantifies the CE ratio of the digital training intervention. The point estimate for the CE ratio of the digital training program is 2.3 dollars in profit for each dollar spent on training, even under the most conservative assumption.

Table 8 shows the results from the cost-effectiveness analysis of digital training. This analysis compares the benefit of the program in terms of store profits with its financial cost. Panel A replicates the annualized effect on profits, obtained by multiplying by 12 the effect on monthly profits depicted in column (1) of table 5.

Panel B presents the intervention costs, obtained directly from the business agreement between IDB Invest and the consulting company, and the opportunity cost of trainee time, defined as the profit value of the time spent completing the training program. To estimate per capita intervention costs, the cost stipulated by the consulting firm for each business item is divided by 250, the contractual number of trainees initially agreed upon with the consulting company. These costs include the costs associated with the business consulting meetings, the costs of developing and maintaining the mobile app, and the costs of other measures implemented to increase take-up (i.e., tablet loans, text reminders, and digital money).

On the other hand, the opportunity cost of time is estimated by multiplying the time requirement of all training components by the average hourly profits of the stores in the experimental sample in the baseline survey. The time requirements of the training program include 67 minutes of video capsules, 90 minutes of business consulting meetings, and 6.5 hours of workbook exercises. Average monthly profits in the baseline survey are 459 USD, and the weekly number of store opening hours is 70, which implies 280 opening hours per month. Dividing 459 USD by 280 hours, and multiplying the resulting number by 9.1 hours, yields 14.9 USD as the opportunity cost of time of the training program per trainee.

The cost-effectiveness of the intervention is reported under alternative assumptions regarding the extent to which the consulting meetings and the mobile app video capsules are indispensable to induce positive treatment effects on knowledge, business practices, sales, and profits. Panel C presents the CE ratio and its corresponding 95 percent confidence intervals under these alternative costing assumptions. Assumption 1 incorporates all training costs and is the most conservative. It implicitly assumes that both training components (i.e., the virtual meetings with business consultants and the app video capsules) are necessary to induce positive treatment effects on profits. Assumption 2 excludes the costs associated with the mobile app, which include the development of a standardized training program, video capsules, and system maintenance. This assumption is the second-most conservative, as it implicitly assumes that the observed training effects arise exclusively from the tele-meetings of training participants with business consultants. Finally, Assumption 3 excludes the costs associated with business consulting meetings, which include a consulting work plan, a training promotional campaign, the individual-level diagnostics of business consulting needs, the piloting of business consulting meetings, and the consulting meetings. This assumption is the least conservative, as it implicitly assumes that the treatment effects arise exclusively through the online training materials offered by the mobile app. This assumption is the most optimistic and should be used as an upper bound to the CE ratio of digital business training.

10. Conclusion

This paper studied the impacts of digital business training coupled with business consultant services on the knowledge, business practices, store sales, and profits of franchisees from a large retail chain in the food sector of Guatemala. It found that digital training improves knowledge and business practices, increases store sales relative to their pre-trend, and boosts self-reported profits. When zooming into the effects on business practices, it found that digital training is particularly successful in inducing the adoption of marketing, finance, and inventory practices through a betterment of the franchisees’ knowledge level. In contrast, it failed to find impacts on operations and time management. Then the paper investigated heterogeneous effects, finding that general program flexibility, access to broadband internet, and trainee initial business experience and entrepreneurial ability are key determinants of treatment effectiveness. It then turned to examine the role of one-on-one consulting meetings in explaining the observed effects and found that they play a central role in incentivizing engagement with digital materials and preventing dropout. Finally, the paper showed that the CE ratio of the digital training program is equal to 2.3 USD in profit per dollar of implementation cost.

The results of this paper have fundamental policy implications. First, they offer insights about the extent to which digital training programs constitute effective instruments to transfer human capital and business skills for the betterment of management practice in developing countries. These insights are particularly important given that skill transfer will likely gain relevance in the future as technological change and shifts in international openness to trade lead to labor displacement.⁴⁰ Second, while the policy promise of online training programs for adults is an active area of research in the United States (Bonvillian 2020; Osterman 2020), the results in this paper shed light on the effectiveness of online training programs for adults in contexts where technology access is lower than in advanced economies.

While this paper relies on experimental evidence, there are several limitations to the reported findings. First, store homogeneity aided in finding positive impacts, by cancelling variation in business outcomes related to the economic sector and the type of activities performed across establishments, which is typically present in the business training literature. However, the attractiveness of this local context does not come without costs, as the validity of the paper’s findings may not extend to other settings. In particular, high program take-up rates and effort put into the program may result from the accountability to which franchisees are held by the franchisor. Second, the heterogeneity analysis suggests that the cost-effectiveness of digital training crucially depends on broadband internet access, internet affordability, and digital literacy, which may be lower in other geographical regions, particularly in Sub-Saharan Africa.

Despite these limitations, the findings of this study are encouraging, as they suggest that digital delivery can be particularly useful in transferring practical business knowledge and bridging the total factor productivity (TFP) gap between developing countries and advanced economies. Furthermore, they highlight the fundamental role of one-on-one interactions in driving trainee persistence rates, engagement with digital content, and the adoption of better business practices. Future research should center on figuring out the optimal frequency and type of one-on-one interactions and the design of reproducible training materials that maximizes digital engagement and practice adoption.

Data Availability Statement

The data underlying this article were provided by IDB Invest and CMI under licence / by permission. Data will be shared on request to the corresponding author with permission of IDB Invest and CMI.

Author Biography

Alejandro Estefan (corresponding author) is a professor at the University of Notre Dame, Notre Dame, United States; his email address is [email protected]. Martina Improta is a PhD student of at the University of Genoa, Genova, Italy; her email address is [email protected]. Romina Ordoñez is a senior rural specialist at the Inter American Development Bank; her email address is [email protected]. Paul Winters is a professor at the University of Notre Dame, Notre Dame, United States; his email address is [email protected]. The research for this article was financed by IDB Invest and the Global Policy Initiative of the University of Notre Dame. The authors thank Jeff Bloem, Taryn Dinkelman, Nilesh Fernando, Lakshmi Iyer, Joseph Kaboski, William Maloney, Patrizio Piraino, Daniel Prudencio, Emma Riley, Diego Vera-Cossio, Bruce Wydick, and seminar participants at the 2023 AEA Annual Meetings, CEIDS, IFPRI, LACEA-LAMES, MWIEDC, the University of Illinois Urbana-Champaign (UIUC), and the University of Notre Dame for insightful comments and Geovanny Alvarado, Fiorella Blanco, Daniel Castañeda, Renato Conde, Savita Diggs, Uriel Galace, Randall Hidalgo, Meghan Howat, Eleanor Jones, Jack O’Leary, José Riley, Diana Spencer, Eugenia Suárez, and Monica Turner for expert research assistance. A supplementary online appendix is available with this article at The World Bank Economic Review website.

Footnotes

References