Hard and Soft Skills in Vocational Training: Experimental Evidence from Colombia

Abstract

This paper studies the effects of an oversubscribed job-training program on skills and labor-market outcomes using both survey and administrative data. Overall, vocational training improves labor-market outcomes, particularly by increasing formal employment. A second round of randomization evaluates how applicants to otherwise similar job-training programs are affected by the extent that hard versus soft skills are emphasized in the curriculum. Admission to a vocational program that emphasizes technical relative to social skills generates greater short-term benefits, but these relative benefits quickly disappear, putting participants in the technical training on equal footing with their peers from the soft-skill training in under a year. Results from an additional randomization suggest that offering financial support for transportation and food increases the effectiveness of the program. The program fails to improve the soft skills or broader labor-market outcomes of women.

1. Introduction

Vocational-training programs are often seen as a means to improve the transition between formal schooling and employment. However, the extent to which vocational-training programs succeed in improving the labor-market outcomes of their participants is highly debated—and empirical evaluations of vocational programs report mixed results. While some programs suggest positive and sustained impacts (Attanasio, Kugler and Meghir 2011; Reis 2015; Diaz and Rosas 2016; Attanasio et al. 2017; Brunner, Dougherty, and Ross 2019; Chakravarty et al. 2019; Alfonsi et al. 2020; Kugler et al. 2022; Silliman and Virtanen 2022), others report few or no effects (Galasso, Ravallion, and Salvia 2004; Card et al. 2011; Hicks et al. 2013; Maitra and Mani 2017).¹ Moreover, even the positive effects of some vocational programs that succeed in improving labor-market outcomes in the short term can dissipate in only a few years (Acevedo et al. 2020; Alzúa et al. 2016; Hirshleifer et al. 2016). A common explanation for this fade-out is that the later outcomes of vocational trainees may suffer if the narrow training they receive does not provide them with the skills to adapt to changes in the nature of work and hold on to their jobs (Krueger and Kumar 2004; Hanushek et al. 2017).

This paper uses a randomized experiment to study whether vocational-training programs can be designed to provide their participants with sustained benefits by exogenously varying the technical and social skills provided in the program. Building off recent work highlighting the importance of social skills in adapting to changes in the labor market (Deming 2017) and allowing people to better hold on to their jobs, the randomization of applicants to oversubscribed vocational-training programs in Cali, Colombia to various treatment arms proceeds as follows. First, applicants are randomly assigned to either receive vocational training or not. Then, within particular vocational programs, applicants are randomly assigned to programs with varying degrees of training in technical and social skills. The effects of providing social and technical training in vocational programs are assessed by tracking applicants through both an extensive array of survey data and data on labor-market outcomes from social security administrative records.

By randomly assigning curricular content within otherwise comparable vocational-training programs, our study explicitly addresses how curricular content in vocational training affects labor-market performance. While a number of studies focus on policy changes to offer some insight into the potential effects of changes in vocational curricula (e.g., Malamud and Pop-Eleches 2010; Hall 2016; Bertrand, Mogstad, and Mountjoy 2019), effects identified by these non-experimental studies might be partly driven by changes in the composition of students entering vocational training.² Further, comparisons between cohorts are challenging, because, as Field et al. (2019) observe, vocational graduates are particularly sensitive to changes in initial local labor-market conditions due to business cycle fluctuations. There are few randomized trials that examine the impacts of soft skills. While Adhvaryu, Kala, and Nyshadham (2018) find positive effects of a soft-skills-training program on the wages of garment workers in India, Groh et al. (2016) find no effects of soft-skills training for female community college women graduates in Jordan. Acevedo et al. (2020) and Ibarraran et al. (2014) both examine the impact of combining soft skills with internships in the Dominican Republic and find positive short-term effects but only for women.³

In the aggregate, the results show that admission to vocational training through random assignment increases formal employment (8–14 p.p.) in both the social and technical treatment arms and for both men and women. These effects are robust to a number of specifications and they are present in both survey and administrative measures of employment. Applicants also experience substantial increases in monthly wages, as measured in the administrative data (USD 21).⁴ Cost effectiveness estimates based on these results suggest that the program pays for itself in about eight months.

The analysis of differences between the effects of the social and technical vocational-training programs on labor-market outcomes highlights two findings. First, the initial benefits of vocational training are smaller for those randomly assigned to social-skills training than for those assigned to the technical-training program. Second, the heightened initial benefits of the training emphasizing technical skills are entirely erased after a little more than half a year, putting them on equal footing with their peers in the social training after this point.

Additionally, we attempt to shed light on the extent to which the training program was capable of shifting social skills. In aggregate, both programs improve organizational (0.06 SD) and communication skills (0.08 SD); effects on other social skills are generally positive, but less precise. Supporting the idea that the increased emphasis on social skills did, in fact, generate additional effects on social skills, applicants assigned to the training emphasizing social skills experience larger gains in communication skills (0.12 SD). However, we are unable to detect differences in other dimensions of social skills between applicants in the two programs. While somewhat speculative, it is possible that these improved communication skills did not affect initial employment outcomes, but helped prevent erosion in the effects of the job training.

A further randomization of the receipt of a modest stipend intended to support program participation provides evidence that nearly all gains from the program are dependent on stipend receipt. One possible explanation for this pattern of results is that resource constraints prevented individuals from fully participating in the program in the absence of the stipend. Alternatively, however, these findings may be explained by the increased conditionality from requiring a minimum of a weekly attendance, or by increased funds during the job-search period.

Importantly, and in contrast with prior studies in Colombia and the Dominican Republic (Attanasio, Kugler and Meghir 2011; Card et al. 2011; Acevedo et al. 2020), the job-training program studied here failed to benefit women. While women were slightly more likely to enter formal employment after job training, they were no more likely to be employed overall, and earned no more than before training. These effects on labor-market outcomes could be explained by at least three stories, each with different policy implications. First, while the training program had substantial effects on men’s interpersonal and organizational skills, women experienced no benefits in either domain. Second, the differential set of effects by gender may reflect distinct constraints to employment experienced by men and women—such as those stemming from household responsibilities. Third, the lack of benefits of job training for women may stem from institutional factors such as gender discrimination in hiring. Of these, the job-training program itself is most capable of changing the skills that are taught or the ways in which they are taught, making the training more equitable for both men and women. This finding echoes recent work in India, suggesting that soft-skill interventions can have differential effects for men and women, and that household responsibilities can mute the labor-market effects of job-training programs (McKelway 2020).

The aggregate results from this study are in line with previous studies from Colombia, the Dominican Republic, and Mongolia, which suggest that vocational programs can improve labor-market outcomes in the short term (Attanasio, Kugler and Meghir 2011; Field et al. 2019; Acevedo et al. 2020). Moreover, Kugler et al. (2022) and Attanasio et al. (2017), who are able to follow their sample for up to 10 years through administrative data, find that the initial benefits of vocational training in Colombia may persist in terms of labor-market outcomes into the longer term.⁵

Furthermore, this study advances the ongoing debate in the literature on vocational training by experimentally testing the extent that technical and soft skills produce different labor-market dynamics in the field. A common view suggests that general (versus specific) skills are important in providing individuals with the flexibility to adapt to changes in the demands of the labor market (Goldin and Katz 2009; Acemoglu and Autor 2011; Goos, Manning, and Salomons 2014; Deming 2017; Deming and Noray 2020). Applying this view to the context of vocational education, some scholars have argued that any initial benefits of vocational education, which tends to emphasize technical education specific to a trade, are likely to disappear with time (Krueger and Kumar 2004; Heckman and Krueger 2005; Hampf and Woessmann 2017; Hanushek et al. 2017; Alfonsi et al. 2020).

The random assignment of applicants to intensive vocational-training programs emphasizing either technical or soft skills directly tests how labor-market dynamics are affected by the extent to which vocational curricula include social skills, which are likely to be a particularly important form of general skills (Deming 2017). Comparing vocational programs that differ only in the degree to which they include social skills, this study builds on the few studies that evaluate vocational programs with components related to social skills (Groh et al. 2016; Acevedo et al. 2020).⁶ In line with theory, while those exposed to the program emphasizing technical skills are quicker to find employment, these initial benefits of the training emphasizing technical skills are quickly erased.

These findings add nuance to the debate of general and specific skills in vocational training, providing evidence that technical skills may be helpful in boosting initial labor-market outcomes—potentially because they provide practical skills helpful for finding jobs, but also showing that the benefits of technical skills may be particularly fragile over time. While they experience attenuated short-term benefits, it is interesting that participants exposed to the training program emphasizing soft skills do not experience an erosion in the benefits of the job-training program.⁷ This pattern of results provides empirical support for the idea that vocational programs focusing exclusively on technical skills may be ineffective at providing long-term benefits. One reason this might be is that technical skills can become obsolete over time (Deming and Noray 2020), while more general skills—such as social skills—may better equip people to adapt to changes in the labor market (Hanushek et al. 2017). Alternatively, it is also possible that skills facilitating teamwork or communication with managers may simply help people hold to jobs longer (Adhvaryu, Kala, and Nyshadham 2018; Weidmann and Deming 2021) or that soft-skills-training programs promote the psychological skills (such as self-efficacy) needed for employment (McKelway 2020).

The rest of the paper proceeds as follows. After describing the program and experimental design, we document the survey and administrative data used in the analysis. We then present the results and cost-benefit analysis before concluding.

2. Program Description and Experimental Design

This study focuses on applicants to oversubscribed vocational-training courses which were part of the Inclusive Employment Program (IEP) offered by the Carvajal Foundation in Cali, Colombia⁸ between June and December 2018.⁹ In total, 18 classes—each lasting 160 hours—were offered in eight different areas of the service sector: sales and client services, general services, surveillance and security services, cashiers, quality control assistant, cooking assistant, delivery assistant, and storage assistant. The program offered two of each of these types of courses, except for four courses in general services. While one course ended in July and one in August of 2018, the vast majority ended in the fall of 2018 (four in September, two in October, eight in November, and two in December).¹⁰

Since all these programs were oversubscribed, applicants were randomly assigned to receive either vocational training versus no training at all (i.e., treatment versus control groups). In a second round of randomization, each applicant admitted to receive vocational training is assigned to a version of the course with either an emphasis on social (Treatment 1) or on technical (Treatment 2) skills. Third, half of those assigned to vocational training were also provided with a stipend for transportation and meals. See fig. 1 for a visual depiction of the design of the study.

Participants registered voluntarily into classes in response to a call for registrations by the Carvajal Foundation. The foundation established this program to help the poorest in the community access jobs. Thus, the foundation reaches broadly to enroll participants through radio, social media, loud-speakers in cars that go through poor neighborhoods, flyers, and through the public employment office¹¹ and offices that provide other public services to the poor. As reported in the next section, most individuals who registered for these courses were in the lowest socioeconomic strata according to the Census of the Poor in Colombia.

Individuals who were interested in registering for the classes attended an informational meeting and registered for the specific classes they wanted to take. Each class had between 23 and 31 spots and registration in each class ranged from 28 to 47 registrations per class. Given oversubscription in the classes, the foundation randomly selected individuals to either receive a spot or not receive a spot in the course. The lotteries for each course were recorded by video to ensure everyone knew people were allocated into the courses by chance. Those who did not win a spot in the training courses through the lottery were in the control group and were not provided with other services by the Carvajal Foundation for an entire year following registration. As shown in the next section, the randomization divided people into groups that were very similar on average in terms of their characteristics, thus giving credibility that the lottery worked well in terms of randomly assigning individuals into and out of the classes. There were initially 663 people who registered in the courses and of these, 451 were randomly assigned to the training and 212 were assigned to the control group.

The next step of the random assignment determined whether individuals admitted to vocational training would attend classes with a greater emphasis in teaching social skills or technical skills. Half of the courses in each area emphasized social skills (Treatment 1: 100 hours social skills, 60 hours technical skills), while the other half emphasized technical skills (Treatment 2: 60 hours of social skills, 100 hours of technical skills).¹² Since there were (at least) two classes in each area (for example, in security services), a coin toss determined whether an individual applicant would be admitted to the version of a course with either an emphasis on soft or technical skills. Social workers provided the social-skills content of the courses, which included self-esteem, work ethic, organizational skills, interpersonal skills, and communication skills. Prior to the experiment, all courses consisted of 40 hours of soft skills; an additional 20 and 60 hours of social-skills curricula were developed specifically for the experiment for courses with 60 and 100 hours of social-skills training, respectively. While the content was similar, the course with 100 hours of social-skills training offered deeper coverage of organizational, teamwork, leadership, communication, and interpersonal skills. The content of the technical skills varied depending on the course (e.g., security and surveillance services, cashiers, or cooking-assistant skills). In all cases, the content was specific to a job and the skills were taught both in the classroom and through practical hands-on experience in the training center of the Carvajal Foundation. Courses offering 60 hours instead of 100 hours of technical training taught the same content but reduced the number of hours spent in practical hands-on training. Upon completion of the course, there was a graduation ceremony, and students received an informal (not formally recognized) diploma from the Carvajal Foundation in the content area (e.g., security services). These diplomas did not specify whether students received the social or technical versions of the courses, and it is unlikely that applicants themselves knew that courses varied by degree of soft versus hard skills. In total, there were 222 individuals assigned to training with greater emphasis on social skills and 229 assigned to training with greater emphasis on technical skills.

The last step of the randomization involved either offering or not offering a stipend to participants. The monetary stipend consisted of US$1.50 per day. This monetary transfer was meant to help participants cover the costs of transportation and meals while taking the courses. The offer of a stipend was randomized at the class level to avoid envious comparisons among participants in the same course. The stipend was paid at the end of the week for the entire week to ensure that participants attended the classes each day of that week before receiving the payment. The toss of a coin determined which classes received (or did not receive) the stipends.

While the program did not include an explicit internship or job placement, it did have relationships with local employers. These relationships with local employers provided a common path by which training participants often accessed their first job.

3. Data Description

The data supporting this project come from survey data collected as part of this project, as well as administrative data from the Ministry of Health and Social protection. Each is described in the subsections below. The section on survey data also reports information on randomization balance.

Survey Data

Survey data provides insight into the extent that the randomization achieved balance and short-run effects of the program (Barrera-Osorio, Kugler, and Silliman 2019). We collected data on both treatment and control individuals and their families by conducting our own pre-treatment surveys. Further, we collected information on labor-market characteristics and social skills in the pre-treatment surveys. Importantly, these pre-treatment surveys were conducted in November 2017, before people found out whether they were randomly assigned into or out of the program. We then collected information between March and April of 2019 using a reference period of February 2019 for all labor-market questions and a reference period of March and April 2019 for social-skills questions for both treatment and control individuals after the courses had concluded. This time line is illustrated along with our descriptive data in fig. 2.

Table 1 shows covariate balance checks between applicants assigned to treatment and control groups. The first column shows descriptive statistics for the control group, while the second column shows the difference in characteristics between the treatment and control groups. In the control group, 66 percent of individuals are women. They are on average 26 years old and have 11 years of education: 99 percent have an elementary school education, 94 percent a secondary school education, and 29 percent have technical higher education. Most individuals, 55 percent, are Afro-Colombians and 17 percent report being Mestizo. Control group households are relatively poor with an average household income of US$19.23/day and an average household size of 4.45 individuals. Importantly, column (2) shows that treatment individuals are very similar to control group individuals in terms of all their characteristics. None of the treatment–control differences are individually or jointly significant (the F-test is 0.50). Further, since the administrative data cannot be linked to course fixed effects, we include a second test of balance between treatment and control groups without the inclusion of course fixed effects (the F-test is 0.59, see table S1.1, column (2) in the supplementary online appendix).

We also check that the characteristics between the control and treatment groups are similar for each treatment arm. Columns (3) and (4) report the treatment–control differences for Treatment 1 (with emphasis on social-skills training) and for Treatment 2 (with emphasis on technical-skills training), respectively. Column (3) shows that the only difference between Treatment 1 and the control group is the household size, which is slightly bigger for those in the treatment. However, the test of joint significance of all characteristics yields an F-test of 0.94 and a p-value of 0.53, showing that these are not jointly significantly different between the social-skills treatment and the control group. Similarly, column (4) shows that Treatment 2 and the control do not differ across any observed covariate. The joint significance test shows that the differences of all the characteristics are not jointly significant (F-test is 0.60 and p-value 0.89). We conduct further tests for balance across treatment arms in columns (3) and (4) of table S1.1 These tests suggest little evidence of imbalance between those randomly assigned to the technical- versus social-training program (column (3), F-test is 1.34) or for people randomized into and out of stipend receipt (column (4), F-test is 1.21; see also table S1.2). And, although a few event-study estimates in the pre-period are individually significant (hard versus soft skills) (fig. 3), joint tests of significance suggest that we cannot reject the null hypothesis (p = 0.76, p = 0.817, and p = 0.919) for each of the three outcomes respectively.

Table 2 reports pre-treatment differences between the treatment and control groups in terms of labor-market and social-skills outcomes in May 2018, before any of the courses started. As with other pre-treatment characteristics, we conduct a balancing test for indicators of employment, whether the worker has a contract, and whether the worker is a casual worker (panel A). We also include balancing tests of days worked per month and hours worked per week, as well as wages per hour and monthly earnings. The control group means in column (1) show that 55 percent were employed, but only 28 percent had a written contract. Control group individuals worked only 13.5 days per month and 24.5 hours a week.¹³ Not only did they not work full time, but also their wages were very low, only US$1.20/hour. Column (2) shows that the treatment group was very similar. In fact, none of these variables are significantly different between treatment and control group individuals. Likewise, the differences between the control and each of the treatment arms are also insignificant for nearly all labor-market outcomes.

Panel B of table 2 reports measures of soft skills at baseline. These index measures are aggregated from items collected by the Carvajal Foundation, using a measurement instrument they had used internally in prior work. This instrument contains several questions across multiple dimensions of soft skills—work ethic and responsibility, interpersonal skills and ability to get along with others, leadership skills, teamwork, and communication skills. The items contained in this instrument measure specific skills using a self-reported Likert scale.¹⁴ Item-level information functions (IIF) for each item are reported for each measure in fig. 3. These IIFs suggest that while the measures do a good job discriminating between individuals with low levels of social skills, they may provide limited information on individuals whose social skills are above the median. Given the ordinal responses in the survey, we build from item-response theory (IRT) and use graded response models (GRM) to develop indices of these measures that maximize the information each provides. The indices we use are anchored in the baseline measures to have a standard deviation of 1 and mean of 0.¹⁵ To test for the validity of our social-skill measures, we report correlations between post-period labor-market outcomes and social skills in table S1.3. Though the correlations are weak, they are in the expected direction, suggesting that these measures capture relevant skills for the labor market. Column (1) in table 2 reports the mean scores for the control group. Differences from the control mean are reported for the different treatment groups together (column (2)) and separately (columns (3) and (4)). There is little evidence that groups are out of balance at baseline, as suggested by the joint significance test.

Administrative Data

Longer-term analysis of the program is based on data from Social Security records through December 2019 (Ministry of Health and Social Protection 2020). These data allow us to examine the labor-market outcomes for individuals between 11 months and 17 months after finishing the program. Since the social security records only provide information about formal sector jobs that provide social security benefits, we are not able examine informal sector jobs as we do with our survey data. We impute zeros for all outcomes for which a person does not show up in the administrative data that month. Overall, there are 107 individuals who never show up in the administrative data. Of these, 50 individuals are from the control group, 28 are from the training program emphasizing social skills, and 29 are from the training program emphasizing technical skills. In addition, due to confidentiality, for the process of merging our experimental sample with social security records—a process undertaken by the Ministry of Health and Social Protection—all personal information was deleted, leaving only a marker identifying who was in different treatment arms and who was in the control group. Therefore, all the analysis with administrative data does not control for observable characteristics. Nevertheless, given the baseline balance in characteristics across treatment arms, it is highly likely that controls for baseline characteristics would not make a difference in the point estimators.

Figure 2 shows data on employment characteristics from monthly administrative records for the control and treatment groups. Panel (a) shows the trajectory of days of formal employment, panel (b) shows formal employment probabilities, and panel (c) shows monthly wages. The solid lines in the figures represent the control group, the dashed lines represent the social treatment and the discontinuous lines represent the technical treatment. Figure 2(a) shows that prior to treatment (most courses ended by November 2018 except for two that ended on 3 December), all groups had on average about five days of formal employment per month. Days of employment jumped for all groups starting in December 2018, but the jump was greater for the two treatment groups than for the control group and this persisted until December 2019. Similarly, fig. 2(b) shows that the probability of formal employment was only around 0.2 for the treatment and control groups prior to conclusion of the courses, the probability of formal employment jumps disproportionately for the two treatment arms relative to the control group after December 2018 and remains relatively higher until December 2019. Finally, fig. 2(c) shows also similar wages prior to the courses, and relatively higher wages for the treatment individuals compared to the control individuals after the conclusion of the courses. The jump in the outcomes of applicants in the control group is due to some applicants who end up receiving treatment (see table 3). These figures preview similar findings in the next section using our survey data, which allow us to control for other characteristics, as well as a more detailed event-study analysis of the administrative data.

4. Results

This section reports the overall effects of the job-training program, as well as heterogeneity in these effects by curricular emphasis, stipend receipt, and gender. The section ends with a cost-benefit analysis.

Aggregate Effects of Vocational Training

Figure 2(a)–(c) shows the descriptive data underlying our estimates. Even from just these descriptive figures, the aggregate results of the paper are clear. Random assignment to vocational training increases monthly formal employment and contributions (a proxy for income).

These same patterns are apparent in our event-study analysis of the data where we estimate differences between treatment and control groups (β_t) month by month, and include month fixed effects (π_t). Given that individuals are observed in more than one period, standard errors are clustered at the individual level. Since we are unable to link the administrative and survey data, these estimates are run without any control variables, course-fixed effects, or a measure of days since the course ended¹⁶:

Figure 4 shows the event-study analysis of the training program on these three outcomes. While the first course finished on 20 July 2018 and the last ended on 3 December 2018, most courses ended by November 2018. Figure 4(a) shows that days worked were very similar between treatment and control individuals up to November 2018, but that they increased sharply for those assigned to treatment relative to control individuals starting in December 2018 and were much higher for this group during the year 2019. Figure 4(b) shows a similar increase in employment probabilities for treatment relative to the control group individuals starting in December. Also, monthly wages show a similar jump in December 2018 that remains for the rest of 2019.¹⁷Figures 5 and 6 show separate effects for the two treatment arms.

Of individuals assigned to treatment, 81 percent participated in job training, while only 13 percent from the control group were taken off the wait lists (table 3). As a result, assignment to treatment increased enrollment in job training by 68 percent. While we could estimate the effects of treatment on treated individuals by scaling the reduced-form results by 0.68, we focus on the reduced-form estimates since we are interested in the effects of offering training.¹⁸

To provide a single estimate of the effects of the training programs we pool the monthly administrative records (table 4, panel A). We run these estimates using a differences-in-differences model, where we compare the outcomes of individuals before taking part in the experiment to those after all coursework ended (equation (1)).¹⁹ All estimates are clustered at the individual level. Our model allows differences in outcomes prior to treatment (β₁) as well as for changes in employment outcomes while treated individuals are exposed to the training (β₂).²⁰ The coefficient of interest (β₃) measures the average effect of the training program after December 2018:

These results suggest that the vocational-training program increased employment by 2.16 days per month, the likelihood of being employed each month by nearly 8 percentage points, and social security contributions (a proxy for wages) by US$20.66 per month.

Next, to study the effects in more detail, we turn to our survey data. Unfortunately, as is shown by the second shaded region in fig. 2, the survey was collected in February 2019, a month that appears to diverge from other months in the post-period. In panel B of table 4, we show that the point estimate for all outcomes using administrative data is near zero in this month. The results from this table suggest that the increase in employment with a contract (in table 5) may not correspond to formal employment, as measured by the social security records. Still, the estimates using survey data provide insight into what is happening under the hood, and can be used to gauge the sensitivity of our estimates to the inclusion of course fixed effects and controls.

First, we simply estimate differences in average outcomes (Y_ict) of those individuals i assigned to course c at time t to treatment (T) and those assigned to the control group, including course fixed effects (γ_c). Then we add additional covariates for baseline characteristics (X_i) and days from the final day of scheduled training.^21,22 Standard errors are clustered at the individual level. To perform these estimates, we use variations of a model of the form

The results from these estimates are reported in columns (1) and (2) of table 5. Assignment to training increases employment by 8 percentage points (15 percent) and the likelihood of having employment with a contract by 14 p.p. (50 percent), and decreased the likelihood of being a casual worker by 15 p.p. (34 percent). Additionally, we also find positive effects of training on days and hours of work in the survey data. Training assignment increased days worked per month by 2.5 days (19 percent) and hours worked per week by 4.6 hours (19 percent). These results suggest an increase in overall employment, as well as a shift towards formal employment. By contrast, we find no evidence of increased productivity as measured by hourly wages. These results are stable to the inclusion of controls (compare columns (1) and (2)) as well as to the inclusion of strata fixed effects (compare columns (1) and (2) in table S1.5).²³

These labor-market effects estimated with survey data for February 2019 are replicated using administrative data for the same month (table 4, panel B). As shown in fig. 2, compared to other months, the labor-market participation that month was low for treated individuals. In fact, using only administrative data, it looks like assignment to vocational training had no effects in February 2019. Overall, formal employment increased only a statistically insignificant 4 p.p. in the administrative data in February 2019 compared to 14 p.p. in the survey data, while the days worked and monthly income are negative—both are positive in the survey data. These discrepancies between survey and administrative results for February 2019 suggest that the effects in the administrative data may be underestimates of overall effects for other months as well.

The next four columns (columns (3–6), table 5) report estimates of vocational training separately for those with greater emphasis on social training and those with greater emphasis on technical training. The results for the social treatment arm show a shift from informal towards formal employment, while the results for the technical treatment arm show the same increases on overall employment, days and hours worked, and a shift towards formal employment found when combining both treatments. These results suggest that treatment improved labor-market outcomes and that the effect was greater for those with more emphasis on technical training.

Panel B of table 5 reports estimates of vocational training on our survey measures of social skills. These results suggest that vocational training may improve organizational and communication skills, though the results are somewhat imprecise.²⁴ This imprecision may be due to challenges in discriminating between individuals with different levels of social skills (see fig. 3). That said, the results in other areas are generally positive, suggesting that vocational training may improve social skills.

Next we take advantage of the dynamic nature of the monthly administrative data to test whether or not the benefits of vocational training fade out over time (table 6, The row labeled “Treatment X Post X Months Post” in all specifications). To estimate the dynamic effects of training on employment, we focus on the term interacting the treatment in the post-period with the number of months following treatment (equation (2)):

The third row in panel A measures any potential dynamic effects of training on employment. These coefficients are small and statistically insignificant, suggesting that we are unable to detect any evidence of fade-out of the post-training labor-market outcomes between those assigned to treatment and control groups.

We estimate the dynamic effects of vocational training separately for applicants admitted to the program emphasizing social skills (panel B) and the program emphasizing technical skills (panel C). While applicants assigned to the program emphasizing social skills sustain the benefits of job training over time, applicants assigned to the technical treatment appear to lose ground with time (row 3 of panel C).²⁵ These differences between applicants assigned to different branches of the training program are analyzed more closely in the following section.

Hard and Soft Skills in Vocational Training

We then examine differences in the dynamics between the labor-market outcomes of people admitted to the social and technical tracks using administrative data. First, to ensure that we can make a comparison between participants randomized to receive the technical- versus soft-skill training, we check for balance in observable characteristics (table S1.1), and also in rates of enrollment into and graduation from the program (table 3). These show no signs of imbalance, suggesting that any differences between the two groups can be interpreted as causal effects of program content.

Plotting the data descriptively, fig. 7 shows the intention to treat estimates (ITT) employment dynamics between social and technical treatment groups using the administrative data. Figures 7(a)–(c) show clear negative effects of social training (versus technical training) in the initial post-treatment period on days, employment probabilities, and monthly wages. As time passes, however, the differences in labor-market outcomes between the social and technical treatment groups disappear. That is, the differences between the two groups go from being negative in December 2018 and the first few months of 2019 to being zero in the second half of 2019. The differences in the employment outcomes between participants of programs emphasizing soft versus hard skills are largest in the months immediately after the training ends (see the spike in employment for the technical-skill participants).

Panels B and C of table 6 examine impacts separately for those that receive more emphasis on social and technical training, respectively. As with the survey data, these results show a larger impact for those who received more emphasis on technical training. For those with more social-skills training, there is a 0.11 increase in the probability of monthly employment—1.72 days a month—and insignificant increases of US$21 in wages. Those who receive technical-skills training experience an increase of 3.88 days worked monthly, an increased probability of employment of 0.15 p.p. and an increase in monthly wages of US$39, all significant at the 1 percent level. We are unable to detect differences in social skills between individuals randomized into the programs with a social versus technical emphasis (table 6, panel B).²⁶

Panel D of table 6 presents a formal test of whether the post-training dynamics are affected by assignment to the social- versus technical-training programs. We include a social treatment dummy, social treatment dummy interacted with a post-treatment dummy, and a social treatment dummy interacted with both the post-treatment dummy and the months since November. The negative sign in the double interaction term suggests that there may be an initial cost to social-skills training, but the positive and statistically significant coefficient for the triple interaction term of all three outcome measures (days of employment 0.28**, months of formal employment 0.01*, and social security contributions 2.35**) suggests that the gap between technical- and soft-skill-training participants dissipates over time.²⁷ This is in line with the decreasing benefits of technical skills over time suggested by Deming and Noray (2020).

Though lacking precision, differences in the effects of trainings emphasizing social versus technical skills are corroborated by estimates using survey data (table 7). Also, since we are unable to follow participants for a period of time using the survey data, we cannot capture the dynamics we are able to see in the administrative data. In line with Adhvaryu, Kala, and Nyshadham (2018), the survey data also suggest that applicants exposed to the social training may experience improvements in communication skills (0.12 SD) compared to those who receive technical training.²⁸

Does Receiving a Stipend Make Vocational Training More Effective?

Table 8 tests for whether stipend receipt magnifies or reduces the effects of admission to vocational training. Before studying the effects of the stipend, we check for balance between admits with and without the stipend (tables S1.1 and S1.2). While there are a few more differences than we might expect in demographic covariates, these differences are not jointly statistically significant. Moreover, baseline outcome measures suggest balance between stipend recipients and others. Table 8 reports the results from a specification that includes an interaction of the treatment with the stipend to see whether the training has a bigger or smaller effect when a stipend is provided to the students. At the extensive margin, the results show that providing training together with the stipend receipt had large positive and significant effects on employment. The effects on employment, days, and hours worked are larger when training and the stipend are provided together.²⁹ To test for whether these effects might be sensitive to the slight imbalance in demographic covariates at baseline, we show that these estimates are robust to the inclusion of covariates (panel A versus panel B of table 8).

This pattern of results highlights the importance of the stipend in ensuring effects of job training. As suggested by Kugler et al. (2022), one reason that the stipend may be important is that resource constraints—the cost of food and transport during the training—exhibit a serious obstacle for people, preventing them from vocational-training programs, and the stipend reduces these constraints. Another potential explanation for why stipends might be important is that, since they were given weekly to participants upon attendance, the conditionality may increase participation (Shutes and Taylor 2015). Yet another potential explanation for this difference is that stipend recipients are better funded for the duration of the job search due to increased savings.

The data available do not provide strong evidence to distinguish between these hypotheses. There are no effects of stipend receipt on enrollment in job training. In line with the potential importance of conditionality, the point estimate on graduation is higher upon stipend receipt, even if statistically insignificant (table 3). However, this small difference in program completion is too small to explain the magnitude of the differences in the effects of job training by stipend receipt. Unfortunately, we do not have data on daily attendance or job search to test these hypotheses.

Differential Impacts by Gender

To understand whether the job-training program affected men and women differently, we study differential effects of the program by gender (tables 9 and 10). In order to run these pooled models, we completely saturate the regression with interactions of all demographics with the gender indicator.³⁰ Though differences in effects for men and women are statistically insignificant for most outcomes, the coefficient on almost all the point estimates is negative for females (third row), and we see clear evidence of heightened effects for men compared to women in terms of monthly earnings. Likewise, table 10 also suggests that men were more likely to benefit from vocational training in terms of acquiring interpersonal skills than their female counterparts.

These results contrast with previous evaluations of training programs for young people, which find bigger effects of training on young women (Attanasio, Kugler and Meghir 2011; Card et al. 2011; Acevedo et al. 2020). Some potential reasons for differences between our study and other studies in Latin America are that there are differences in the target population (older) and industry mix (service sector) in Cali, Colombia. In addition, the training program we evaluate provides more in-depth social-skills training, a component that appears to be more useful for men.

There are several reasons why the program we study may not have been effective for women. First, it is possible that the program was ineffective in providing women with the skills they needed in the labor market. This is supported by the fact that the gains in social skills accrue almost entirely to men. Second, it is possible that women face additional constraints due to responsibilities at home. Third, women may experience greater discrimination in the labor market. Of these three potential stories, it is important for policy makers and practitioners considering the design of training programs with a soft-skill component to ensure that the training equitably provides skills to both men and women.

Cost-Benefit Analysis

Although the program may have non-wage benefits, wage returns represent a lower bound for a program’s benefits. The reduced-form estimates from the administrative data suggest that, on aggregate, program participants experience a monthly US$21 wage gain by participating in the program for the first year following program participation. We focus on estimates from our administrative rather than survey data because they are measured more accurately, offer insight into effects across a greater span of time, and because the endline survey was, unfortunately, collected during February, a month not representative of typical labor-market conditions (see table 4). Still, comparing the administrative data estimates in February 2019 to those from survey data suggests that the effects using administrative data may, if anything, underestimate the effects of vocational training. Moreover, the wage returns recorded in administrative data do not include the non-pecuniary benefits of formal employment, which can provide non-pecuniary benefits such as health insurance and social security. Given the average age of program participants of 26, we assume that they will work another 35 years. Since we can only follow participants for one year after the program, the overall cost-benefit analysis will hinge on assumptions regarding whether or not the benefits individuals experience are permanent, or whether they fade out over time. In panel D of table 11, we present two scenarios: one in which the benefits of the program persist, and another where we assume 10 percent annual depreciation. Under the first scenario, participants experience a benefit of US$4,262 over their lifetimes, whereas under the second scenario participants experience a US$1,186 benefit over their lifetimes.³¹

Including the stipend, the direct marginal costs of operating the program are US$176. As such, the two scenarios represent lifetime wage gains of US$4,086 and US$1,010 respectively. The cost-effectiveness of the vocational programs emphasizing social versus technical skills are similar. Put another way, our results suggest that the vocational-training program pays for itself in about eight months. To estimate the treatment on treated, we would divide these estimates by 0.60, suggesting that the program would pay for itself even sooner—in just over four months. Of course, the COVID-19 pandemic may affect the returns to vocational-training programs in unexpected ways. In a new working paper (Barrera-Osorio, Kugler, and Silliman 2021), we find that the pandemic-induced recession fully erased the gains from the program. Yet, given that the program pays for itself in four to eight months, the costs of this program would have been covered prior to the time COVID-19 hit Colombia.

5. Discussion and Conclusion

We use a randomized experiment in Cali, Colombia to study the effects of vocational training on labor-market outcomes. In aggregate, vocational training shifts people to the formal sector of the labor market and increases their monthly earnings and overall employment. Importantly, the results show positive impacts both in the short and medium terms, with effects persisting up to 12 and 17 months after the program.

The key novelty of this study is that we examine how providing different intensities of soft or technical skills as part of vocational training affects labor-market dynamics. Admits to vocational training are randomized to receive varying degrees of social-skills training. Our results show that both programs with an emphasis in social and technical skills have a positive impact on labor-market outcomes, the program with emphasis on technical skills has a bigger short-term effect. However, those with an emphasis on soft skills catch up to those who receive more technical training about 6 to 12 months later in terms of employment, earnings, and hours worked. This catch-up is largely a result of an erosion of the benefits from the technical training.

These results provide experimental evidence that while technical skills may improve immediate labor-market outcomes, these skills—in contrast to social skills—may become fragile with time. Our paper contributes to the literature on soft skills in the workplace. First, these results suggest that soft skills can be taught even to older people. Second, there are only a handful of randomized trials on soft-skills training. Groh et al. (2016) find no effect, but their training takes place in a nine-day period and only lasts a total of 45 hours. Acevedo et al. (2020) find only short-term impacts on young women but not on young men in the Dominican Republic. However, not only was soft-skill training offered after hours and it is possible that young men did not attend, but it is not possible to disentangle whether the effect of this program was due to an internship offered in conjunction with the soft-skills training. Like our study, Adhvaryu, Kala, and Nyshadham (2018) show that soft-skills training to female government workers in India increases their communication skills but only had small effects on labor-market outcomes. Unlike our study which finds larger effects, they do not include men, nor do they contrast between soft- and technical-skills training. Further, these results highlight the importance of providing stipends that help training participants with resources for food and transport, and show that the training studied largely benefited men.

Footnotes

Data Availability Statement

Data available on request.

Notes

Felipe Barrera-Osorio (corresponding author) is a professor at Vanderbilt University, Nashville TN, USA; his email address is [email protected]. Adriana Kugler is US Executive Director of the World Bank, an NBER, CEPR, and IZA affiliate, and on a leave of absence from Georgetown University; her email address is [email protected]. Mikko Silliman is a doctoral student at Harvard University; his email address is [email protected]. We thank the Carvajal Foundation for partnering with us to run and implement the program, and in particular, Mario Gonzales, Ana Enriquez, and Angela Gonzales. We are also grateful to the staff at the Ministry of Health and Social Protection for providing us with the administrative data. We thank David Deming, Nada Eissa, Robert French, Carolina Gonzales, Carmen Pages, and Ken Wolpin as well as seminar participants—including Isaac Mbiti, Derek Neal, Lesley Turner, and Hanna Virtanen—at E-con of Education, Vanderbilt, The German Institute for International Affairs (GIGA), LACEA-LAMES, and the Research Institute for the Finnish Economy (ETLA) for helpful comments, and Milagros O’Diana for expert research assistance. Carvajal provided funding for the RA who helped with randomization, design of surveys, sample calculations, data cleaning, and initial analysis as well as trips to do the fieldwork. A supplementary online appendix for this article can be found at The World Bank Economic Review website.

References