Media literacy interventions: meta-analytic review of 40 years of research

Abstract

We conducted a comprehensive meta-analysis of 160 media literacy interventions from 1983 to 2023, finding overall positive effects. Effects were larger for media-over behavior-relevant outcomes and for social over health topics. Stronger effects were found for knowledge outcomes at both immediate-post and delayed follow-up assessments. Attitudes and critical beliefs predicted behavior. No substantial decay in effects was observed between immediate-post and follow-up assessments for multiple outcomes but not for behavior. Interventions targeting social media showed smaller effects for some outcomes compared to those that did not. Online delivery had smaller effects for various outcomes compared to in-person delivery. These findings suggest challenges and opportunities regarding digital and social media for the interventions. Dose was inconsistently linked to outcomes. No significant relationship was found between study year and effect size. The heterogeneity of effects observed across variables suggests a need for more parsimonious frameworks in media literacy research.

Media literacy interventions refer to educational programs aimed at equipping individuals with the knowledge and skills to make informed judgments and decisions in their interactions with the media. Representing an important nexus between communication research and societal concerns about harmful media effects, the interventions have been developed and evaluated over the past 40 years. The proliferation of digital and social media has increased the importance of media literacy, as seen in the growing number of states requiring media literacy education in K-12 curricula. Currently, about 17 states in the United States require some media literacy instruction or standards or utilize other legislative solutions (Media Literacy Now, 2022).

Jeong et al. (2012) were the first to offer a meta-analytic review of media literacy interventions. Covering studies spanning 1983 to 2010, they provided an average effect size of the interventions and examined the moderators of their effects. This review offered valuable initial insight into media literacy interventions. However, the review was limited to a smaller set of studies (k = 51). Since then, over 100 additional studies have been identified, demonstrating an accelerated rate of adoption, which in turn necessitates a more comprehensive review.

The smaller number of studies in the initial review constrained the moderator analyses to combine heterogeneous outcomes, hindering the analysis of the extent to which the variation in effect sizes was due to sampling error or true population variation. Other meta-analyses of media literacy interventions have since emerged, reflecting the increasing interest in understanding their effects. However, these analyses focused on intervention effects involving specialized topics or specific populations (e.g., Hindmarsh et al., 2015; Vahedi et al., 2018; Xie et al., 2019).

Therefore, a more comprehensive and up-to-date meta-analysis is needed. The goal of the present study is to provide a comprehensive meta-analytic review of media literacy interventions over the past 40 years (1983–2023) and to offer a more nuanced analysis of their effects and contingent conditions. Based on this analysis, we offer suggestions for the development of theory and research on media literacy and interventions efforts. Findings from this meta-analysis can inform researchers, educators, and policymakers, providing guidance on effective approaches to capitalize on and meet the challenges posed by the evolving media ecosystem.

Media literacy interventions

Conceptualizing media literacy

Foundations

A classic conceptualization of media literacy was formulated at the National Leadership Conference on Media Literacy in 1993. A report on this conference (Aufderheide, 1993) produced a definition of media literacy which is perhaps the most widely utilized: “the ability to access, analyze, and produce information for specific outcomes” (p. v). Researchers have viewed knowledge and skills as central to attaining media literacy. Knowledge consists of sets of organized information within an individual that provide the contexts which individuals can use to make sense of media messages, while skills are tools with which to make sense of media messages, and include analysis, evaluation, grouping, induction, deduction, synthesis, and abstraction (Potter, 2019).

Media literacy interventions have used a range of theoretical frameworks. Inoculation theory (McGuire, 1964) serves as a guiding principle of media literacy education. It postulates that just as medical vaccination can protect people from potential future threats of a disease, providing an audience with a sampling of future attacks by harmful media and equipping them with refutational preemption strategies can prevent the danger of unhealthy media exposure and its effects on audiences. Social cognitive theory (Bandura, 1986) explains how cognitive processes affect individuals’ behavior in interaction with personal and environmental factors. Interventions using this theory have often aimed to address how media, as a social environment, influences the audience’s cognitions and behaviors. The goal of these interventions has been to alter beliefs about the outcomes of behavior and the environment. The media interpretation model (Austin & Johnson, 1997a) delineates the components and processes that media users may employ to come to a behavioral decision. It posits that behavioral decisions are based on logic and affect. In logical comparison, users evaluate whether the media messages are an accurate representation of reality. In affective evaluation, users examine whether they like the messages. Based on these analyses, users consider the outcomes of adopting a behavior, leading to behavioral decisions.

Expanding the scope

As media literacy has evolved, scholars have expanded its conceptualization to encompass the interplay between the content and the medium. Meyrowitz (1998, 2009) has argued for multiple media literacies, shifting the focus from the content of the media as the primary source of influence (i.e., content literacy) to examining the unique characteristics and capacities of each medium and their impact on individual behaviors and social interactions (i.e., medium literacy). The proliferation of new media confers heightened relevance to this perspective.

With the advent of digital and social media, further scholarly attention has been directed to the potential of the medium. Jenkins (2009) advocated for a digital participatory culture, emphasizing the importance of social skills and cultural competencies. Hobbs (2010) proposed the integration of reflection and action in digital and media literacy to empower users and support their lifelong learning. Extending this, Mihailidis (2018) reframed media literacy to develop civic media literacies in which intentionality—concern for the common good—is central. These conceptualizations embrace the participatory potential of media literacy for positive social impact, harnessing digital and social media.

New directions

Most recently, Cho et al. (2024) introduced a framework for social media literacy that moves away from the mass media-based premises to recognize the dynamic interaction between the user’s self and their choices of messages and networks. In the traditional mass media-based approach, the media and users were discrete entities in separate spheres, allowing for detached analysis and evaluation. This boundary is porous in social media, where content is selected, fed back, and refined through iterative interactions between the media and the user. Consequently, in this social media literacy framework, the analysis focuses on the self and its choices, while evaluation centers on one’s views and experiences resonating with the chosen content. In contrast, mass media literacy focuses on media content and its values presumed to be independent of the user.

Based on this review, media literacy interventions can be defined as planned actions to develop individuals’ abilities to analyze and evaluate the media in ways that foster informed engagement. This conceptualization expands on traditional views. Historically, a fundamental objective of media literacy was “critical autonomy in relationship to all media” (Aufderheide, 1993, p. 1), which emphasizes independence from media influence. The new perspective highlights the importance of the awareness of the interdependence between the media and the user. From this viewpoint, a media literate person is cognizant of the mutual influence between media messages and personal and social contexts, understanding how media shape and are shaped by one’s beliefs, values, and experiences (Cho et al., 2024).

While these developments reflect changes in the media landscape driven by digital and social media, the core pillars of media literacy have remained constant: critical analysis and evaluation of the media. Given the increasing importance that media literacy conceptualizations place on medium itself, research is needed to investigate the capacities of diverse media and their interplay with content, as well as the distinct characteristics of the content.

Outcomes of media literacy interventions

Guided by theoretical frameworks such as inoculation theory, social cognitive theory, and the media interpretation model, media literacy interventions have sought to impact a broad range of outcomes. Generally, these outcomes encompass media-relevant outcomes and behavior-relevant outcomes (Jeong et al., 2012).

A major media-relevant outcome is knowledge (Potter, 2019). Studies on this outcome have assessed understanding related to media industry practices (Solhi et al., 2017) and media influenced beliefs (Sciarrillo, 2017). Another media-relevant outcome is perceived realism, which refers to judgments about the degree to which media depictions reflect the real world within the media effects research tradition. The logical comparison component of the media interpretation model (Austin & Johnson, 1997a) aligns with perceived realism. Interventions have used this concept by contrasting media portrayals with the actual reality of objects and events, to raise awareness about the nature of media messages and their relation to reality (e.g., Austin & Johnson, 1997b; Liao et al., 2016).

Critical beliefs encompass a group of interrelated concepts that together form a skeptical stance towards media content, sources, and techniques. This construct integrates divergent terms including media skepticism (e.g., Chen, 2013), critical thinking (e.g., Austin et al., 2020), understanding media myths (e.g., Pinkleton et al., 2012), media deconstruction skills (e.g., Scull et al., 2014), perceived media literacy, media literacy beliefs, and news media literacy (e.g., Tully & Vraga, 2017; Vraga & Tully, 2015). Despite being labeled differently in various studies, their conceptualizations and operationalizations share a common underlying meaning: the ability to question and scrutinize the media. As concern over misinformation rises, developing critical beliefs can be instrumental to combating it.

Media literacy interventions can also influence decisions and actions (Austin & Johnson, 1997a). Theories of action (Ajzen, 1991; Fishbein & Ajzen, 1975) and the integrative model of behavioral prediction (IMBP; Fishbein & Yzer, 2003) suggest that a set of variables are relevant to behavior. These include outcome expectancies, attitudes, social norms, and efficacy beliefs. Outcome expectancies refer to the beliefs about the positive or negative outcomes to be obtained after following the practices presented in media messages. Because research has often found that the media tend to highlight the positive outcomes to be gained after practicing the risk behavior featured and omit negative outcomes associated with the same behavior, interventions were designed to correct these beliefs. Attitudes are the evaluation of a behavior which can be based on outcome expectancies (Fishbein & Yzer, 2003). Social norms pertain to the perceptions about the prevalence, frequency, and acceptance of a behavior presented by the media (e.g., Lucidi et al., 2017). Efficacy refers to beliefs about one’s ability to carry out a given course of action (Bandura, 1986; Fishbein & Yzer, 2003). Finally, theories of action and IMBP postulate that behavioral intentions are an immediate predictor of behaviors.

Furthermore, theories predict that media-relevant outcomes can influence behavior-relevant outcomes. The media interpretation model (Austin & Johnson, 1997a) suggests that media-induced cognitions and affect can determine media users’ subsequent decisions. IMBP (Fishbein & Yzer, 2003) suggests that media exposure can give rise to attitudes, perceived norms, and efficacy beliefs which then predict behavioral intention and behavior.

On this basis, we expect that media literacy interventions will increase knowledge and critical beliefs about the media, while decreasing perceived realism of the media (H1a). Concurrently, it is expected that media literacy interventions will decrease positive outcome expectancies, attitudes, norms, efficacy beliefs, and behaviors related to risk or antisocial behaviors, while increasing the same factors for health or prosocial behaviors (H1b). We further anticipate the size of media-relevant outcomes to positively predict the size of the behavior-relevant outcomes (H1c).

Long-term effects of media literacy interventions

Because media literacy constitutes abilities that one should possess and utilize throughout one’s lifelong relationship and interactions with the media (Aufderheide, 1993; Hobbs, 2010; Potter, 2019), it is necessary to examine temporal changes in intervention effects. In keeping with the importance of fostering enduring abilities, media literacy interventions have often employed delayed follow-ups, although some relied on immediate post-test only. Long-term effects, however, were not investigated in the initial review by Jeong et al. (2012).

In examining longitudinal effects of interventions, two aspects are particularly important: decay and the interval between the intervention and the follow-up (e.g., Flay et al., 2005). Duration of effect is a core criterion for determining efficacy: “In general, for outcomes that may decay over time, there must be a report of significant effects for at least one long-term follow-up at an appropriate interval” (Flay et al., 2005, p. 161). It is considered that a long-term efficacy assessment should be at least six months after the intervention (Flay et al., 2005). Media literacy interventions have utilized longer-term follow-ups, with some studies assessing effects after one to three years (e.g., Gonzalez et al., 2011; Gordon et al., 2021). Examining decay in effects could inform intervention design aspects, such as dose and dose interval and frequency, in various educational settings. A related question is whether the interval between the intervention and the follow-up assessment impacts intervention effects. Media literacy interventions have frequently employed longer term delayed effects assessment.

Another important longitudinal question is whether the effects of the interventions have changed over the 40-year timespan that this study is based on. It is possible that the programs and techniques have become more sophisticated and therefore more effective. Alternatively, the changing media ecosystem and usage behavior fueled by digital and social media may have posed challenges to the more traditionally designed media literacy programs. Therefore, we propose to investigate whether the effects of media literacy interventions diminish over time (RQ1a), whether the time interval between the intervention and the follow-up assessments impacts the outcome (RQ1b), and whether there is an association between study year and effect size (RQ1c).

Characteristics of media literacy interventions

In addition to the long-term effects discussed above, we investigate other new dimensions of media literacy interventions not examined in Jeong et al. (2012). These new features include the roles of digital and social media. We also revisit other key aspects of the interventions initially investigated in Jeong et al. (k = 51). This was deemed necessary and important as about 100 more studies have accumulated since 2012. Jeong et al. identified three primary features that may moderate effects of these interventions: the dose and source of the intervention and audience involvement. However, their initial investigation, using a smaller sample, found no significant results, demonstrating the need for further scrutiny.

Dose, source, and audience involvement

Determining the optimal dose is a critical factor in developing interventions (Voils et al., 2014). One approach to selecting dose parameter values is to conduct a retrospective analysis of data from existing interventions (Voils et al., 2014). As media literacy interventions have used varying numbers of sessions, it is important to examine whether and how these factors influence effects, so that future interventions can be optimally designed. Understanding dose effects can also inform implementation efforts, particularly in educational settings, as time and resources can be limited for both the participants and the program.

The source of the intervention concerns the individuals that presented the program to the audience. Two primary types of interventionists have been identified: peers (e.g., students, community members) and experts (e.g., teachers, researchers). The source is an important factor in determining the effectiveness of interventions (Rothman & Sheeran, 2020). However, past research has yielded mixed results. Some studies found that peer-delivery was more effective (e.g., Kelly, 2004), while others reported that expert educators were more effective than non-expert educators (e.g., Covey et al., 2016; Durantini et al., 2006). A systematic review found no significant differences in behavioral outcomes between interventions delivered by peer educators and by non-peer educators (e.g., Webel et al., 2010). Expert-led interventions may be more effective because of their knowledge and skills, while peer-led interventions may be more effective due to perceived similarity and identification (Kelly, 2004).

Audience involvement can be another crucial factor to examine. Media literacy interventions have used multiple levels of audience involvement, ranging from didactic modes to more active modes such as discussion and activity among participants. Perhaps the most involving is the production component of media literacy interventions, where participants design their own counter messages against dominant media messages endorsing unhealthy behaviors. In various other domains of interventions, participant involvement has been found to increase intervention effectiveness, although facilitating engagement remains a challenge (e.g., Perski et al., 2017). However, Jeong et al. (2012) found that higher levels of audience involvement in media literacy interventions were unrelated to increased effectiveness. Therefore, in our second research question, we propose to examine whether media literacy intervention effects differ per (RQ2a) dose, (RQ2b) source, and (RQ2c) audience involvement.

Digital and social media

Digital media have offered new opportunities and challenges for media literacy interventions. Whereas earlier media literacy interventions were delivered in-person at physical locations, a growing number of recent interventions have utilized online delivery. Generally, online and in-person delivery each offer distinct advantages over the other. Online delivery enables remote reach and access, thus potentially impacting a far larger number of people than in-person delivery. Online delivered interventions can also eliminate fidelity issues associated with in-person delivered ones (Dusenbury et al., 2003), with robust control over the dose delivered (cf. dose received), a key component of fidelity integral to internal validity of the intervention (Berkel et al., 2011).

Concurrently, in-person delivery provides unique advantages that online delivery cannot. In-person communication comes with a rich set of nonverbal cues that online communication may not afford (Kiesler et al., 1984). These cues can facilitate communication, potentially leading to greater engagement, understanding, and learning. On the other hand, perspectives such as social information processing theory (Walther, 1992) and hyperpersonal theory (Walther, 1996) suggest that the disadvantage of reduced nonverbal cues can be offset by other cues and features available in online environments. Considering the respective advantages and disadvantages of each delivery mode, which produces greater effectiveness for media literacy interventions has yet to be determined.

Citizens’ media use behaviors have greatly changed with the diffusion and dissemination of social media. As reviewed earlier, the core pillars of media literacy, critical analysis and evaluation of media messages, remain constant across both mass media and social media. However, the focus for the analysis and evaluation should differ for several reasons (Cho et al., 2024). Mass media are a backdrop for the conception and practice of media literacy. In this environment, media users were rather passive receivers of information and entertainment created and transmitted by media institutions and professionals. The nature and functions of social media differ from those of mass media in various dimensions. Whereas the mass media and users exist as discrete entities in separate spheres, allowing for a more detached analysis of media messages, social media are coinhabited by both the media and the user.

Specifically, social media platforms are sites where users’ interactions with peers, influencers, and personal networks shape and reinforce their reality. Public concern over the harmful effects of social media on users’ health has been increasing. In response, a growing number of media literacy interventions have sought to counteract these unhealthy effects. Based on the preceding discussion, our third research question concerns the roles of new media related features. First, we ask whether intervention effects differ between in-person and online delivery modes (RQ3a). Second, we examine if the effects vary based on whether the intervention addresses social media effects (RQ3b).

Topic

In addition, it is plausible that the effects of the interventions are contingent upon the topic area being addressed. Topics of media literacy interventions may be generally categorized into two types: health topics and social topics. Health-related topics, such as alcohol, sex, and food, may have immediate relevance to personal well-being. Social topics, encompassing advertising, technology, and misinformation, often rise from broader societal concerns. The connection between these social topics and personal wellbeing can be more indirect compared to health topics. We ask whether the topic moderates media literacy intervention effects (RQ4).

Study location and publication status

Finally, study location and publication status are essential moderators to be investigated. The United States, where the majority of the interventions have been implemented, has different systems of media ownership and regulation from those in other countries (Hoskins et al., 2004). Publication status should be examined to avoid bias in effect estimation, as published studies are more likely to have statistically significant positive outcomes compared to unpublished studies. Our final research question asks whether media literacy intervention effects differ per (RQ5a) study location and (RQ5b) publication status.

Methods

Search

Various databases were used to search for published and unpublished media literacy interventions. Databases used for the search included Communication Abstracts, PsycINFO, MEDLINE/PubMed, ProQuest Dissertations and Theses, Education Research Complete, Education Full Text, and Google Scholar. We included the two education research databases to expand the breadth of the search from Jeong et al. (2012) because education is an area where media literacy has been actively studied. Since Jeong et al. included media literacy interventions published up to December 31, 2009, the search for this study focused on identifying articles published from January 1, 2010 to August 31, 2023. The search terms used for this study were the same as Jeong et al. (2012) and included: “media literacy,” “media literacy intervention,” “media literacy curriculum,” and “media literacy program.” We employed these search terms in combination with Boolean logic to identify published studies and unpublished reports in English. Additionally, we scanned the reference lists of review articles on media literacy interventions to identify additional studies (e.g., Andersson & Danielsson, 2021; Hindmarsh et al., 2015; Rasi et al., 2021; Scharrer & Ramasubramanian, 2015; Vahedi et al., 2018; Xie et al., 2019)

Selection

The initial database searches yielded a total of 10,106 articles. After removing duplicates, 6,856 articles remained. For inclusion, studies had to use empirical quantitative methods using experimental design and reporting statistical information. Removing irrelevant articles resulted in 143 articles. Out of these 143, 106 were retained after further examination, resulting in the exclusion of 37 articles which were not interventions (n = 9), were not focused on media literacy (n = 5), did not use quantitative methods (n = 2), were not published in full text (n = 2), or did not include one of the specified outcomes (n = 19). During this process, we contacted the corresponding authors of all articles with missing statistical information to request the data. The scanning of the reference lists of the six review articles identified three additional articles, resulting in a total of 109 articles. Combining these with the studies from Jeong et al. (2012), the present meta-analysis included a grand total of 160 articles, shown with sample sizes and effect sizes for each outcome in Table 1. Screening of studies was done by one investigator in consultation with the rest of the investigators. We took the following steps to ensure accuracy. To prevent exclusion of relevant articles, we tested various keyword combinations and compared the number and coverage of articles for comprehensive retrieval. To avoid including irrelevant articles, the investigator manually screened titles and abstracts, followed by a full-text review. Any uncertain cases were discussed among all investigators for appropriate inclusion.

Outcomes

The outcome variables examined in this study included knowledge, perceived realism, outcome expectancy, social norms, efficacy beliefs, critical beliefs, attitudes, behavioral intentions, and behavior. Regarding variables such as knowledge, realism, expectancy, norms, and efficacy, they have been consistently labeled across the interventions. As discussed earlier, the critical beliefs variable subsumes a class of interrelated variables. These included skepticism, criticism, critical thinking, perceived media literacy, media literacy beliefs, news media literacy, and understanding of media myths. Interventions treated and assessed these variables as beliefs. Despite different labels, these variables share a common ability to question, criticize, and critically analyze and evaluate media messages. Because there is insufficient theoretical basis indicating that these variables differ significantly, we grouped them as critical beliefs. Finally, we combined behavioral intention and behavior because these variables show high correlation (Fishbein & Ajzen, 1975; Kim & Hunter, 1993).

Moderators

This study examined the following moderators: dose, source, audience involvement, delivery mode, social media targeting, topic, study year, measurement time, study location, and publication status. The dose of the intervention was operationalized as the number of intervention sessions. The sources of the intervention were coded as peer, researcher, or teacher. Audience involvement was assessed based on whether the intervention included one or more active involvement components including discussion, activity (e.g., games, role-playing), and production (generation of counter-messages) by participants. For delivery mode, we coded whether the intervention was given in-person or online. We also coded whether the intervention sought to address social media effects. The topics addressed by the interventions generally fell into two main types: health issues and social issues. Health issues included topics such as alcohol, tobacco, sex, violence, food, substance use, and body image. Social issues included advertising, technology, misinformation, media perception, and media literacy.

For the publication year, we coded the year when the journal article or other forms of study reports (e.g., conference proceeding, dissertation) were published. Measurement time was coded as immediate post-test or delayed post-test. Location of the study was operationalized as the U.S. or non-U.S.-based. The U.S. was the location where 65% of the studies were conducted, although the rest represented a wider range of continents and countries than those in Jeong et al. (2012) including: Africa, America, Asia, Europe, Middle East, and Oceania. Publication status included published and unpublished categories.

Coding of moderators was done by one investigator in consultation with other investigators throughout the process. Initially, two investigators independently coded a random sample (n = 25, about 15% of the total sample), reaching complete agreement across the variables. Later, another random sample (n = 32, about 20%) was validated by the same secondary coder. Any discrepancies were resolved through discussion.

Statistical analysis

The meta-analytic calculations that follow used Schmidt and Hunter’s (2015) random effects methods for producing meta-analytic estimates. The population-level mean correlation (ρ) was estimated by taking the sample-sized weighted mean with corrections for measurement error. Positive correlations indicate that the intervention had the desired effect for that outcome. Estimates were computed using the R statistics (v. 4.2.1) package psychmeta (v 2.6.5; Dahlke & Wiernik, 2019), which was also used to estimate confidence intervals, credibility intervals, and the percentage of variance explained by sampling error and measurement error. For meta-analytic estimation and its interpretation, we relied on the recommendation of having ten or more studies (e.g., Kepes et al., 2012; Schmidt & Hunter, 2015). The data underlying this article will be shared on reasonable request to the corresponding author.

Results

Overview

While it may be inaccurate to combine disparate outcomes in a meta-analysis due to differences in populations (Carpenter, 2020), comparing the overall effect in this meta-analysis with the estimate in Jeong et al. (2012) may provide valuable insight. They found a weighted mean effect size of r = 0.18. Averaging across outcomes and focusing on immediately measured outcomes, the current set of 160 studies shows a weighted average effect of r = 0.17.

For each outcome shown in Table 2, outcomes are divided by the measurements that occurred immediately after the intervention (Immed) and those that were measured after some delay (Delay). Table 2 shows the meta-analytic estimates for the population correlation coefficient (ρ), the number of studies in each estimate (k), and the confidence interval. The 80% credibility interval indicates how widely the estimate is expected to vary at the population level based on moderators of the effect. Thus, setting aside sampling and measurement error, Table 2 shows the true population effect of media literacy interventions on knowledge is expected to be between 0.00 and 0.56 about 80% of the time. As a result, there are moderators that may lead to these interventions having no effect on knowledge, while others may cause a strong effect, even without the sampling error that causes the confidence interval. In general, the wider the credibility interval, the broader the range of values one could expect in replication studies, even with very large samples with minimal measurement error. The percentage of variance explained, shown in the last column, indicates how much of the variation in effects can be attributed to sampling and measurement error. Schmidt and Hunter (2015) argued that if 75% of the variance can be thus attributed, it is likely a homogeneous effect such that the estimate of the effect size is unlikely to vary in the population.

As the percentage of variance explained column in Table 2 shows, no effects in this study were homogeneous and the 80% credibility intervals show that some are expected to vary quite widely. Examination of the variance explained columns in Tables 3–7 (see also Supplementary Tables S1 and S2) shows that although some moderators were associated with substantial variation in effect sizes, very few reached the 75% of variance explained threshold needed to indicate that the effect was unlikely to vary further because of additional moderators. Therefore, nearly all of these estimates should be interpreted with the caveat that although the differences in effects we discuss are likely, additional moderators may alter the size of these differences. Forest plots for the largest sets of outcomes are shown in Figure 1 (critical beliefs) and in Supplementary Figures S1 (attitudes) and S2 (behavior).

Outcomes of media literacy interventions

Impact on media- and behavior-relevant outcomes

H1a-b predicted that media literacy interventions, relative to controls, would increase positive media-relevant and behavior-relevant outcomes while decreasing negative outcomes. The results in Table 2 in the ρ column show the meta-analytic estimates for the population correlation coefficient such that positive values indicate that the media literacy intervention had the desired effect, relative to the control. Those results of the meta-analysis show that media literacy interventions tend to produce effects in the predicted directions, for both cognitive and behavioral outcomes. Examination of the 80% credibility intervals (80% CredI column) suggests that although moderators may occasionally make the effects a little lower than zero, they will rarely cause media literacy interventions to have substantially negative effects.

Examination of the ρ column in Table 2 shows that effects are generally more pronounced for media relevant outcomes compared to behavior relevant outcomes. In particular, the largest effect was for the delayed effect for perceived realism, but this effect was estimated with less than ten (k = 7) studies that measured perceived realism at a later post-test (see k column of Table 2). Therefore, these estimates should be interpreted with caution. The next largest effects were found for knowledge outcomes, both immediately after the intervention and after a delay before another measurement. However, as with realism, fewer than ten studies measured knowledge at a delayed post-test (k = 9).

Relationship between outcomes

We also tested H1c, predicting a positive association between the sizes of the effects of interventions on the various media-relevant outcomes and the sizes of the effects of the interventions on behavior-relevant outcomes. Only for critical beliefs and attitudes was there a sufficient number of studies (k ≥ 10) examining both behavior and each of these theoretical predictors. Of the two, attitudes had a very strong relationship with behavior (r = 0.61, k = 23, p < .05). The size of the relationship between critical beliefs effect sizes and behavior effect sizes was also ample (r = 0.44, k = 34, p < .05).

Decay

Examination of the ρ column for the effects of the interventions that were measured sometime after intervention exposure (Delayed) in Table 2 provides insight into RQ1a, which asked whether the effects of media literacy interventions decline over time. Given the enhanced difficulty of assessing outcomes longitudinally, there were far fewer studies with longitudinal outcomes. But for a few, there were a sufficient number of studies to make an estimate. In particular, for the outcomes of knowledge, critical beliefs, attitudes, and behavior, there were a sufficient number of studies to make a meaningful comparison by comparing the immediate effects and the delayed effects in the ρ column. This dataset showed only small differences between the effect size for studies with an immediately measured outcome and those with a delayed measurement, except for the case of behavior. Although the effects are often small, they do not appear to decay substantially for the variables of knowledge, critical beliefs, and attitudes.

Time interval

We explored whether a longer time interval between the end of the intervention and the delayed measurement would affect the intervention’s impact on outcomes at those later measurement points (RQ1b). There were only three outcomes with sufficient studies to make an estimate. There appeared to be no substantial effect on attitudes, (r = 0.02, k = 32) suggesting that media literacy interventions have long-lasting impacts on attitudes. For critical beliefs, there appears to be a small negative relationship (r = −0.11, k = 33) which suggests there may be some small decay after the intervention ends and a moderate effect for behavior, also suggesting more substantive decay (r = −0.29, k = 36).

Publication year

RQ1c was to investigate the association between effect size and publication year. If there was a negative correlation between study publication year and study effect size, there would be evidence of a decline effect. Examination of Table 8 shows inconsistency in the extent to which a decline effect may be occurring across time. Decline effects are represented by a negative relationship for some outcomes such that older studies produced stronger effects shown by the negative correlation (immediate knowledge, immediate expectancy, immediate attitude, immediate realism, immediate efficacy, and immediate norms). Yet, the correlations were inconsistent for the rest and even somewhat positive for critical beliefs. The inconsistency of a clear decline effect over time suggests that other moderators of the effect are more likely candidates for explaining variation in effects.

Characteristics of media literacy interventions

The categorical moderators proposed for RQs2-5 included: source, audience involvement, delivery channel, social media targeting, topic, study location, and publication status (results shown in Tables 3–7 and Supplementary Tables S1 and S2, for each outcome). The continuous moderator proposed in RQ2 was dose or the number of sessions in the intervention (shown in Table 9).

Dose

The correlations between effect sizes and the number of sessions varied (Table 9). Here a positive correlation would indicate that more sessions produced a stronger desirable effect. For immediately measured knowledge, attitudes, and realism, there were negative correlations suggesting that more sessions produced worse outcomes. There were somewhat smaller negative effects for both immediate and delayed behavior. Yet, for immediately measured efficacy, there was a larger positive effect. Positive associations were also found for immediate critical beliefs and norms. The inconsistency in the effect of the number of sessions suggests that this variable may not be a consistent way to produce a better outcome.

Source

We examined the extent to which using a peer, the researchers, or the audience’s teachers to deliver the intervention affected intervention success. Examination of the ρ columns in Tables 3–7 and Supplementary Tables S1 and S2 for the effects of the Source moderator shows that for knowledge (see Table 3) and perceived realism (Table 4), researcher or teacher-based interventions appear to have yielded more effective interventions than those using peers for interventions. Yet for immediately measured attitudes and efficacy, peer-based interventions appeared to show stronger effects than both teachers and researchers for attitudes (see Table 5) and researchers for efficacy (Supplementary Table S1). For all outcomes, however, peer-based interventions were often only about five, making comparisons less than certain. For behavior, there was only one peer-based intervention to assess this moderator.

Audience involvement

The variation in approaches to creating audience involvement in the intervention itself was associated with some variation in the effectiveness. Examination of the ρ columns in Tables 3–7 and Supplementary Tables S1 and S2 for the effects of the Audience Involvement moderator. The results were first broken down by examining each approach’s absence or presence separately. The effects associated with those interventions using all three were also estimated. Generally, no coherent pattern emerged from the results. The presence or absence of a discussion element in the intervention appeared to affect the size of the effects only trivially with a few outcomes showing slightly stronger effects. Notably, some interventions showed stronger effects when they did not include either activities or production. For example, for the outcome of efficacy, interventions without activities were more successful than those with activities (see Supplementary Table S1). In the case of realism, studies without production were more successful than those that included one (Table 4). Overall, increasing audience involvement did not seem to have a consistent effect on the effectiveness of media literacy interventions.

Delivery mode

We also examined the extent to which interventions delivered in-person might produce different effects than those delivered online. Examination of the ρ columns in Tables 3–7, Supplementary Tables S1 and S2 for the effects of the Delivery moderator shows an interesting pattern. Across most outcomes, the effect size was slightly smaller for online-delivered interventions than in-person-delivered interventions. But for the two outcomes with sufficient longitudinal studies to make a comparison (behavior, critical beliefs), the effect of online interventions appears to have had stronger effects over time than the in-person interventions; this was especially the case for behavior.

Social media

We compared the interventions that included content for addressing social media effects and those that did not. The ρ columns with the Social Media moderator in Tables 3–7, Supplementary Tables S1 and S2 present the results. The pattern across outcomes was somewhat inconsistent. For realism, efficacy, and attitudes (immediate and delayed measurement), studies including a social media component were less successful. But when critical beliefs was the outcome (immediate and delayed measurement), the effect of the intervention was larger. When behavior was the outcome, there was a very small difference such that the interventions with a social media component produced slightly larger effects with immediately measured outcomes but when behavior was measured at a later time point the social media interventions were slightly less successful.

Topic

Another potential categorical moderator was the topic. We compared studies focused on health topics to those focused on social topics. Examination of the ρ columns in Tables 3–7, Supplementary Tables S1 and S2 for the effects of the Topic moderator shows that when the outcome was knowledge, critical beliefs, attitudes, social norms, and behavior measured at immediate post-tests, social topics were associated with more positive effects than health topics. Yet, in the cases where there were also sufficient numbers of delayed measurements to assess this moderator, those differences tended to shrink (critical beliefs, attitudes).

Study location

We also attempted to assess whether the study was conducted in the U.S. or not makes a difference. Examination of the ρ columns in Tables 3–7, Supplementary Tables S1 and S2 for the effects of the Location moderator shows that for most outcomes, there were only small differences with some slightly favoring studies outside the U.S. and some in the U.S. Yet, in the case of knowledge and perceived realism, the studies outside the U.S. showed substantially more effective media literacy interventions than those inside of it. But there were only five and four non-U.S. samples (respectively) for knowledge and realism, creating wide credibility intervals around the effects for the non-U.S. samples that included the effect sizes from the U.S. samples (shown in the 80% CredI columns). Therefore, these results must be interpreted with caution.

Publication status

We were able to compare the studies published in academic journals to those that were not with a sufficient number of dissertations. This comparison provides a rough estimate of the extent to which publication bias is a problem in this area. Examination of the ρ columns in Tables 3–7, Supplementary Tables S1 and S2 for the effects of the Publication Status moderator shows that for some outcomes the differences were trivial in size (knowledge, realism, attitudes, and behavior) whereas for others there was a larger effect associated with the published studies relative to the unpublished (critical beliefs, norms, and efficacy). These results paint an inconsistent picture.

Tests of publication bias are less accurate in the presence of a large amount of heterogeneity (Peters et al., 2010). Yet, we tentatively examined several indicators. We chose critical beliefs, attitudes, and behavior as the outcomes for which we would assess publication bias because they had the largest set of studies. The trim and fill (Duval & Tweedie, 2000) analysis estimated that for critical beliefs, if all the unpublished studies had been included the population effect size estimate would be 0.05 higher, for attitudes it would be 0.04 lower, and for behavior it would be 0.08 higher. These were inconsistent and none would likely change the interpretation of the results. Egger’s funnel plot asymmetry tests (Egger et al., 1997) did not result in a statistically significant relationship for critical beliefs, attitudes, or behavior, suggesting an absence of publication bias.

Discussion

This study meta-analyzed media literacy intervention studies spanning 40 years to estimate their effects and to identify moderators. To our knowledge, this is the most comprehensive review of media literacy interventions to date. The sample size of this study (k = 160) is over three times that of Jeong et al. (k = 51). The accumulation of more than 100 studies since the initial meta-analysis necessitates reassessment of some of the key questions, as well as investigating new questions in light of the increasingly salient roles of digital and social media.

Methodologically, we aimed for a more comprehensive and fine-grained understanding of media literacy intervention effects by separating the outcomes rather than combining a disparate group of outcomes together (Carpenter, 2020). Combining outcomes together tends to reduce the accuracy of the meta-analysis. It also makes interpretation more difficult because it is unclear which outcomes are affected more or less by interventions. We also corrected for measurement error to improve estimates and provided both confidence intervals and credibility intervals to help interpret the findings.

Overall, the results indicate that media literacy interventions have produced positive effects ranging from media relevant and behavior relevant outcomes, demonstrating their utility in addressing unhealthy media effects on social and health behaviors. These outcomes included knowledge, critical beliefs, perceived realism, outcome expectancy, attitudes, social norms, efficacy beliefs, and behavior. Consistent with the findings from the initial meta-analysis by Jeong et al. (2012), the results of this study indicate that media literacy interventions remain an effective approach to helping users make informed decisions and actions.

Our new questions were of three broad types. They were designed to illuminate how the opportunities and challenges posed by new media impact media literacy intervention effects, to shed light on the longitudinal outcomes of media literacy interventions, and to understand the relationship between the more proximal outcomes of media-relevant variables and the more distal outcomes of behavior-relevant variables. In addition, reexaminations of questions about key features can generate a more robust understanding of the outcomes and moderators.

To address the first broad question, we investigated new moderators, including delivery mode and social media targeting. In-person rather than online delivery was more advantageous across most outcomes, suggesting potential pitfalls in utilizing digital media for intervention delivery. This pattern, however, was counterbalanced by two outcomes (critical beliefs, behavior) for which online-delivered interventions demonstrated stronger effects at delayed posttests than in-person studies. These findings suggest that there is room for improving online delivery. Perhaps strategic management of digital features could enhance online interactions. Walther (2011), for example, suggests that a selective rather than comprehensive use of digital cues could produce a better online interaction experience.

Interventions addressing social media showed weaker effects on realism, attitudes, and efficacy beliefs, but stronger effects on critical beliefs, compared to those that did not. Interventions could develop and test innovative approaches to address the distinct nature and functions of social media (Cho et al., 2024). A key characteristic of social media is the blurred boundary between media and user, with which users manage a dynamic relationship with content and platforms. This centrality of the self suggests that effective social media literacy strategies may need to differ from those designed for mass media literacy. In tandem, the conceptual bases of media literacy should embrace, while its empirical efforts should actively evaluate, computer mediated communication theories (e.g., Sundar et al., 2015) in improving the intervention’s ability to effectively counter the harmful effects of social media. Altogether, the findings about the roles of digital and social media underscore the challenges and opportunities presented for media literacy interventions in this dynamic new media environment.

Although only a subset of studies utilized longitudinal assessments, our analysis of the studies showed no substantial difference between outcomes assessed at immediate-post and follow-up assessments. These encouraging findings suggest that media literacy intervention effects may have stability over time. Given that media literacy is a lifelong skill for navigating one’s relationship with media, these results may hold important implications for media literacy education policies. The findings can especially be relevant in light of the growing number of states adopting media literacy education mandates (Media Literacy Now, 2022).

An exception to this pattern of findings for longitudinal effects was observed in measurements of behavior. Often considered the most significant outcome, behavior may be influenced by factors other than changes in cognitions and affect achieved through media literacy interventions. As IMBP suggests, the likelihood of behavior is moderated by environmental constraints and skills specific to the behavior (Fishbein & Yzer, 2003). These factors can be outside the scope of a media literacy intervention. In the future, media literacy interventions could be combined with other efforts to remove environmental constraints and provide necessary behavioral skills to improve their impact on behavior, while continuing to refine and bolster their immediate effects on critical beliefs and attitudes.

Another question related to long-term effects of media literacy interventions was whether the effects changed over the study years ranging from 1983 to 2023. Specifically, the relationship between study year and effect size was examined. No coherent pattern emerged from the results, suggesting that study year alone may not sufficiently account for changes in effects.

We also examined the differential effects of interventions on media-relevant and behavior-relevant outcomes, as well as the relationship between these two groups of variables, in an effort to understand the mechanisms underlying intervention effects. The results indicate that media literacy interventions had stronger effects on media-relevant outcomes (e.g., knowledge, perceived realism, critical beliefs) compared to behavior-relevant outcomes (e.g., norms, efficacy, behavior). Among media-relevant outcomes, knowledge showed medium-sized and undecayed effects, while perceived realism emerged as the strongest effect at delayed assessments. Notably, critical beliefs and attitudes demonstrated significant positive associations with behavior. These findings suggest the importance of these factors in media literacy interventions. Future interventions should incorporate these factors when relevant, and research could investigate how interventions can more effectively target these constructs.

Our moderator analyses provide an improved understanding of the conditions under which media literacy interventions are effective. While the issues addressed by the interventions ranged from health to social topics, the results indicate that the interventions targeting social issues performed better than those targeting health issues at immediate posttest for outcomes such as knowledge, critical beliefs, attitudes, and behavior. The finding that media literacy interventions effectively alter social issues-related attitudes and actions is promising. The relatively weaker effects observed for health issues may necessitate contextual consideration. Health issues, compared to social issues, may involve more environmental constraints or facilitators that moderate behavioral likelihood.

We reassessed the effects of three moderators—dose, source, and audience involvement—that were initially examined by Jeong et al. (2012). Regarding dose, or the number of sessions, no consistent improvement in effects was evidenced when the dose increased. Although it is often assumed that increased dose can lead to improved outcomes, our analysis indicates otherwise. This finding is consistent with Jeong et al.’s (2012), providing greater confidence. Given the reliability of these results, increasing dose may not be an effective way to improve efficacy of future media literacy interventions. These findings also offer practical insight for designing and delivering interventions at real world settings such as schools, where time and resources could be limited.

Effects of source, peer and expert presenters, were mixed in this meta-analysis. Peer-delivered interventions were found to be more effective for immediate efficacy and attitudes. However, researcher- or teacher-delivered interventions appeared more effective for other outcomes including knowledge, realism, and behavior. Future research could further examine whether effects of the presenter vary across different types of outcomes. Meanwhile, the findings from this study, along with results from Jeong et al. (2012), suggest that the presenter of the intervention may play a limited role as a moderator.

Audience involvement indicated no significant impact, consistent with the findings of Jeong et al. (2012). Revisiting this question was important because production is one of the core components of media literacy. With the proliferation of digital and social media, scholars have highlighted the participatory potential of media literacy for enhanced processes and outcomes. Furthermore, active involvement is generally considered to generate better outcomes. As interventions tend to use more than one of these audience involvement factors—discussion, activity, and production—future studies could use factorial designs to allow for a clearer understanding of these components’ effects (Collins, 2018).

Location was not a significant moderator. The growing volume of studies conducted outside the U.S. is encouraging, although still relatively small compared to those conducted within the U.S. The widespread adoption appears to demonstrate increasing needs for media literacy education worldwide. As more studies outside the U.S. accumulate, future research could investigate how different media systems and infrastructures may moderate the effects of media literacy interventions, moving beyond the current U.S. and non-U.S. comparison. Such investigations can generate valuable insight (see for useful frameworks Hallin & Mancini, 2017; Ostini & Fung, 2002).

One important caveat is that the effects across variables are heterogeneous, which complicates the interpretation of the findings. The wide credibility intervals for most of the estimates illustrate this challenge. We advocate for the development of more parsimonious frameworks to guide future theory, research, and intervention efforts in media literacy. Greater clarity will facilitate replication and extension, spurring theoretical and methodological advances and practical contributions of media literacy to society.

The heterogeneity, on the other hand, might reflect other strengths of this area and may make meta-analyses such as this study all the more important. The divergence may demonstrate the broad applicability of media literacy interventions to wide-ranging topics and behaviors and the resulting variability. As media literacy is a construct and practice that grew out of societal concern about harmful media effects, it may be only natural that a variety of conceptual frameworks and their components have been utilized. Different issues and behaviors may entail differential routes to behavior change. Such heterogeneity in extant media literacy research makes review and synthesis work such as the present study even more necessary.

Theoretical and practical implications

Unlike many other meta-analyses in our field, this study is not focused on a single theoretical framework. Instead, it examines media literacy interventions designed to counter harmful media effects and addresses broader and integrative questions emanating from existing conceptualizations and practices. Currently, there is no unifying theory of media literacy or its interventions. The findings from this study offer essential implications for building such a framework.

Overall, this study indicates that media literacy interventions are a viable and useful approach to addressing harmful media effects. In this meta-analysis, we strove to provide a deeper and more nuanced understanding of the effects of media literacy interventions and the conditions under which they can be more effective. These findings offer directions for future research and action.

While the heterogeneity in effects may reflect wide applications and applicability of media literacy interventions, future research would benefit from building on the findings to develop more parsimonious frameworks. Advances are also needed in the effective use of digital media for intervention delivery, as well as in designing content that specifically addresses social media effects which stem from their unique nature and ecology distinct from the mass media-based tradition of media literacy research.

Implications for designing interventions can be gleaned from the results. Content components that focus on cognitive and affective constructs such as critical beliefs and attitudes may be more important than structural features such as dose, source, and types of audience involvement as these latter factors showed no coherent impact. While the intervention effects on cognitive and affective outcomes appeared stable, sustaining the effects on behavior over time may require integrating efforts that consider environmental facilitators or inhibitors. For example, examining and adapting the social contexts within which users engage with media could help support and sustain behavior change. Given that media literacy involves making informed decisions in one’s lifelong engagement with the media, more interventions should utilize longitudinal designs to assess changes in effects over time.

By investigating theoretical moderators, digital and social media factors, and long-term effects, this study contributes to enhancing both the conceptual foundations and practical applications of media literacy interventions. Although a unified theory can not yet be formulated, the findings provide a necessary and important step toward developing more systematic frameworks for media literacy and media literacy interventions.

Limitations

This meta-analysis has limitations. We were unable to include all media literacy interventions, such as unpublished studies (e.g., conference presentations) and those that were not published in English. Studies without necessary statistical information could not be included either. There may be additional moderators that affect the effect sizes. Future meta-analyses of media literacy interventions should seek to address these limitations.

Final comments

This study advances and refines our understanding of media literacy interventions. It illustrates their effectiveness, while indicating the challenges the field faces at this critical juncture—40 years since the first media literacy intervention. Over the decades, media literacy research has substantially expanded, reflecting increasing relevance and attention. However, the findings suggest the need to strengthen the conceptual foundations of the interventions and to integrate new theories and approaches to more effectively address the distinctive nature and effects of digital and social media. It is hoped this work serves a turning point for the field.

Acknowledgments

The authors would like to thank the three anonymous reviewers, associate editor Jake Jensen, and editor Steve Wilson of HCR and Brad Bushman for their helpful comments on this manuscript. We also appreciate the authors of previous interventions who provided additional information about their studies.

Supplementary material

Supplementary material is available at Human Communication Research online.

Conflicts of interest

The authors have no conflicts of interest to disclose.

Funding

This study was supported in part by grant R01CA176196 from the National Institutes of Health.

References