https://orcid.org/0000-0003-2479-4063
Abstract
Values are consequential for opinion formation and remain a persuasive factor in shaping public attitudes. Still, the role of values remains under-researched in the context of online news production and engagement. This study investigates the intricate role values play by analyzing patterns of value references in online news coverage of coronavirus disease 2019 (COVID-19) in two culturally distinct nations, Romania and the U.K. Conceptually, the study is based on Schwartz’s value typology; methodologically, it relies on the Concept Mover’s Distance method. Analyzing half a million Facebook posts, the study identifies four types of value references: universal, cultural, topic-based, and situational. We show that cultural values prevalent in British posts, such as stimulation and tradition, resonate with the culturally congruent context. Universal values, however, do not guarantee a high level of engagement. The findings underscore the nuanced impact of values in shaping online news engagement.
Beyond framing societal issues, values fundamentally shape the formation of opinions (Nelson et al., 2015), as they remain a persuasive factor in swaying public opinion (Kaiser, 2020; Rokeach, 1968). Values are versatile due to their abstractness and the same value can be instrumentalized to fit opposing sides of the same issue, such as that freedom of expression can be used as an argument for allowing anti-immigration speech and for banning it (Nelson et al., 2015). Moreover, when evaluating a policy, values have been found to be more important to how individuals make decisions than policy-specific content (Lindgren, 2022).
Values, defined as guiding principles in individuals’ lives that transcend specific situations and can be ranked in order of importance (Schwartz, 1994), can be expressed at different levels on social media: they can be personal in the form of user-generated content, or cultural in the form of community norms and practices (Hallinan et al., 2022). While values are broad enough to be adapted to different social issues, they are employed differently in media discourses across cultural contexts, as different values are expected to resonate among different groups (Rodriguez et al., 2023). Values are especially critical in online environments, yet we know little about how different types of values predict audience engagement on social media.
This study builds on the body of scholarship on audience engagement, which aims to identify patterns of engagement with news on social media as well as the driving factors of engagement such as news values, emotions, and other characteristics of news content (Heidenreich et al., 2022; Salgado & Bobba, 2019). As such, the study contributes to recent systematic efforts to capture “cultures of engagement”, a term that refers to how engagement with social media content from news media outlets varies across national borders (Ferrer-Conill et al., 2023). We propose that among the multiple possible explanations, a cultural variation of references to values in social media content is a plausible reason why cultures of engagement vary along national lines. Additionally, we expect to observe cultural resonance since culturally specific values in news media posts would lead to an increased positive reaction from the audience in that cultural context (Gamson, 1992). To explore these relationships, the study aims to (1) map the use of value references in Facebook media posts across two culturally diverse contexts (i.e., Romania and the U.K.) and (2) identify which value references are associated with increased or decreased user engagement. By focusing on the role of different value references in news media posts in increasing and decreasing user engagement, this study aims to contribute to the empirical investigation as well as theoretical understanding of cultural resonance processes.
To achieve these research goals, we analyze Facebook posts from a diverse set of mainstream media outlets about the COVID-19 pandemic. The study follows these analytical steps: (1) identify value references by media outlets in their Facebook posts through leveraging a word embeddings approach already successfully employed in cultural analysis–Concept Mover’s Distance (Stoltz & Taylor, 2019); (2) examine differences in the prevalence of value references across the two culturally distinct countries’ Facebook posts; and (3) test how value references predict two aspects of online user engagement—reactions and comments.
The theoretical foundation for this study is organized along three main areas. First, we emphasize how values manifest through references in news media posts, and how values play a role in framing and cultural resonance processes. Second, we address the concept of cultures of engagement and how we expect that these references not only align with the available cultural repertoire, but also contribute to heightened audience engagement. Third, we anticipate news media in the two chosen countries to employ culturally resonant value references in their reporting on COVID-19 and highlight the unique international societal issue of COVID-19, underscoring its distinct unfolding in the two diverse national contexts.
Values in journalistic frames
Value references
Values constitute a foundational concept in understanding public opinion structure (Converse, 2006), encompassing individual and political values to explain personal choice, for example, in party preferences and candidate evaluations (Rokeach, 1968). Often studied from diverse disciplinary perspectives, values exist at personal, cultural, and infrastructural levels, reflecting individual guiding beliefs, collective norms, and desirable conduct on technological platforms, which often expand beyond national borders (Hallinan et al., 2022).
In this study, we utilize Schwartz’s personal values organized in a relational system, offering a well-established, concrete conceptualization of values at the individual level (Schwartz, 2012). These values have been found to be common across countries yet vary in their hierarchical importance to individuals due to economic, social and cultural influences (Miles & Yeh, 2022). Personal values are defined as abstract beliefs about desirable goals that transcend specific situations, serve as guiding principles in individuals’ lives, and can be ranked in order of importance (Schwartz, 1994). Importantly, a selection of values serves the self-interests of both the individual and the ingroup, such as stimulation, hedonism (enjoyment), and self-direction. Other values in Schwartz’s system serve the interests of others, both within the ingroup and in broader societal contexts, such as security, tradition, and conformity (Schwartz, 1990). The presence of values that promote either self-interest or the interest of others in news highlights what is deemed a desirable goal in relation to specific societal issues, within particular cultural contexts.
On social media platforms, values are communicated at multiple levels: first, at the personal level, user-generated content and reactions to social media content indicate users’ own values; second, at the cultural level, values are made visible through community norms and practices (Hallinan et al., 2022). At both the personal and cultural levels, values can be inferred from user content, through expressed positions on specific issues and specific norms common within a community. This process of inferring values from expression or behavior requires a reliance on value references. Therefore, in this study, we conceptualize values embedded in media-generated posts as value references.
Values have been referenced in political and media content in various ways. In the case of news coverage about the Abu Ghraib prison scandal, for instance, cultural values and a shared national identity were employed to construct a culturally resonant message (Rowling et al., 2011). Furthermore, national cultures shape how people use and perceive different aspects of social media, for instance how users interpret emotions or how values are referenced in New Year’s resolutions across different languages (Hallinan et al., 2023; Park et al., 2014). Rodriguez et al. (2023) examined the visual framing of government-led COVID-19 vaccination campaigns online in three culturally diverse contexts: Brazil, Indonesia, and the U.S.A. The official campaigns not only used distinct symbols and meanings in each cultural setting, but also aligned with each country’s cultural characteristics, reflecting unique ideological positions. Brazilian visuals emphasized national pride in vaccine acquisition and distribution, while Indonesian campaigns highlighted the concept of gotong royong, emphasizing collectivism and respect for elders. In both traditional news outlets and social media, values are invoked to support the issue at hand in order to make the message more persuasive and connect with the target audience. However, the construction of collectivity varies across different cultural contexts, with distinct sets of values employed for this purpose.
Framing and cultural resonance
In the context of journalistic framing, “frames are organizing principles that are socially shared and persistent over time, [and] that work symbolically to meaningfully structure the social world” (Reese, 2008, p. 11). While news framing has been studied for decades, recent approaches in the framing paradigm seek to separate the informational content related to the issue being reported on from the tools through which the frame is constructed (i.e., framing devices) (Kaiser, 2020). The shift is then toward conceptualizing frames as providing context to issues that journalists cover (Baden, 2018). Values, among other types of framing devices (Pan & Kosicki, 1993), and references to broader cultural discourse constitute such tools to contextualize an issue and serve as an organizing principle. In this study, we adhere to an approach to framing that posits a frame as comprising both informational content related to an issue and, crucially, a framing device—in this case, a value reference—that functions to contextualize the issue under discussion (Baden, 2018; Pan & Kosicki, 1993; Reese, 2008).
Furthermore, cultural resonance is central to explaining how values work to frame an issue for the public, and the dynamics of public discourse more broadly (Entman, 2003). Cultural resonance explains how and why ideas are successfully communicated, when they are based on cultural themes or widely shared knowledge between people inhabiting a cultural space (Gamson, 1992). The main idea behind the concept of cultural resonance refers to the congruence between a value reference in a media message and the individuals’ preference for that value (Shen & Edwards, 2005) or to the fit between social identity and references to one’s ingroup in media messages (Sheets et al., 2022). However, there is an additional congruence to consider when it comes to cultural resonance. Cultural resonance has also been conceptualized as the alignment between a media frame and the cultural themes it draws upon, subsequently influencing frame success or mobilization (Entman, 2003; Geiß et al., 2017; Snow & Benford, 1988). Cultural resonance in media frames involves two key points of congruence. First, the frame taps into cultural themes, establishing alignment at a cultural level. Second, the media frame, enriched with cultural references, aligns with the audience’s values and interpretation schemas.
Research on cultural resonance provides mixed findings: cultural resonance was essential in the economic policy framing of the Lehman Brothers’ bankruptcy (Geiß et al., 2017), but not crucial during the Danish Muhammad cartoon controversy (Tromble & Meffert, 2016). Our investigation aims to understand both points of congruence in cultural resonance, avoiding assumptions about it occurring at the cultural level within media frames. We explore how news media integrate value references emphasizing both self-interest and the interest of others, analyzing their impact on user reactions. This nuanced approach seeks to uncover the complexities of cultural resonance in news media discourse.
Cross-nationally variable audience engagement with news
Engagement on social media is often equated with terms such as interactivity and participation and is often captured through a proxy of visible markers such as likes, comments and shares (Hallinan et al., 2022). The field’s over-reliance on reductionist markers of engagement on social media has been criticized for capturing too little of what users experience online (Steensen et al., 2020). Yet, metrics have the ability to point to the content with which audience choose to engage (Ferrer-Conill et al., 2023), so they reveal important aspects related to user attention and levels of interaction with news media content on social media. In an effort to systematize our understanding of national differences in user engagement with news media content online, the term cultures of engagement captures “explicit patterns of engagement across different national boundaries [that] were based on a broader set of characteristics, such as practices, norms, and values” (Ferrer-Conill et al., 2023, p. 113).
Going beyond the journalistic cultures of production and the ownership of the news outlets, we propose that cultures of engagement with news media content are bolstered by cultural resonance. Both journalists producing media content and audiences consuming that content rely on the same values available in their cultural repertoire to make sense of a social issue (see van Gorp, 2010). The two contexts were selected following a diverse case selection strategy, which has proven useful when aiming to explore the breadth of a social phenomenon (Seawright & Gerring, 2008). Therefore, we expect to observe not only different cross-national patterns of value references but also that those value references will generate diversified user attention and interaction with the media content specific for the culture of engagement of each country.
In Romania, the most prevalent news users are the minimalists—a large group of individuals that is likely to be stuck in echo chambers, but less prone to being accidentally exposed to news stories. This group is characterized by low levels of trust in both mainstream and social media sources (Buturoiu, Corbu, & Boīan, 2022). Yet, Romanian citizens that engage more on with social media news, tend to trust that news more, but also news from legacy media, and information from family and friends (Buturoiu, Corbu, Oprea, et al., 2022). The lack of trust in news of Romanian citizens is coupled with a media system that is characterized by a high level of political influence in journalism and low levels of press freedom (Herrero et al., 2017). Thus, how Romanian audiences will engage with values emphasizing self-interest or the interest of others will be shaped by societal and journalistic factors. We expect that users’ low trust in Romanian news sources may render values appealing to the interest of others less appealing than self-interest ones.
Comparatively, British audiences trust media sources to a greater extent—51.9% among legacy media users and 40.8% among digital-born users (Vara-Miguel, 2020). Furthermore, individuals who use online media sources for news have a more diverse news repertoire (Fletcher et al., 2023). The media system in the U.K. is characterized by a strong public broadcasting system that has a significant impact on citizens’ political knowledge (Brüggemann et al., 2014). Given the generally more positive relationship between news media sources and British audiences, we do not expect British audiences to show the same reluctance towards values that emphasize the interest of others.
COVID-19 as a global issue
From a comparative perspective, the COVID-19 pandemic presents a unique opportunity to study the same social phenomena in different national contexts, as different countries developed local strategies for implementing lockdowns, masking or vaccination campaigns when dealing with the global pandemic. While the strategy in choosing the country contexts was to increase the cultural and societal case variety, the choice in topic was intended to capture the same debates during the unsettling time of an unprecedented international crisis, a time that was expected to produce a range of social and political discourse. Such unsettled times exhibit an increased reliance on cultural values in media communication because of the need for heightened cultural reasoning in times of crisis (Tenenboim-Weinblatt & Baden, 2018).
COVID-19 unfolded differently in the two countries. In the U.K., the British Medical Association assessed the government response to the pandemic as inadequate (British Medical Association, 2023). Lacking a clear policy approach from the start and the underfunding of the public health care system led to a long first wave of COVID-19 infections between February and September 2020. The confusion about COVID-19 restrictions continued around Christmas 2020, and even with the vaccine rollout, January 2021 was the month with the second highest average deaths due to COVID-19 in the U.K., with 675.4 deaths per 100,000 people in England (Cornish, 2021).
Conversely, the first wave of infections was much shorter in Romania (February–June 2020), and the pandemic was met with a swift and strict lockdown that forbade citizens to be outside except for professional activities as essential workers or for walking their pets (Vulcan, 2020). After the initial restrictive measures, however, the pandemic became politicized in Romania: the newly founded, right-wing party AUR used disinformation and COVID-19 related conspiracy theories as its platform and received 9% of the vote in the parliamentary elections that year, while the country further faced vaccine skepticism and the proliferation of disinformation (Muntele et al., 2022). Comparing the two countries, COVID-19 took a higher human toll in the U.K. with an excess mortality of 99.3% in 2020, compared to 62.9% in Romania—which places Romania among the middle in a ranking of European countries (Coombs & Ward, 2021).
Although COVID-19 led to similar challenges in both Romania and the U.K., how policies were received, and the context of each country shows that they faced unique challenges in dealing with the pandemic. Therefore, this is an opportune moment to study what role values played in the media coverage of the pandemic and whether and how differences map across the two cultural domains.
Given the unique context of COVID-19, we opt for an exploratory research design based on open research questions, not on a priori formulated hypotheses. Specifically, we focus on two research questions: How do value references in news media Facebook posts differ between Romania and the United Kingdom, for all news content and for Covid-19 posts? (RQ1) With this question, we compare the prevalence of both self-interest and in the interest of others value references within the two culturally distinct contexts. The objective is to compare the frequency of these values, taking into account COVID-19 as the main topic within this time period. The second research question (RQ2) asks: How do value references predict user engagement? Our goal is to understand which types of values are more likely to prompt reactions or comments from users on Facebook.
Methods
Data collection and preprocessing
In order to capture traditional media content in each country, we selected prominent news outlets from the following categories: (1) broadcast channels (public and commercial broadcasting); (2) newspapers (broadsheets and tabloids); and (3) news websites (see Supplementary Appendix A1, Tables A1.3 and A1.4 for a complete list of media outlets). The posts from these Romanian and British news media outlets were systematically collected from their official Facebook accounts using Facepager (Jünger & Keyling, 2019) between May 2020 and April 2021, resulting in a total of 535,676 posts. This time frame covered different phases of the COVID-19 crisis and was expected to be associated with a higher reliance on value references in news media content and a heightened response rate from users. Mainstream news brands still attract most attention around news through Facebook and Twitter (Newman et al., 2021), but the increased length of Facebook posts provided more suitable material for identifying value references. We removed duplicates and posts lacking textual content, both in the post and link text (we included the text of links, as they often contained clues about the topic of the post). In the U.K., we retained 226,041 posts, and in Romania, 309,635 posts were kept for this time period. By utilizing a bilingual dictionary, we identified posts related to COVID-19, accounting for 27.0% of the Romanian dataset and 24.0% in the U.K. dataset.
T-tests for mean differences in value reference scores across countries and COVID-19 posts
| COVID-19 differences | |||||||
|---|---|---|---|---|---|---|---|
| COVID-19 | Non-COVID-19 | t(535674) | p | Cohen’s d | |||
| M | SD | M | SD | ||||
| Stimulation | 0.30 | 0.52 | 0.39 | 0.64 | −54.79 | <.001 | −0.16 |
| Hedonism | 0.19 | 0.50 | 0.42 | 0.74 | −129.13 | <.001 | −0.33 |
| Self-direction | 0.40 | 0.55 | 0.36 | 0.59 | 19.93 | <.001 | 0.06 |
| Tradition | 0.30 | 0.51 | 0.38 | 0.64 | −48.19 | <.001 | −0.13 |
| Security | 0.46 | 0.56 | 0.35 | 0.55 | 63.99 | <.001 | 0.20 |
| Conformity | 0.42 | 0.63 | 0.34 | 0.63 | 39.36 | <.001 | 0.12 |
| COVID-19 differences | |||||||
|---|---|---|---|---|---|---|---|
| COVID-19 | Non-COVID-19 | t(535674) | p | Cohen’s d | |||
| M | SD | M | SD | ||||
| Stimulation | 0.30 | 0.52 | 0.39 | 0.64 | −54.79 | <.001 | −0.16 |
| Hedonism | 0.19 | 0.50 | 0.42 | 0.74 | −129.13 | <.001 | −0.33 |
| Self-direction | 0.40 | 0.55 | 0.36 | 0.59 | 19.93 | <.001 | 0.06 |
| Tradition | 0.30 | 0.51 | 0.38 | 0.64 | −48.19 | <.001 | −0.13 |
| Security | 0.46 | 0.56 | 0.35 | 0.55 | 63.99 | <.001 | 0.20 |
| Conformity | 0.42 | 0.63 | 0.34 | 0.63 | 39.36 | <.001 | 0.12 |
| Country differences | |||||||
|---|---|---|---|---|---|---|---|
| U.K. | Romania | t(535674) | p | Cohen’s d | |||
| M | SD | M | SD | ||||
| Stimulation | 0.48 | 0.75 | 0.28 | 0.47 | 112.73 | <.001 | 0.33 |
| Hedonism | 0.51 | 0.82 | 0.25 | 0.56 | 132.39 | <.001 | 0.39 |
| Self-direction | 0.31 | 0.58 | 0.42 | 0.57 | −68.14 | <.001 | −0.19 |
| Tradition | 0.44 | 0.75 | 0.30 | 0.48 | 81.28 | <.001 | 0.24 |
| Security | 0.34 | 0.58 | 0.40 | 0.54 | −38.39 | <.001 | −0.11 |
| Conformity | 0.34 | 0.68 | 0.38 | 0.58 | −24.45 | <.001 | −0.07 |
| Country differences | |||||||
|---|---|---|---|---|---|---|---|
| U.K. | Romania | t(535674) | p | Cohen’s d | |||
| M | SD | M | SD | ||||
| Stimulation | 0.48 | 0.75 | 0.28 | 0.47 | 112.73 | <.001 | 0.33 |
| Hedonism | 0.51 | 0.82 | 0.25 | 0.56 | 132.39 | <.001 | 0.39 |
| Self-direction | 0.31 | 0.58 | 0.42 | 0.57 | −68.14 | <.001 | −0.19 |
| Tradition | 0.44 | 0.75 | 0.30 | 0.48 | 81.28 | <.001 | 0.24 |
| Security | 0.34 | 0.58 | 0.40 | 0.54 | −38.39 | <.001 | −0.11 |
| Conformity | 0.34 | 0.68 | 0.38 | 0.58 | −24.45 | <.001 | −0.07 |
Note. N = 535,676 for all t-tests reported in this table. Since all tests are statistically significant due to the large sample, we consider mean differences in value references to be meaningful across the two countries and for COVID-19 and non-COVID-19 posts when Cohen’s d is equal to or greater than 0.20, which is the threshold for a small effect.
T-tests for mean differences in value reference scores across countries and COVID-19 posts
| COVID-19 differences | |||||||
|---|---|---|---|---|---|---|---|
| COVID-19 | Non-COVID-19 | t(535674) | p | Cohen’s d | |||
| M | SD | M | SD | ||||
| Stimulation | 0.30 | 0.52 | 0.39 | 0.64 | −54.79 | <.001 | −0.16 |
| Hedonism | 0.19 | 0.50 | 0.42 | 0.74 | −129.13 | <.001 | −0.33 |
| Self-direction | 0.40 | 0.55 | 0.36 | 0.59 | 19.93 | <.001 | 0.06 |
| Tradition | 0.30 | 0.51 | 0.38 | 0.64 | −48.19 | <.001 | −0.13 |
| Security | 0.46 | 0.56 | 0.35 | 0.55 | 63.99 | <.001 | 0.20 |
| Conformity | 0.42 | 0.63 | 0.34 | 0.63 | 39.36 | <.001 | 0.12 |
| COVID-19 differences | |||||||
|---|---|---|---|---|---|---|---|
| COVID-19 | Non-COVID-19 | t(535674) | p | Cohen’s d | |||
| M | SD | M | SD | ||||
| Stimulation | 0.30 | 0.52 | 0.39 | 0.64 | −54.79 | <.001 | −0.16 |
| Hedonism | 0.19 | 0.50 | 0.42 | 0.74 | −129.13 | <.001 | −0.33 |
| Self-direction | 0.40 | 0.55 | 0.36 | 0.59 | 19.93 | <.001 | 0.06 |
| Tradition | 0.30 | 0.51 | 0.38 | 0.64 | −48.19 | <.001 | −0.13 |
| Security | 0.46 | 0.56 | 0.35 | 0.55 | 63.99 | <.001 | 0.20 |
| Conformity | 0.42 | 0.63 | 0.34 | 0.63 | 39.36 | <.001 | 0.12 |
| Country differences | |||||||
|---|---|---|---|---|---|---|---|
| U.K. | Romania | t(535674) | p | Cohen’s d | |||
| M | SD | M | SD | ||||
| Stimulation | 0.48 | 0.75 | 0.28 | 0.47 | 112.73 | <.001 | 0.33 |
| Hedonism | 0.51 | 0.82 | 0.25 | 0.56 | 132.39 | <.001 | 0.39 |
| Self-direction | 0.31 | 0.58 | 0.42 | 0.57 | −68.14 | <.001 | −0.19 |
| Tradition | 0.44 | 0.75 | 0.30 | 0.48 | 81.28 | <.001 | 0.24 |
| Security | 0.34 | 0.58 | 0.40 | 0.54 | −38.39 | <.001 | −0.11 |
| Conformity | 0.34 | 0.68 | 0.38 | 0.58 | −24.45 | <.001 | −0.07 |
| Country differences | |||||||
|---|---|---|---|---|---|---|---|
| U.K. | Romania | t(535674) | p | Cohen’s d | |||
| M | SD | M | SD | ||||
| Stimulation | 0.48 | 0.75 | 0.28 | 0.47 | 112.73 | <.001 | 0.33 |
| Hedonism | 0.51 | 0.82 | 0.25 | 0.56 | 132.39 | <.001 | 0.39 |
| Self-direction | 0.31 | 0.58 | 0.42 | 0.57 | −68.14 | <.001 | −0.19 |
| Tradition | 0.44 | 0.75 | 0.30 | 0.48 | 81.28 | <.001 | 0.24 |
| Security | 0.34 | 0.58 | 0.40 | 0.54 | −38.39 | <.001 | −0.11 |
| Conformity | 0.34 | 0.68 | 0.38 | 0.58 | −24.45 | <.001 | −0.07 |
Note. N = 535,676 for all t-tests reported in this table. Since all tests are statistically significant due to the large sample, we consider mean differences in value references to be meaningful across the two countries and for COVID-19 and non-COVID-19 posts when Cohen’s d is equal to or greater than 0.20, which is the threshold for a small effect.
Measures
Dependent variables
User engagement was the dependent variable of interest. Following prior research, this variable was measured by capturing several forms of interaction with Facebook posts (Salgado & Bobba, 2019). First, we recorded user reactions, a metrics requiring a lower level of user engagement (Dimitrova et al., 2024). Next, we captured user comments, another metric of engagement demanding more effort from users. Embracing a reductionist approach to capturing user engagement, these metrics offer a good proxy of social media interactions (Ferrer-Conill et al., 2023).
Our focus was on reactions and comments, as reactions provide a gauge for comparing the attention posts receive, while comments indicate the level of discussion a post generates. We treated these two metrics separately, resulting in two primary dependent variables: (1) reactions count, encompassing the sum of all emoji-like reactions users can attribute to a post, such as “like,” “love,” “haha,” “wow,” “sad,” and “angry”; and (2) comment count, referring to the sum of all first-level comments constituting a direct response to the Facebook post (Heidenreich et al., 2022). For the models including interaction terms, we divided the reaction and comment count by number of followers to improve visualization (see Supplementary Appendix A3 for details).
Independent variables
Value references
The study employs the Concept Mover’s Distance (CMD) method to identify value references in Facebook posts. CMD, based on word embeddings, gauges a text’s engagement with a specific concept by measuring the distance between the words in the text and an ideal pseudo-text composed only of words denoting the concept of interest (Stoltz & Taylor, 2019). We validated the CMD scores in three steps (Supplementary Appendix A4, Table A4.1), achieving F1 scores ranging from 0.63 to 0.86 for all six value references. Supplementary Appendix Section A5 contains the value references validation codebook with complete coder instructions and examples for each value. CMD proves useful as it aligns with sensitizing concepts chosen based on theory (Blumer, 1954), focusing on the specific values of interest. Values, being broad concepts not explicitly expressed in texts, align well with CMD’s reliance on pre-trained language models. Machine translation of Romanian posts yielded satisfactory results in 95% of cases.
Value reference scores are calculated by adding and subtracting relevant vectors for each anchor, with a positive score indicating proximity to the desired pole of the value and a negative score moving towards subtracted values, following Stoltz and Taylor’s approach (2019). The continuous scores are comparable and standardized. In further data processing, posts with scores below 0 were labeled as having “no reference.” The highest value reference score was categorized with a “primary” and, if applicable, all others above 0 were “secondary” value references. This classification allows for an assessment of the ranking of value references in posts. All posts in our dataset contained at least one value reference or more, such that 1.6% contained one value reference, 23.8% contained two, 50.2% with three, 22.9% with four, 1.5% with five, and 0.004% with six value references.
Post topic
We determined whether a post is about COVID-19 or not using a dictionary approach, because it offers a reliable method to identify specific concepts that are a priori defined. We selected common keywords that indicated the topic of COVID-19 from a small sample of posts, then refined the dictionary with iterative validation rounds. Comparing the annotations yielded from the dictionaries with the validation set, for English an F1 = 0.92, and for Romanian F1 = 0.91 (see Supplementary Appendix A4).
Control variables
In each country, we selected the media outlets based on country profiles from the Digital News Report and supplemented with online searches for online media outlets with the greatest market share (Newman et al., 2023). The type of media outlet (i.e., broadcast channel, newspaper, online website), and type of post (i.e., link, note, video or photo) were used as controls.
Results
The results section is structured in two parts. First, we describe differences in the prevalence of value references in news media posts across countries and for COVID-19 and non-COVID-19 posts. Based on the observed meaningful differences, we differentiate between different kinds of value references in Facebook media posts. In the second part of the results section, we turn to the relationship between value references and user engagement, investigating which value references predict engagement, and whether their effect is contingent upon the cultural context.
Value references in news media posts
Utilizing a two-prong approach to assess the presence of value references in news media posts, we first calculate a standardized score that indicates the degree of presence of each value within a post for each of the six value references (stimulation, hedonism, self-direction, tradition, security, conformity) based on the CMD method, and then identify the primary and secondary value references in the post. This approach takes into account that, in practice, any media post will have one value as the primary focus, even if more than one value is present. Identifying primary and secondary value references simplifies visualizing value references at the post level and considers the different values in relation to each other.
To answer the first research question, how do value references in news media Facebook posts differ between Romania and the United Kingdom, for all news content and for Covid-19 posts, we show differences across countries and for COVID-19 posts in the prevalence of value references, and test whether the differences are meaningful.
Overall, the number of social media posts for news media outlets does not differ substantially between the two countries: British media outlets post on average 1,108.04 during the 12-month period, while Romanian ones post 1,290.16 times, which amounts to 16.4% more. In terms of the primary value references present in the Romanian and British media posts, two out of six value references show country differences in the direction expected by the theoretical categorization of values (i.e., self-direction and tradition—see Supplementary Appendix A1, Figure A1.1). Hedonism (21% of all posts) and stimulation (20%) are the most prevalent primary value reference in British media posts. Conversely, self-direction (20%) and security (19%) are most prevalent in Romanian news media posts. Looking at the pattern across the two countries, the British news media makes more use of stimulation, hedonism and tradition than Romanian media on Facebook, while self-direction, security, and conformity posts are more prevalent in the Romanian context. The second dimension we examine to determine differences in the prevalence of primary value references in news media posts is the main global concern during this period: COVID-19. As the biggest issue on the global agenda at the time, it could activate value references in media coverage, regardless of the national context. For COVID-19 posts, security (23%), conformity (19%), and self-direction (18%) are the most prevalent value references.
Main effects for mixed-effects models predicting user engagement.
Note. Interpretation of Incidence Rate Ratio (IRR) = percentage change in expected counts.
Table 1 displays effect sizes of the mean differences for value reference scores across countries and COVID-19 posts. Security displays a meaningful positive effect, which indicates that it is a value used more often in COVID-19 posts, while hedonism appears as a non-COVID-19 value. Stimulation, self-direction, tradition, and conformity are not meaningfully associated with either COVID or non-COVID posts. Further, in terms of country differences, stimulation, hedonism and tradition are more prevalent in the U.K., while self-direction, security and conformity do not display meaningful country differences.
Considering the meaningful differences in value reference scores both across the two countries and between COVID and non-COVID posts, four distinct kinds of value references emerge. First, values can play a universal role in media posts, indicating that their use remains consistent across countries and topics. Our results show that there are no meaningful differences in the extent to which media posts make use of references to self-direction and conformity as a value across countries, nor in posts related to COVID-19 compared to non-COVID-19 posts. In other words, self-direction and conformity are embedded in coverage on a variety of topics across different cultural contexts. Second, values can play a cultural congruency role in media posts, indicating that they are more prevalent in some countries than others, irrespective of the topic reported (i.e., stimulation and tradition). For example, references to stimulation were more common in the U.K. (M = 0.48, SD = 0.75) compared to Romania (M = 0.28, SD = 0.47), with no meaningful difference between COVID-19 and non-COVID-19 posts. Third, values can assume a topic-based role in media posts, meaning that their use differs by the issue covered in the news post, but not by country (i.e., security). For instance, references to security are more prevalent in COVID-19 posts (M = 0.46, SD = 0.56) compared to non-COVID-19 posts (M = 0.35, SD = 0.55), but their presence does not meaningfully differ across the two national contexts. Fourth, values can assume a situational role in media posts, meaning that their use differs both by the issue covered in the news post and by country (i.e., hedonism). References to hedonism are more prevalent in the U.K. (M = 0.51, SD = 0.82) compared to Romania (M = 0.25, SD = 0.56). Additionally, references to hedonism are also less prevalent in COVID-19 posts (M = 0.19, SD = 0.50) compared to non-COVID-19 posts (M = 0.42, SD = 0.74).
Value references as predictors of user engagement
Before addressing the second research question, we examine differences in user engagement with news media posts across the two countries. The results show a striking contrast in the average engagement levels with media posts between the U.K. and Romania. In general, users engage up to 26 times more with British media posts compared to Romanian ones. Specifically, media posts in the U.K. garner an average of 1,196 reactions, whereas Romanian posts receive only 88 reactions. Furthermore, posts in the U.K. receive an average of 172 comments, while their Romanian counterparts receive only 18 comments.
A series of hierarchical negative binomial regressions with random intercepts were conducted to capture the association between post value references and user reactions and comments, controlling for post-level and actor-level variables (see Supplementary Appendix A2, Table A2.1 for regression estimates). The chosen statistical approach is beneficial as it accounts for clustering the data at the news media outlet level and is an appropriate approach for over-dispersed dependent variables (i.e., reaction and comment counts).
To answer the second research question, how do value references predict user engagement, Figure 1 summarizes Models 2 and 4 displaying the main effects of value references. Model 2 estimates reaction counts based on value references, as well as type of media outlet, COVID-19 as post issue, country of media outlet, and number of followers of Facebook account. A conditional pseudo R2 of 0.74 indicates a good model fit. The results indicate that three out of six value references are associated with an increase in reactions to a media Facebook post, although there is no observable clear pattern according to value roles. Among the universal values, self-direction is associated with a 270% increase in reactions when it is present in a post (β = 3.70, p < .001), while conformity reduces reactions by 74% (β = 0.26, p < .001). For cultural values, tradition increases reactions by 259% when present in a post (β = 3.59, p < .001), while stimulation does not show a statistically significant effect. The topical value of security is associated with a reduction in reactions of 91% when present in a post (β = 0.09, p < .001), and hedonism—as situational value, shows the highest increase in reactions of 401% (β = 5.01, p < .001).
Model 4 predicts comment counts from post value references, type of media outlet, COVID-19 as post issue, country of media outlet, and number of followers of Facebook account. A conditional pseudo R2 of 0.71 indicates a good model fit. Five out of six value references elicit a statistically significant change in the number of comments a news media post receives, except for references to hedonism. For universal values, self-direction is associated with an 301% increase in comments when it is present in a post (β = 4.01, p < .001), while conformity reduces comments by 37% (β = 0.63, p < .001). For cultural values, tradition increases comments by 96% when present in a post (β = 1.96, p < .001), while stimulation decreases the number of comments by 82% (β = 0.18, p < .001). The topical value of security is associated with a reduction in comments of 72% when present in a post (β = 0.28, p < .001).
When comparing the impact of value references across the two models, several general trends emerge. First, posts that reference universal values display mixed patterns: posts containing self-direction receive amongst the highest rates of engagement both in terms of reactions and comments, yet posts with conformity show a consistent pattern of decreased engagement. The presence of universal values does not guarantee a high rate of engagement. Second, posts that reference cultural values display mixed patterns as well: posts with tradition receive both increased reactions and comments, while the presence of stimulation has no effect on reactions and decreases the number of comments a post receives. Again, the cultural preference for these values of British news media does not guarantee their resonance with Facebook audiences. Third, the situational value of hedonism is associated with a stark increase in reactions but has no effect on comments, implying that user involvement with posts containing this value is relatively low effort (Salgado & Bobba, 2019). Fourth, the topical, COVID-19 related value of security is associated with a decrease in both reactions and comments, indicating a lower level of engagement.
In a final step to assess whether cultural resonance serves as a mechanism explaining the impact of value references on user engagement, we conducted a series of tests to examine the differential effects of universal (i.e., self-direction, conformity) and cultural values (i.e., stimulation and tradition) across the two cultural contexts (Supplementary Appendix A3, Table A3.1 for models including interactions). Out of all the interaction effects tested for both reactions and comments, the interaction effect between references to self-direction and country on post reactions is not statistically significant (β = 0.92, p = 0.39) (see Figure 2). Then, among the universal values, the effect of self-direction on comments is stronger for Romania than for the U.K., thus leading to a higher rate of increase in comments in Romania, indicating a stronger resonance with Romanian audiences. For conformity, the effect on reactions and comments is negative in Romania and positive in the U.K., which indicates that this value resonates in the British context and does not in the Romanian one. Among the cultural values, stimulation has a strong negative effect on reactions and comments for Romania, but the effect is weaker or not present for the U.K. Tradition has a stronger positive effect on reactions and comments for the U.K., but the effect is weaker for Romania. Taken together, the two cultural values that are more prevalent in the U.K. resonate more with their audiences.
Interaction effects for universal and cultural values on reactions and comments by country.
Discussion
Contributing to research on online engagement with news, our goal was to identify what values elicit audience engagement by focusing on news media Facebook posts during COVID-19 in two national contexts—Romania and the U.K. Starting from the premise that these countries have distinct cultures of engagement, we expected that Romanian users will engage less with values that emphasize the interest of others than British ones. Overall, news media outlets in Romania do not post substantially more than those in the U.K. However, British posts receive, on average, 26 times more engagement than Romanian ones. This striking difference in engagement points to markedly different cultures of engagement in the two countries. Our findings are in line with evidence on cross-national news sharing patterns, as de León and Vermeer (2023) show that while the news production of political articles on Facebook does not drastically change during election times, the sharing of political articles during elections in Romania increases compared to the U.K.
Focusing on the analysis of value references, three values displayed a meaningful country difference, where stimulation, hedonism, and tradition were more prevalent in British posts than in Romanian ones. Considering that the remaining three values showed no meaningful difference between the two countries, we conclude that British news outlets make use of value references to a greater extent than Romanian news media. In addition, we identified security as a value used significantly more in COVID-19 posts, and hedonism as a non-COVID-19 value.
The multivariate results allowed us to identify four types of value references in news media Facebook posts, as follows: universal values, such as self-direction and conformity, show consistent use across countries and topics; cultural values, such as stimulation and tradition, were more common in one cultural context than the other, regardless of the topic; topic-based values, such as security, vary by topic but not by country; and situational values, such as hedonism, vary both by country and topic. Analyzing how news media outlets utilize diverse patterns of universal, cultural, topic-based, and situational values within different media and political systems, offers valuable insights for discussions of news framing processes (Vreese, 2005). First, from an available repertoire of general values, some values will have a universal appeal—such as conformity and self-direction, lending themselves to framing news stories across topics and cultures. Conversely, situational values lend themselves only to specific uses—such as hedonism, that was more prevalent in non-COVID-19 posts in the U.K. Second, when values display cultural specificity—as we identified for stimulation and tradition in British news media posts—they may indicate value-attitude associations in news stories that reflect cultural particularity. Thus, tracking changes in these culturally variable values and their associated attitudes would enable us to map broader transformations in societal value systems (Rokeach, 1968).
Then, we tested how value references impact user engagement and whether that impact differs across the two national contexts. The analysis focusing on the main effects of value references showed that value references in posts do not guarantee a high rate of engagement. One universal value—self-direction generated amongst the highest rates of engagement, while the other—conformity was associated with decreased engagement. Cultural values display a similar pattern: tradition garners high engagement, while stimulation does not. These differences cannot be explained through the self-interest versus interest of others distinction either, as the high engagement values belong to both categories. Perhaps an explanation resides in the topics or attitudes these values were associated with, and how these associations contributed to generating engagement. Values can be co-opted for different political issues (Nelson et al., 2015), so further investigation of value-issue associations is necessary.
To explore cultural resonance as an explanation for the impact of value references on user engagement (Gamson, 1992; Sheets et al., 2022), we compared the effects of universal and cultural values across British and Romanian contexts. Focusing on the effects of universal values, conformity resonates in the British context and does not in the Romanian one, while self-direction indicates a stronger resonance with Romanian audiences than with British ones. As conformity—a value that emphasizes the interest of others resonates less in Romania, and self-direction—a value that emphasizes self-interest resonates more in the same context, we find support for the claim of cultural resonance initially proposed.
Furthermore, the cultural resonance explanation holds equally well regarding cultural values. Both stimulation and tradition resonate less in Romania compared to the U.K., which suggests that the two values fulfill their role as British cultural values successfully. Our findings demonstrate that cultural resonance processes are more complex than initially theorized: values emphasizing self-interest and in the interest of others do not neatly align along national lines. Although our findings show that cultural values resonate more in the congruent cultural context, universal values do not guarantee a higher level of engagement in all cultures. In line with extant research, we suggest that the cultural resonance concept is in need of specification and further investigation, as a multitude of factors render a textual reference culturally resonant (McDonnell et al., 2017).
Theoretical implications
This study has important implications for three distinct research areas. First, there are clear theoretical implications for journalism studies. Looking at the input side of what shapes media content, in addition to journalistic routines and organizational features, along with the individual characteristics of journalists (Brüggemann et al., 2014; Herrero et al., 2017), we can now add evidence for cultural values as a factor. Furthermore, scholars have often included dominant ideology as an antecedent of news framing in the frame-building process (Lecheler & de Vreese, 2019). Our findings extend this model by adding underlying cultural values as a predictor. Extant research shows that journalists follow audience metrics in shaping their newsroom practice (Ferrer-Conill & Tandoc, 2018; Zamith et al., 2019). We show that journalists use values with cultural variability in their online content, and that values generate different levels of audience engagement. As such, journalists may take note of this triadic relationship between journalistic content, values, and audience engagement and adjust their references to values strategically to further increase engagement.
Although journalists intuitively know what story angles engage audiences and try to make news content appealing and catchy (Boesman & van Gorp, 2018), the online environment poses a particular challenge. This study confirms the efficacy of such efforts and indicates that news engagement is related to cultural connection. We suggest that this connection is circular, with the audience engaging with stories that appeal to personal values, and then journalists, either consciously or unconsciously, making such stories more prominent in future coverage. In such a way, social media audiences seem to be activating individual schemas for journalists who tend to embrace the same values. Nonetheless, traditional news values and news selection criteria remain central to creating news stories, as values play an additional role undergirding audience engagement.
Second, in the realm of public opinion research, scholars have shown that the media reflect a “hidden” value system and thus mirror the values that are dominant in society at large, transcending specific topics or issues (Rokeach, 1968). Political communication scholars could benefit from a deeper exploration of how such culturally bound value systems affect public opinion. While value systems have been shown to be generally stable over time, future research could examine how a specific value—for instance, self-direction or conformity, relates not only to media coverage of political issues (e.g., elections) but also to support for specific policy proposals or democratic institutions in general.
Third, and perhaps even more importantly, this study shows the underlying role of cultural resonance in news production by suggesting that values engage audiences in different ways. Not surprisingly, news coverage that fits with the audience predispositions tends to produce higher levels of online engagement, an indication of higher interest and psychological appeal. Conceptually, this also means that news content that is culturally incongruent would be less appealing and less engaging for media audiences. Thus, the link between cultural resonance of media content and audience reactions to that content should be further investigated.
Future research
While the notion that values influence social norms and attitudes is certainly not new, the link between distinct values, media production and online user engagement—a behavioral outcome, remains elusive in previous work. Future research should explore how journalists and, by extension, other online communicators adapt their messages not only to the social media environment but also to the underlying (cultural) values. Future research investigations should extend beyond health and risk communication topics to broader socio-political issues such as elections.
As Dimitrova and Matthes (2018, p. 337) note, “the uses and effects of social media can only be understood by taking the specific context into account.” We agree that external context, such as political and media systems need to be considered as well. Future research should incorporate countries and cultures outside of Europe. Furthermore, scholars should explore the bi-directional link between social media content producers (also known as journalists) and online consumers (also known as audiences) to determine more precisely how that mechanism works.
While this research contributes to our understanding of value references in online media posts and user engagement, several limitations of the study need to be acknowledged. First, the study results offer a snapshot of cultural variation in engagement and the role of value references on a specific platform, specifically Facebook. As others have suggested, changes in platform affordances could impact cultures of production and engagement (Bossetta et al., 2023). Furthermore, the goal of the study was to understand the role of values in news media content, with a focus on national-level differences and the societal issue of COVID-19. We identify general patterns of how value references affect user reactions and comments, noting variations along national lines at the height of the global pandemic (Ferrer-Conill et al., 2023). In addition to issue specificity, other organizational-level factors could impact the role values play in news media content (Strömbäck & Dimitrova, 2006). Thus, future research should explore differences in the role of values in state-owned versus private-owned news outlets as well as different media channels.
The study is one of the first to transfer the conceptualization of universal values from survey research to computational text analysis that does not rely on individuals’ value preference self-assessment (Schwartz, 1990). While recognizing the limitations of the research design in choosing a particular subset of values, we encourage future exploration of Schwartz’s entire value system as a framework for understanding the use of values in news content. While ambitious, this framework is well-suited for communication research due to its relational qualities. Furthermore, the Concept Mover’s Distance method was deemed valuable for identifying broad concepts in text analyses, although the concepts identified here are dependent on the subjective identification of concept anchors, as noted. Conceptually, this study probed whether and how value references are linked to COVID-19 posts, neglecting visual and multimodal aspects of social media content, which would be worthwhile avenues for future research, as values are powerfully communicated through visuals (Rodriguez et al., 2023).
Supplementary material
Supplementary material is available online at Human Communication Research online.
Data Availability
The data described in this article cannot be made publicly available due to Facebook’s TOS. The processed dataset and analysis scripts will be made available for reproducibility upon request.
Conflicts of interest
The authors declare no conflicts of interest.
Acknowledgements
We would like to thank the editorial team and the three anonymous reviewers for their thoughtful comments and attention to detail. Special thanks to Sophie Dickie for her assistance with the data validation process. We thank Toril Aalberg, Christian Baden, Audun Fladmoe, and Stefan Geiss for their feedback on previous versions of this manuscript.
