Asking about Social Circles Improves Election Predictions Even with Many Political Parties

Abstract

Traditionally, election polls have asked for participants’ own voting intentions. In four elections, we previously found that we could improve predictions by asking participants how they thought their social circles would vote. A potential concern is that the social-circle question might predict results less well in elections with larger numbers of political options because it becomes harder to accurately track how social contacts plan to vote. However, we now find that the social-circle question performs better than the own-intention question in predicting two elections with many political parties: The Netherlands’ 2017 general election and the Swedish 2018 general election.

Traditionally, national election polls in different countries have asked for participants’ intentions to vote for specific candidates (henceforth: “own-intention question”). In four elections, we previously found that we could improve election predictions by asking poll participants to estimate the percent of their social circles who will vote for different candidates (henceforth: “social-circle question”). Specifically, we reported this finding for the 2016 U.S. presidential election and the 2017 French presidential election (Galesic et al., 2018), as well as the 2018 U.S. midterm election and the 2020 U.S. presidential election (Olsson et al., 2021). Several possible explanations have been proposed for the effectiveness of the social-circle question, which are not mutually exclusive (Galesic et al., 2021). First, the social-circle question may indirectly allow participants to capture information about other people who otherwise would not have been represented in the election poll. Second, participants may be shy about their own intentions to vote for a seemingly controversial candidate and be more willing to admit their social contacts’ intentions to do so. Third, participants’ perceptions of their social contacts may influence how they actually end up voting. Fourth, the social-circle question may capitalize on friends’ social connections to predict emerging social trends.

However, a potential concern is that the social-circle question might predict elections less well in countries with a very large number of political parties. U.S. elections only include two major political parties. The French 2017 presidential election had six candidates. Yet, other countries may have many more parties participating in their elections. In countries with a large number of political parties, the social-circle question might produce less accurate election predictions because it could become difficult for participants to keep track of how their social contacts plan to vote.

Here, we compare how well the social-circle question and the traditional own-intention question predict two elections with many political parties. Specifically, we conducted two election polls that included the social-circle question and the own-intention question. The first election poll was conducted before the Netherlands’ 2017 general election. The second election poll was conducted before Sweden’s 2018 general election. We also investigate whether prediction accuracy differs for larger and smaller parties, and for potentially controversial populist parties.

Methods

Survey in the Netherlands

The 2017 general election in the Netherlands was held on March 15, 2017 and included 28 parties. Turnout was 81.6%, which was so much higher than in previous elections that some polling stations ran out of ballots. The newly elected parliament consisted of 13 parties. The populist Party for Freedom (PVV) became the second largest party with 13% of the votes (e.g., Damhuis, 2019). The liberal People’s Party for Freedom and Democracy (VVD) remained the largest party, but dropped from 41 to 33 seats (out of 150 seats in total). The Labor Party (PvdA) suffered the largest electoral defeat in Dutch Parliamentary history by dropping from 38 to 9 seats.

Sampling frame

The Dutch data were collected through the Longitudinal Internet Studies for the Social Sciences household panel, which is managed by Centerdata at Tilburg University in the Netherlands. This panel includes a representative sample of the noninstitutionalized adult population of the Netherlands. Households were selected through address-based sampling from a comprehensive municipal database. Participating households were given an internet connection and a computer if needed. The panel consists of approximately 8,000 individuals in 5,000 households, with a panel recruitment rate of 48% (Scherpenzeel, 2011).

Sample

Here, we analyze an online survey conducted from February 1 to 7 of 2017, more than a month before the election. In this survey, participants were presented with both the social-circle question and the own-intention question. Valid responses to both questions were provided by 2,230 (64%) of 3,500 panel members who were initially invited. From January 18 until March 14 of 2017, the Tilburg Election Study had already been running weekly online surveys that included the own-intention question but not the social-circle question.

Questions

The own-intention question asked: “Could you indicate for each party how likely it is that you will vote for it in the general elections on March 15, 2017?” (Dutch original: “Kunt u voor elke partij aangeven hoe groot de kans is dat u daar bij de Tweede Kamer verkiezingen op 15 maart 2017 op gaat stemmen?”). This probabilistic own-intention question was better at predicting participants’ actual votes, as compared with a deterministic own-intention question that asked participants which party they intended to vote for (De Bresser & Van Soest, 2019). This finding held especially for participants with high numeracy, for whom it might have been easier to provide useful probability responses (De Bresser & Van Soest, 2019).

The social-circle question asked: “For this question, only consider those social contacts who you expect will vote in the general elections. Of all your social contacts who you expect will vote, what percent do you think will vote for each of the parties below?” (Dutch original: “Neemt u voor deze vraag alleen uw sociale contacten in gedachten waarvan u denkt dat zij waarschijnlijk gaan stemmen bij de komende Tweede Kamerverkiezingen. Van al uw sociale contacten die waarschijnlijk gaan stemmen, welk percentage zal, denkt u, stemmen op elk van de onderstaande partijen?”). Social contacts were defined as “friends, family, colleagues, and other acquaintances of 18 years of age or older that you have communicated with at least briefly within the last month, either face-to-face, or otherwise.”

For each question, response options for the 13 largest parties appeared together on one screen. These 13 largest parties were the ones expected to end up in parliament, out of the 28 that participated (Table 1). Both questions also had an additional “Other” option, though its implementation differed between questions. The own-intention question, which had been part of Tilburg’s ongoing election polls before the social-circle question was introduced, explicitly asked participants to indicate which “Other” parties they had in mind. To reduce respondent burden, the social-circle question provided an “Other” option without asking participants for the specific “Other” parties.

Participants were not forced to provide responses that summed to 100%. We treated responses that added to 90% or more (across all parties and the “Other” option) as valid responses (N = 2,230). We excluded 1.1% of responses to the own-intention question and 1.3% of responses to the social-circle question because they summed to less than 90%. For the 1.4% of responses to the own-intention question and 2.2% of responses to the social-circle question in the Netherlands, responses ended up adding to a number of at least 90% but below 100%. We transformed those responses so that they did add up to 100%, while preserving their relative frequencies.

In addition to the social-circle question and the own-intention question, participants were asked to indicate how likely they were to vote at all, and what percentage of their social contacts were likely to vote at all.

Survey in Sweden

The 2018 general election in Sweden was held on September 9, 2018. Turnout was 87.2%, which was the highest in 33 years. The election included the eight parties that were already in the Swedish parliament, as well as multiple minor parties, which often varied locally. The same eight parties ended up in parliament again after the election. The populist Sweden Democrats (SD) party became the third largest party, with a vote share of 18% (e.g., Tomson, 2020). The Social Democrats (S) remained the largest party, but at 28% their vote share was the lowest since the early 1900s.

Sampling frame

The Swedish survey was conducted by the opinion institute Enkätfabriken, which maintains a panel of participants recruited by telephone from the Swedish national registry (Folkbokföringsregistret, at the time of the study containing roughly 70,000 participants).

Sample

Online survey data were collected from September 5–7, 2018, a few days before the elections. A total of 2,572 panel members who had not yet voted agreed to participate in the online survey, with 1,963 (76%) completing both the own-intention question and the social-circle question.

Questions

The own-intention question asked: “If you will vote, how likely is it that you will vote for any of the following parties?” (Swedish original: “Om du tänker rösta, hur sannolikt är det att du kommer rösta på något av följande partier?”). The social-circle question asked: “Please consider only those of your social contacts who are likely to vote in the upcoming election. Of all of your social contacts who are likely to vote, what percentage do you think will vote for the following parties?” (Swedish original: “Av alla dina sociala kontakter som tänker rösta, hur stor andel tror du kommer rösta på följande partier?”). Social contacts were defined as “friends, family, colleagues, and other acquaintances of 18 years of age or older that you have communicated with at least briefly within the last month, either face-to-face, or otherwise.”

For each question, all response options appeared together on one screen, and responses were forced to sum to 100% by the online interface. For each question, response options were provided for the eight largest parties that were expected to end up in parliament (Table 1). An “Other” option was included for both questions.

As in the Netherlands, participants were additionally asked to indicate how likely they themselves are to vote at all, and what percentage of their social contacts are likely to vote at all.

Analyses

First, we compared the election prediction errors of the own-intention question and the social-circle question for each party (Table 1). Election predictions of the own-intention question and social-circle question for each party were weighted, respectively, by participants’ estimates of how likely they themselves and their social contacts are to vote (following established procedures used by Galesic et al., 2018; Gutsche et al., 2014). For the own-intention question, a participant’s stated likelihood of voting for each party (0%–100%) was weighted by that participant’s stated likelihood of voting at all (0%–100%). The overall election prediction of the own-intention question for each party across participants was the mean of this variable divided by the mean stated likelihood of voting at all. The latter “ratio mean” ensured that predictions across parties added up to 100%. Similarly, for the social-circle question, a participant’s stated percentage of social contacts who were likely to vote for each party (0%–100%) was multiplied by that participant’s stated percent of social contacts who will vote at all (0%–100%). For each party, the overall election prediction of the social-circle question was the mean of this variable divided by the mean stated percent of social contacts who will vote at all. Standard errors were calculated through 500 bootstrap replications. For each party, t tests were used to compute whether differences between the mean prediction errors of the two questions were statistically significant.

We also compared the election prediction errors of the own-intention question and the social-circle question across all parties (Table 2). Following our previous work (Galesic et al., 2018), we made this question comparison across all parties before and after including the “Other” option. We additionally made this question comparison for the larger parties (receiving ≥10% of the vote), mid-size parties (6%–10% of votes), and smaller parties (≤5% of votes). The analysis for the smaller parties was computed without the “Other” option, and with the “Other” option. Across groups of parties, prediction errors for each question were computed in two ways: as average absolute errors across parties compared with the election outcome (Mosteller 3 measure; Mosteller et al., 1949) and as multinomial error measure reflecting the average absolute log ratio of predicted and actual proportion of voters supporting each party, weighted by parties’ vote share (B_w, Arzheimer & Evans, 2014). Standard errors were calculated through 500 bootstrap replications. t tests were used to compute whether mean prediction errors across parties were significantly different for the social-circle question and the own-intention question. All analyses were conducted in Stata.

Results

Table 1 shows that, despite the many participating parties, the social-circle question generally predicted election outcomes better than the own-intention question. For 9 of 13 parties in the Netherlands, and for 6 of 8 parties in Sweden, the social-circle question showed smaller errors in predicting the election outcome than did the own-intention question. The overall pattern held in analyses conducted across parties in each country. Table 2 shows that across the parties in the Netherlands and Sweden, both the mean absolute errors and multiparty error measure (B_w) were smaller for the social-circle question than for the own-intention question. Overall, this pattern seemed slightly more pronounced in Sweden than in the Netherlands.

The advantage of the social-circle question over the own-intention question generally remained after including the “Other” option in the analyses for both countries. However, overall differences between questions’ election prediction errors seemed somewhat larger when the “Other” option was excluded rather than included.

Table 2 additionally compares how well the social-circle question and the own-intention question predicted election results for parties of different sizes. In both countries, the mean absolute errors and the multiparty error measure (B_w) across election predictions for larger parties (getting ≥10% of votes) were generally smaller for the social-circle question than for the own-intention question. In both countries, the social-circle question also seemed to perform somewhat better than the own-intention question across election predictions for smaller parties (getting ≤5% of votes). However, the relative advantage of the social-circle question was less strong for the smaller parties than for larger parties, especially when including the “Other” option in the analyses. For mid-size parties (getting 6%–10% of votes) in the Netherlands, the own-intention question outperformed the social-circle question. That finding did not hold for mid-size parties (getting 6%–10% of the votes) in Sweden.

Additionally, it may be worth noting the findings for the potentially controversial populist parties, namely PVV in the Netherlands and SD in Sweden. Table 1 shows that the social-circle question was slightly more accurate than the own-intention question for these potentially controversial populist parties. However, the differences between the questions’ prediction errors did not reach the conventional level of significance (p < .05).

Conclusions

In previous papers, we reported that we could improve predictions of elections in the United States and France, by asking participants how they thought their social circles would vote, rather than by asking them traditional polling questions about their own voting intentions (Galesic et al., 2018; Olsson et al., 2021). A potential concern was that the social-circle question might predict results less well in elections with many political parties. However, our present results show that the social-circle question performed better than or as well as the own-intention question in predicting two elections with many political parties: The Netherlands’ 2017 general election and the Swedish 2018 general election.

Results appeared relatively less strong for the Netherlands than for Sweden, perhaps reflecting the larger number of political parties in the Netherlands, or the larger amount of time that passed between the poll and the election in the Netherlands. Yet, even small gains in election predictions may be of importance to election pollsters, especially because recent elections have proven harder to predict.

The relative advantage of the social-circle question appeared slightly stronger when predicting election outcomes for the larger parties (getting ≥10% of votes), though results were in the same direction when predicting election outcomes for the smaller parties (getting ≤5% of votes). The social-circle question may have worked somewhat better for larger parties because it is easier for people to know how their social contacts feel about the commonly discussed larger parties (vs. the less commonly discussed smaller parties). However, it is also possible that the seeming effectiveness of the social-circle question for larger parties was driven by specific characteristics of the two polls, the two countries, or their elections.

Including the “Other” option in the analyses seemed to slightly reduce the advantage of the social-circle question over the own-intention question. The “Other” option was meant to capture the newest and smallest parties, which are typically also the least well-known. Combined, the “Other” parties received about 2%–3% of the votes in both the Netherlands and Sweden. Even with familiar options, it can be hard to remember all the ones that are combined in an “Other” category (Fischhoff et al., 1978). Participants’ limited awareness of the newest and smallest parties may have especially undermined their ability to assess the percent of social contacts who would vote for those parties. Answers to the “Other” option of the own-intention question should be less affected, especially among participants who were not considering voting for these parties anyway.

The social-circle question did not significantly outperform the own-intention question for potentially populist parties, namely the PVV in the Netherlands and the SD in Sweden. Previously, it had been suggested that the social-circle question may encourage participants to disclose preferences for potentially controversial parties (Galesic et al., 2018). However, it may be the case that the voters in the Netherlands and Sweden did not think of these populist parties as controversial.

Social-circle questions aim to leverage people’s knowledge of their immediate social environment, which is one aspect of wisdom of crowds. By averaging people’s social-circle reports across a large national sample, any biases in social-circle perceptions may cancel out, allowing a useful signal to emerge. The social-circle question used here goes beyond the traditional wisdom-of-crowd questions in which participants are asked to forecast who will win the election (Murr, 2017). To make such forecasts, people may rely on media reports as well as their knowledge about their social environments. In recent elections, media predictions have been inaccurate (e.g., the 2016 Brexit referendum, the 2016. U.S. Presidential Election). The social-circle question bypasses the problem of biased media reports, and in addition benefits from capturing people’s apparently valid perceptions of their social contacts (Galesic et al., 2021).

One potential limitation of our social-circle question is that it did not ask participants to list their specific social contacts, or assess each contact’s likelihood of voting for each party. Instead, our social-circle question only asked participants to assess the percent of their social contacts who would vote for each party. Nevertheless, our relatively more concise social-circle question managed to capture social-network information that improved predictions of election outcomes in the Netherlands and Sweden. Asking people to list each social contact and describe their characteristics can easily take an average of 11 min (Burt, 1984). Our social-circle question took much less time to complete, and is therefore more suitable for election polls. Participants in the Netherlands took on average only 2.2 min to answer both the own-intention and social-circle questions, despite the large number of parties. We had no timing information for Sweden.

Another limitation is that our questions were asked as part of an online survey. We did not test our questions in other survey modes, and it is possible that in a telephone mode the social-circle question would need to ask about fewer political parties. In a face-to-face mode, the question should be feasible by implementing show cards or computer-assisted personal interviewing.

Overall, our findings suggest that election pollsters can improve their ability to predict election outcomes by asking participants about the voting intentions of their social contacts. In other contexts, we have also found that such social-circle questions can improve the prediction of people’s behaviors and societal trends (e.g., Bruine de Bruin et al., 2019).

Conflicts of interest: None declared.

Data collection in Sweden was funded by the Swedish Research Council (2014-1371). Data collection in the Netherlands was funded by Data Science Center Tilburg and the Tilburg School of Economics and Management. W. Bruine de Bruin also gratefully acknowledges funding from the Swedish Foundation for the Humanities and the Social Sciences (Riksbankens Jubileumsfond) Program on Science and Proven Experience “Science and Proven Experience,” Mirta Galesic from the National Science Foundation (NSF-DRMS 1949432 and NSF MMS 2019982) and Jochem de Bresser from the research program Innovational Research Incentives Scheme Veni with project number 451-15-018, which is financed by the Netherlands Organisation for Scientific Research (NWO).

References