Geographic Natural Experiments with Interference: The Effect of All-Mail Voting on Turnout in Colorado

Abstract

1. Introduction

In recent decades, the USA has witnessed an increase in methods of voting that differ from traditional in-person voting on Election Day. These reforms, commonly referred to as ‘convenience voting’, include in-person early voting (where voters may cast a vote in person before election day), no-excuse absentee voting (where voters may apply for an absentee ballot without providing a reason for doing so), and all-mail voting (where voting by mail is mandatory)—see Gronke et al. (2008) for a review. These policies are often implemented with the goal of reducing the costs of voting, which is in turn expected to increase voter participation.

Among the different convenience voting policies that have been adopted, all-mail voting is the most drastic, since, under this policy, in-person precinct voting is eliminated and there are no polling places; instead, citizens receive a ballot in the mail several weeks in advance of Election Day and then return it by mail to the election administration office. Since all-mail voting is the only convenience method that eliminates precinct-place voting, its effects on turnout could be different from the effects of other types of convenience voting. While all-mail voting may be more convenient, the move to all-mail elections reduces the social aspect of voting, which can be a key motivator for political participation (Gerber et al. 2008). All-mail voting also removes the possibility of using Election Day as a focal point for mobilization efforts by political parties. Combined, these factors might result in fewer voters casting a vote.

Given the far-reaching nature of all-mail voting reforms, scholars have been interested in studying whether they affect voter turnout. As various states have implemented vote-by-mail systems either on a statewide or more local basis, a number of studies have attempted to estimate whether this mode of voting increases or decreases turnout. Much of the focus has been on the state of Oregon, where polling-place voting was gradually eliminated during the 1990s, and since 1998 all statewide primary and general elections are conducted by mail only. Several studies have concluded that Oregon’s all-mail voting reform increased turnout, though the estimated magnitude of the change varies considerably from study to study (Southwell and Burchett 2000; Karp and Banducci 2000; Berinsky et al. 2001; Richey 2008; Gronke and Miller 2012). Some additional evidence on this question comes from other states. Kousser and Mullin (2007) and Bergman and Yates (2011) study California, where county election officials can assign voters to all-mail voting precincts in low-population areas, and find that turnout appears to be lower under an all-mail voting system. Gerber et al. (2013) study the large-scale move from polling-place to all-mail elections in the state of Washington, and they find that all-mail voting increases turnout by 2–4 percentage points.

We examine this question using a geographic natural experiment in Colorado, where in 2010 counties were given the choice to require that votes be cast by mail during the primary election. Counties that adopted all-mail elections removed other alternative methods of voting, while counties that did not adopt all-mail voting still offered traditional polling-place voting on Election Day (and also allowed by-mail no-excuse absentee voting). In general, given that voter administration is conducted by county governments, counties may chose their mode of voting to try to accomplish their specific voter turnout goals. This type of strategic decision-making may complicate naive statistical inferences that simply compare all-mail counties to in-person counties. In an attempt to minimize these complications, our study focuses on voters in Basalt, a town that is split by the border between Eagle county, which adopted all-mail election voting, and Pitkin county, which retained in-person voting. Our research design focuses on a narrow area around the boundary between both counties, and makes the assumption that, within this small area, voters in the town of Basalt are split in a haphazard fashion between Eagle and Pitkin counties, after conditioning on covariates. Based on this research strategy, we draw inferences about the effects of all-mail elections on voter turnout.

A key element of our research design is our focus on a small geographic area around the boundary that separates both counties, since citizens who reside close to the county border on either side share important predetermined characteristics that may be related to voter turnout decisions. This focus on treated and control voters who reside near each other, while increasing comparability, may also increase the likelihood that treated voters interact with control voters in a way that affects their outcomes. The presence of interference between voters would undermine the interpretation of our estimates as the average treatment effect of all-mail voting. In general, research designs based on comparisons of units who reside in close geographic proximity to each other are more likely to suffer from spatial forms of interference between units, and will face a trade-off between increasing comparability and reducing interference (Keele et al. 2017). However, the patterns of interference between units may be more general than those induced by residential proximity. For example, workers could be influenced by colleagues whose residence is geographically far from their own, but with whom they interact at the workplace. Even more drastically, interference could arise between units who are never geographically or physically close in any capacity, in particular via social media interactions. The framework we use below could be applied to these more general forms of interference, replacing our notion of residential proximity with a notion of proximity in a social network, assuming that the latter is known—for an example of a study that considers both types of proximity, see Verdery et al. (2012).

One goal in our study is to understand whether interference alters our inferences about the effect of all-mail voting. Our empirical analysis builds on prior work that employs the potential outcomes framework to study interference between units. Sobel (2006) characterizes several estimands of interest under interference, and shows that the usual difference-in-means estimator in a completely randomized experiment is no longer unbiased for the average treatment effect. Hudgens and Halloran (2008) consider a two-stage randomization, in which interference occurs within but not between groups, and define direct and indirect causal effects that consider how the outcomes of one unit change as the treatment assignment of all other units stays constant or changes. Hierarchical models in which interference occurs within but not between groups are also considered, among others, by Tchetgen Tchetgen and VanderWeele (2012), Vanderweele (2008), and VanderWeele et al. (2013). Aronow and Samii (2017) consider the estimation of average causal effects under general forms of known interference. Rosenbaum (2007) and Bowers et al. (2013) consider hypothesis testing under interference in a Fisherian framework. Gerber and Green (2012) consider the problem of spatial interference in randomized experiments, and Sinclair et al. (2012) design a multilevel voter-mobilization experiment to detect spillovers within and between households.

Methodologically, our approach is most similar to the setting in Hong and Raudenbush (2006) and Verbitsky-Savitz and Raudenbush (2012). Hong and Raudenbush (2006) study the effect of retaining low-achieving children in kindergarten versus promoting them to 1st grade, and model interference effects by means of a scalar function of the treatment assignment vector within each school. Verbitsky-Savitz and Raudenbush (2012) apply the ideas in Hong and Raudenbush (2006) to a spatial setting to study the effect of a community policing program on neighborhoods’ crime rates in Chicago. They assume that the potential outcome of a given unit depends on the other units’ potential outcomes via a scalar function that contains the proportion of contiguous units.

In our analysis of the effects of all-mail voting under spatial interference, we also impose the assumption that interference is a scalar function of the treatment assignment vector. In particular, we assume that interference is a function of geographic proximity to a sufficiently dense area of voters of the opposite treatment status, which vastly reduces the number of potential outcomes for every unit and leads to two estimands of interest. This approach is similar to the approach in Verbitsky-Savitz and Raudenbush (2012), who model interference as a function of the proportion of contiguous units, although we impose an additional restriction. Our setup treats the geographic locations of the voters in our sample as random, and the boundary between treated and control areas as fixed—an approach that is particularly well suited to geographical natural experiments that focus on a narrow band around a boundary, and differs from other approaches that allow for spatial interference under the assumption that geographic locations are given. Our function of interference can be modified to reflect particular patterns of geographic spillovers.

The remainder of the article is organized as follows. In Section 2 we describe the Colorado application in more detail. In Section 3 we present our notation and describe our causal estimand under the assumption of no interference. In Section 4 we allow for geographic interference, and explore some further issues in Section 5. In Section 6, we use this framework to estimate the treatment effect of Colorado’s all-mail voting on turnout, first ignoring interference between units, and then allowing for interference based on residential proximity (Section 6.1). In this section, we also explore sensitivity to differential registration (Section 6.2). We offer concluding remarks in Section 7.

2. All-Mail Voting in 2010 Colorado Primary

In recent years, the state of Colorado has implemented several reforms aimed at making voting more convenient. Starting in 2008, voters could choose to be placed on a permanent vote-by-mail list. For the 2010 primary, the Secretary of State allowed each county to choose whether to hold either all-mail elections, use voter centers, or hold traditional in-person voting at precincts. Figure 1 contains a map showing the mode of election selected by each county. While urban areas generally selected all-mail elections, many rural counties chose to use in-person voting.

Figure 1.

County map of Colorado with model of voting and location of Basalt highlighted.

Open in new tabDownload slide

The different modes of voting chosen by different counties allow us to study how election mode affects voter turnout. However, given that counties are responsible for election administration and were able to select their preferred mode of election in the 2010 primary, comparisons across counties might be invalidated by unobserved confounding or heterogeneity. To minimize this concern, we looked for some location where a town or city is split by a county border, where one county uses all-mail voting, while the other county uses in-person voting. We found that one town in the southwestern part of the state, Basalt, was split exactly in this fashion.

Figure 1 shows the location of the town of Basalt. According to the 2010 census, Basalt has a total population of 3857. The population is largely White, and about 20% of it identifies as Hispanic. The town is close to the resort city of Aspen, and using property sale records, we found that the median house price in 2010 was over $600,000. Figure 2 contains a map showing the town in greater detail. The central part of Basalt is split by the county border which defines mode of election, and this part of the town contains the main shopping district, residential areas, and schools. While the county border splits the town, the entire area is within the same school district. Moreover, all residents of Basalt attend the same set of public schools which are located within the central part of the town. Although property taxes in Colorado have a county component, property taxes are based on five different tax zones with school district contributing the most to the overall property tax burden.

Figure 2.

The county discontinuity: Basalt split by county boundary.

Open in new tabDownload slide

Primary elections often hold little interest for voters, since primary races are often uncompetitive. The 2010 Colorado primary, however, had three high-profile elections on the ballot. In the Republican gubernatorial primary, a Tea Party insurgent beat Scott McInnis, a six-term U.S. representative, after it came to light that McInnis plagiarized a water study he was paid to conduct. In the Democratic US Senate primary, the candidate endorsed by then-President Obama narrowly beat a more liberal candidate endorsed by former president Bill Clinton. In the Republican US primary Ken Buck, a Tea Party candidate, beat Jane Norton—the candidate endorsed by the Colorado Republican party establishment. Results from the primary received national coverage and were featured on the front page of the New York Times.

3. Estimation and Inference without Interference

We adopt the potential outcomes framework for causal inference (see Holland 1986; Rubin 2005) assuming first that voters do not interfere with one another. We generalize the framework to allow for interference in Section 4.

We conduct our analysis with individual-level voter data. We use the binary variable $Di∈[0,1]$ to denote treatment status for resident i, with D_i = 1 if i resides in Eagle County and is assigned to an all-mail election, and D_i = 0 if i resides in Pitkin county and may vote in-person on Election Day. Each resident has several potential outcomes, only one of which is realized by the assignment of treatment. There are also k predetermined covariates for each resident, which we denote by X_i. The observed data are ${Yi,Di,Xi}i=1n$⁠, which we assume is an i.i.d. random sample from a larger population. We collect all treatment indicators in the n-vector D, and let $Yi(D)$ be the potential outcome of resident i. We denote the observed outcome by $Yi≡Yi(d)$⁠, where d is the realized treatment assignment vector. In general, if we let the treatment status of every resident affect the potential outcome of every other resident, every i will have one distinct potential outcome for every value that the treatment vector D might take, which is 2ⁿ. We start by entirely simplifying this problem and assuming that there is no interference between residents, as formalized in the following assumption.

Assumption 1 (No Interference). The potential outcome of each unit depends only on the treatment received by that unit and not on the treatment assigned to any other unit: for all $D˙≠D¨, Yi(D˙)=Yi(D¨)$ if $Di˙=Di¨$⁠, for $i=1,2,…,n$⁠.

Under Assumption 1, we can write $Yi(D)=Yi(Di)$⁠, since i’s potential outcome only depends on the treatment received by i. In this case, the observed outcome simplifies to $Yi=Yi(1)Di+Yi(0)(1−Di)$⁠. The quantity $τi=Yi(1)−Yi(0)$ captures the effect of all-mail voting for the ith voter. Our interest is estimating the average treatment effect on the treated (ATT), $τ=E[Yi(1)−Yi(0)|Di=1]$⁠, which is only defined under the assumption of no interference.

3.1 A geographic identification strategy

We estimate the effects of all-mail voting on voter turnout using a geographic natural experiment. Under this identification strategy, a geographic or administrative boundary splits units into two adjacent areas, one of which receives a treatment, $At$⁠, and the other of which receives control, $Ac$⁠, and analysts make the case that the assignment of units into treated and control areas occurs in an as-if random fashion (Keele and Titiunik 2015, 2016). Researchers make comparisons between units in the treated and control areas to infer the effect of the treatment on an outcome of interest, relying on the spatial proximity of each unit to the border between $Ac$ and $At$⁠, and on the fact that the treatment changes abruptly along this boundary—that is, D_i = 1 if unit i is located in $At$⁠, and D_i = 0 if i is located in $Ac$⁠. Applying this strategy to our application, we assume that around the county border that divides the town of Basalt into all-mail and in-person voting regimes, individuals choose their residence on either side of the county boundary on an as-if random fashion, possibly after conditioning on predetermined covariates.

In essence, the assumptions behind a geographic natural experiment require that the placement of each unit on either side of the geographic boundary between $Ac$ and $At$ be as-if random or, in other words, that units cannot precisely sort or self-select to one side of the boundary based on unobserved factors that are also correlated with the outcomes of interest. A consequence of this assumption is that observable predetermined covariates should be similar in expectation within some narrow area around the border of interest. A weaker assumption is that treatment assignment is as-if randomized for those who live near the border, after conditioning on a set of observable covariates (Keele et al. 2015). Given the need to condition on covariates, such designs have been characterized as geographic-quasi experiments (GQEs) (Galiani et al. 2017; Keele et al. 2017 ). Since we are interested in the ATT, we adopt a version of this assumption that only restricts the average potential outcome under control: we assume that there exists a small neighborhood around the boundary that separates both areas where the average potential outcome under control is mean independent of the treatment given the covariates. We state it formally below.

Assumption 2 (Conditional Mean Independence in Local Neighborhood). For all units that reside in a narrow band around the boundary that separates $Ac$ and $At, E[Yi(0)|Xi,Di=1]=E[Yi(0)|Xi,Di=0]$⁠.

Note that Assumption 2 implicitly invokes Assumption 1, which reduces the set of potential outcomes to $Yi(1)$ and $Yi(0)$⁠. Moreover, since our boundary of interest is simultaneously the boundary of multiple institutional, administrative, or political units, and we wish to make inferences about the effect of only one of these treatments, we must assume that the treatment of interest is the only treatment that affects potential outcomes, i.e. that there are no compound treatments (Keele and Titiunik 2015). In particular, in our application we must assume that no other county-level factor affects voter turnout other than the administration of voting.

4. Interference between Units in a GQE

In geographical natural experiments generally, and the GQE in particular, the research strategy is based on a comparison of units that are spatially proximate, under the assumption that units very close to one another but with opposite treatment status can provide valid counterfactuals for each other. However, in some cases, this focus on spatially proximate units may introduce the possibility of spillovers or interference between units. In our application, we are concerned about indviduals who reside in the all-mail-voting (treated) area being influenced by individuals who reside in the in-person-voting (control) area because in the latter area Election Day still acts as a focal point, and the act of voting is socially coordinated and shared. This might make the election generally more salient in the in-person area in the weeks before the election, affecting the propensity to vote of all-mail residents on the other side of the boundary. We assume that interference between units depends on geographic proximity to units of opposite treatment status. This is similar to the approach in Verbitsky-Savitz and Raudenbush (2012), although we do not focus on contiguity but rather on the density of control units that reside within a pre-specified distance of a given treated unit. Since we are not similarly concerned about individuals in the in-person-voting area being affected by individuals in the all-mail-voting area, we assume that interference is one-sided, from control to treated—but the setup is generalizable to two-sided patterns of interference.

Thus far, we assumed that resident i’s treatment status did not depend on the treatment status of any other resident, allowing us to write potential outcomes as $Yi(D)=Yi(Di)$⁠, since i’s potential outcome only depended on i’s own treatment status. If we allow for any pattern of interference, we must work with the full vector $Yi(D)$⁠, which allows individual i’s treatment status to depend on the treatment status of every other individual. However, this level of generality is unworkable, as the large number of causal effects per subject makes it difficult to summarize the data in any interpretable way.

The function $gi(·)$ is an indicator for whether individual i receives interference, taking a value of 1 or 0 for every individual. If individual i is treated (i.e., resides in the all-mail-voting area), $gi(D;δ,η)=1$ if there are at least η control individuals who reside within δ meters of i’s location, and $gi(D;δ,η)=0$ if there are less than η control individuals in a δ radius around i’s location. If individual i is control (resides in the in-person-voting area), $gi(D;δ,η)=0$ regardless of how many treated individuals reside close to i—because we have assumed one-sided interference only.

Thus, we capture our model of one-sided geographic-based interference by letting the potential outcomes depend on the full vector of treatment assigments D in a restricted way; in particular, we let individual i’s potential outcome depend on its own treatment status D_i and on the treatment status of other individuals only through the function $gi(D;δ,η)$⁠. Thus, in our interference framework, $Yi(D)=Yi;δ,η(Di,gi(D;δ,η))$⁠, reducing the number of arguments in each individual’s potential outcome from n to 2. This is similar to the approach in Hong and Raudenbush (2006) and Verbitsky-Savitz and Raudenbush (2012), where the dependence of i’s potential outcomes on the treatment assignment of all other units is also modeled via a scalar function that substantially reduces the range of possible potential outcomes that may occur.

Under our specific assumption of geographic-based interference, every unit has three potential outcomes:

• $Yi;δη(1,1)$⁠: Individual i resides in the all-mail-voting area and receives spillovers, i.e. there are at least η control individuals in the in-person-voting area within δ meters of i’s location.

• $Yi;δη(1,0)$⁠: Individual i resides in the all-mail-voting area and does not receive spillovers, i.e. there are less than η control individuals in the in-person-voting area within δ meters of i’s location.

• $Yi;δη(0,0)$⁠: Individual i resides in the in-person-voting area. Since $gi(D;δ,η)=0$ for all individuals in the in-person-voting area, we can simply write $Yi;δη(0,0)=Yi(0)$⁠.

Both $τT,δη$ and $τS,δη$⁠, however, depend on three different potential outcomes, and only one of those is observed for every i. We now investigate assumptions that, in the particular context of geographic natural or quasi experiments, could be invoked to identify these parameters. We first note that, letting Y_i denote the observed outcome for individual i, we have the following equalities between observed and potential outcomes:

• $Yi=Yi(0)$ if D_i = 0

• $Yi=Yi;δη(1,1)$ if D_i = 1 and $gi(D;δ,η)=1$

• $Yi=Yi;δη(1,0)$ if D_i = 1 and $gi(D;δ,η)=0$⁠.

We now consider the following assumption,

A sufficient condition for Assumption 3 is that each unit is randomly assigned to a geographic location in the combined treated area, so that whether they fall in the interference region (⁠$gi(D;δ,η)=1$⁠) or the non-interference region (⁠$gi(D;δ,η)=0$⁠) is unrelated to their potential outcomes. Our assumption is weaker than this, since it requires only that conditional on pretreatment covariates, falling in the interference region in the treatment area is mean independent of potential outcomes, though it still is a strong assumption. However, as we discuss in detail below, under the type of treatment assignment that is typical of geographic natural or quasi experiments, this assumption might be plausible if the neighborhood around the boundary that separates treated and control areas is small enough and enough pretreatment covariates are available.

4.1 The treatment assignment mechanism in geographic natural and quasi experiments

We now discuss the plausibility of Assumptions 2 and 3, in particular the latter. Of course, in the absence of a concrete research design, both assumptions are exceedingly strong. But these assumptions may be more plausible when inferences are based on a GQE, where comparisons are made between units on one side or the other of the boundary, perhaps after conditioning on predetermined covariates. In a succesful GQE without interference, in a sufficiently small neighborhood around the boundary, we would have $E[Yi(0)|Di=0,X]=E[Yi(0),X]$⁠, as stated in Assumption 2. But the random or as-if random assignment of units to treated or control areas does not imply Assumption 3, the assumption on which our derivations under interference were based. We now offer some discussion on the scenarios under which the assumption can be expected to hold.

If we consider the geographic location of every unit as fixed, an experiment where every unit has the same probability of receiving treatment might result in each unit having a different probability of receiving spillovers. This arises from the fact that when the locations of units are fixed, units that are spatially isolated and have no other units near them may have a small or 0 probability of receiving spillovers. In contrast, units that are in close proximity to other units may have a positive and larger probability of receiving spillovers. In this case, estimation of the parameters of interest may require weighting the observations according to each unit’s probability of receiving spillovers (Gerber and Green 2012, Ch. 8).

However, keeping units’ locations fixed and randomly locating the boundary may not be the most plausible way to conceptualize treatment assignment in our application. In many geographic designs, we might view the boundary as fixed, but we assume that, within a narrow band around this boundary, units randomly choose their geographic location. Units that happen to choose a location in the treated area receive the treatment, and units that happen to choose a location within the control area receive the control condition. In this sense, the assignment of treatment is seen as a result of units’ location decisions, and therefore units’ locations are not seen as fixed. Under this assignment mechanism, if every unit is equally likely to select any location within a fixed band around the boundary, each unit is equally likely to receive spillovers and ex ante, within the fixed band, the units’ potential outcomes with and without interference are equally likely to be revealed.

Importantly, if we think this form of assignment mechanism is in operation, Assumption 3 holds naturally, provided the band around the boundary is sufficiently narrow. Why might we think this is true in the context of the quasi experiment in Colorado? First, the boundary is a county border, which has existed for decades, so it is natural to think of the boundary as fixed and individual location decisions near the border as random. Secondly, our focus on a town where residents on each side of the county border share the same city amenities, gives us the basis to assume that the choice of residence on each side of the boundary is unrelated to the (control) turnout potential outcomes, once we condition on predetermined covariates. Nonetheless, we must emphasize that however plausible this form of assignment mechanism appears to be, this is an untestable and strong assumption, and our conclusions about the extent of interference depend on its validity.

5. Exploring Interference Effects and the Extent of Interference

Before turning to the exploration of interference in our application, we use the framework introduced above to explore some features of the interference pattern in more detail. We explore two specific issues. First, we relax the assumption that all individuals in the interference area receive interference. So far, our framework assumed that all treated individuals near enough control individuals—that is, treated individuals with $gi(D;δ,η)=1$—received spillovers, and all other treated individuals were free of interference. We consider a generalization of this condition where only a proportion of the treated individuals near populated control areas receive interference—while individuals with $gi(D;δ,η)=0$ are still assumed to receive no interference. Secondly, we consider how the results from an analysis that mistakenly ignored interference and proceeded to simply compare the outcomes among treated and control units as if there were no spillovers would differ from the interference-free effect in an analysis that took interference into account.

5.1 Sensitivity of interference effect when geographic spillovers affect a subset of units

Above, the effect of interference was captured by the parameter $τS,δη$⁠, which assumed that every individual in the interference area was in fact affected by control individuals. We now investigate what happens when only a fraction of the individuals who reside in the interference area are affected by spillovers.

This analysis reveals that the differences in observed average treated outcomes between the interference-free and the interference areas are equal to the interference effect $τS,δηq$ when q = 1. But when q < 1, the true effect of interference will be larger than the observed difference in outcomes by a factor $1/q>1$⁠. Thus, assuming that interference affects all units in the interference area (q = 1) gives a lower bound on the interference effect.

5.2 Characterizing the extent of interference

We now investigate the degree to which the conclusions from an analysis that ignored interference when interference was in fact present would lead to incorrect conclusions. Recall that, in the absence of interference, we defined the parameter $τ=E[Yi(1)−Yi(0)|Di=1]$⁠. This parameter, however, is undefined in the presence of interference. When we allowed for interference, we focused instead on the parameters $τT,δη=E[Yi;δ,η(1,0)−Yi(0)|Di=1]$ and $τS,δη=E[Yi;δ,η(1,1)−Yi;δ,η(1,0)|Di=1]$⁠.

Moreover, when the potential outcomes are bounded as they are in our voting application where they take values equal to 0 (non voting) or 1 (voting), given a value of $pI$⁠, we can calculate the maximum value of the term $pI·E[Yi|Di=1,X,gi(D;δ,η)=1]$⁠. In our case, this upper bound is $pI$⁠.

This discussion illustrates that it is not only of interest to calculate the effect of interference but also to establish whether interference affects a large enough proportion of units. If it does not—that is, if $pI$ is small—in the case of bounded outcomes, we may be able to assert that an analysis that ignored interference when in fact interference is present would produce treatment effect estimates that would approximate the effects under no interference.

6. Application to the 2010 Colorado Primary

We now apply the framework described above to analyze the 2010 Colorado primary, starting with a description of the data. Our main source of information is the Colorado voter registration file, the database of registered voters maintained by the state of Colorado for administrative purposes. We acquired these data from a private vendor. The administrative data from the state contain a limited number of covariates including date of birth, gender, voting history, voters’ addresses, and the legislative districts in which each voter’s address is included. The private vendor also includes an additional variable estimating the voter’s likely race. In this region, most voters are White with a substantial minority of Hispanic voters. To determine voter locations, we geocoded each voter’s location using the address in the voter file.

We first restrict our data to include only those individuals who in 2010 lived in the central area of Basalt that is split by the border between Eagle and Pitkin counties—the border that determines all-mail or in-person voting. Within the central area of Basalt, our covariate-adjusted results condition on the set of pre-treatment covariates that we have available: age, gender, whether the individual is Hispanic, voting history for 2008 and 2006, party affiliation as declared in the registration file, and an indicator for whether the individual’s registration status is considered active by the state. Since our main data source is the Colorado registration file, all our turnout measures—including both pretreatment turnout shares and the turnout share in the 2010 primary—are constructed as the proportion of individuals in the registration file that turn out to vote in a given election. This means that our turnout measures condition on registration status. We discuss the potential methodological complications associated with such measures in Section 6.2.

Before presenting the estimation results, we examine the observed covariates in our sample within central Basalt, to asses whether the treated and control areas inside this small region are already comparable in terms of these characteristics. Table 1 contains sample means and the absolute standardized differences in means (difference in means divided by the pooled standard deviation between groups before matching) for three demographic characteristics, voter registration status, and turnout in the last four elections. While geographic proximity produces acceptable balance on residents’ Hispanic ethnicity, gender, age, and active registration status, there are larger differences in turnout in past elections, with standardized differences in these variables exceeding 0.20. These differences suggest that unadjusted comparisons between the groups cannot be interpreted as causal effects of all-mail elections on 2010 voter turnout.

We estimate Equation (2) by least-squares using the variables in Table 1 as covariates. Table 2 contains point estimates of τ and associated 95% confidence intervals. We also report the unadjusted least-squares estimator of τ corresponding to Equation (1), which is simply the unadjusted difference in turnout rates between treated and control areas. This unadjusted estimate is reported only for completeness, but we do not believe it can be interpreted as a causal effect due to the observable pre-treatment differences between the areas.

We find that, in a narrow band around the boundary between the treated and control areas and after conditioning on a set of observed characteristics, the voter turnout rate was 6 percentage points lower in the all-mail county, with a 95% confidence interval ranging from −0.091 to −0.027. Given our assumptions, this difference is the average all-mail voting effect on turnout for the residents in the treated area. This analysis assumes that outcomes of an individual do not depend on the treatment status of other individuals, a constraint that we relax in the following section.

6.1 Empirical results allowing for spatial interference

We now re-analyze the effects above based on our GQE. Under our setup and given the assumptions introduced above, we calculate the interference-free treatment, $τT,δη$⁠, and the interference effect, $τT,δη$⁠, for different values of η and δ.

To implement estimation of these quantities, we calculate, given δ and η, the interference set, $Iδη$⁠, which is simply the collection of all treated individuals in our data for whom $gi(D;δ,η)=1$⁠. Once we form $Iδη$⁠, we can estimate $τS,δη$ through a comparison of treated individuals that are in $Iδη$ to those who are not, since this set identifies all treated individuals that are spatially proximate to enough control voters and may have been subject to interference. Similarly, we estimate $τT,δη$ through a comparison of control voters to treated voters not in $Iδη$⁠. To form $Iδη$⁠, we must locate the distance between each treated unit and each control unit, calculate whether any control units reside within δ meters of each treated unit, and then count the number of control units within δ distance. We calculate this set using the following algorithm.

The size of $Iδη$ depends on the values we choose for δ and η. As we make δ larger and η smaller, we allow for interference to become more severe. Once we have formed the set $Iδη$⁠, calculation of the quantities of interest is straightforward.

We apply these methods to the data from the 2010 primary in Colorado. For a set of η and δ values, we estimate $τS,δη$ and $τT,δη$⁠. Different values of η and δ allow for more or less stringent definitions of spatial interference. For example, if we set δ to 250 m and η to 1, our model of interference asserts that all treated units who have at least one control unit within a 250-m radius of their location were subject to interference. We set δ to 250 and 100 m, and η to 1, 5, and 10 individuals. We estimate $τS,δη$ and $τT,δη$ for each set of values to observe whether these quantities change as a function of both distance and the density of control units that treated individuals are near to.

Table 3 contains estimates for $τS,δη$ and $τT,δη$ (which we denote as $τ^S,δη$ and $τ^T,δη$⁠) along with 95% confidence intervals for each set of η and δ values. The table also reports the proportion of treated voters in the interference set (i.e., an estimate of $pI$⁠). All results in the table are covariate-adjusted using least-squares estimates from Equation (3), employing heteroscedasiticty-robust standard errors.

First, we consider the estimates when δ  = 100 (top panel). When η  = 1, we let interference affect all treated units who live within 100 m of at least one control unit. Under this interference scenario, 9.6% of the treated observations are affected by interference, and estimate of $τT,δη$ is −0.059 (95% confidence interval from −0.092 to −0.026), which is very similar to our covariate-adjusted estimate of τ in Table 2—the ATT estimated under no interference. In contrast, the estimate for $τS,δη$ is −0.0056, much closer to 0 and not significantly different from 0 (95% confidence interval ranges from −0.061 to 0.050). Thus, for δ  = 100 and η  = 1, there is not a statistically significant difference in the patterns of voter turnout between the interference and the interference-free area. When η is 5 or 10, the proportion of treated voters inside the interference area is naturally smaller. In this case, the estimates of $τS,δη$ are, respectively, −0.012 and −0.054, both indistinguishable from 0. Since the proportion of treated units in the interference set is very small (⁠$pI$ is equal to 0.045 and 0.031 for η  = 5 and η  = 10, respectively), the covariate-adjusted estimate for $τT,δη$ remains very similar to the analogous estimate for η  = 1—and also to the covariate-adjusted effect under no interference reported in Table 2.

Next, we consider the case of δ  = 250 (bottom panel of Table 3). When η  = 1, the estimated interference-free effect is −0.069 (confidence interval ranging from −0.104 to −0.034), similar to our previous estimates. In addition, turnout is about 3.3 percentage points higher for treated voters in the interference area relative to the turnout of treated voters in the interference-free area, but this effect is again statistically indistinguishable from 0. Note that in this scenario interference is assumed to be much more prevalent, with 29% of treated voters inside the interference area. When we set η to either 5 or 10, the estimated effect of interference, $τ^S,δη$⁠, continues to be indistinguishable from 0. In contrast, the confidence intervals including for $τT,δη$ are $[−0.095,−0.029]$ for η  = 5 and $[−0.091,−0.025]$ for η  = 10, very similar to the confidence intervals under δ  = 100 and also to confidence intervals for τ ignoring interference reported in Table 2.

In sum, our analysis shows that, under our assumptions, the interference effect is indistinguishable from 0 in all cases. Moreover, except under the most extreme scenario of interference with δ  = 1 and η  = 250 where 29% of treated voters are in the interference set, the proportion of treated voters in the interference set tends to be small (between 17 and 3%), leading to an interference-free effect that is similar to the effect estimated assuming that interference is not present. These results suggest that all-mail voting reduced voter turnout in the 2010 primary election by an average of about 6 percentage points.

6.2 Sensitivity to differential voter registration

A potential complication with our analysis is that our data source is the Colorado voter registration file, and thus our measure of voter turnout is constructed as the proportion of registered citizens who turn out to vote. If the decision to register is itself affected by the mode of voting, our reported results could misrepresent the true turnout effects. Eagle county’s decision to adopt all-mail voting was announced in 2010 before the primary election was held, making it possible for citizens to adjust their election registration decisions in response to the upcoming change in the mode of voting.

The ideal solution would be to obtain a list of the total voting eligible population in the treated and control areas at the moment of the 2010 primary election. Unfortunately, such data are unavailable. An alternative is to follow the approach in Nyhan et al. (2017) and explore how much differential registration between the treated and control groups could occur before our observed turnout effects—which construct turnout shares as total voters over total registration—became consistent with a zero effect on the true turnout share—the share of voters to the total voting eligible population. Generalizing the sensitivity analysis in Nyhan et al. (2017) to include covariates, for every estimated effect reported in Table 3, we report the differential registration factor $k⋆$—the amount of differential registration between treated and control groups that would be required to produce the difference in turnout-to-registration rates we observed if the true turnout effects were equal to 0.

Given a treated and a control or reference group, the differential registration factor is estimated by simply dividing the turnout-to-registration share in the control group (⁠$TcReg$⁠) over the turnout-to-registration share in the treatment group (⁠$TtReg$⁠). For estimation of the $k⋆$ associated with $τT,δη, TtReg$ includes all units in the treatment group outside of the interference region (D_i  = 1 and $gi(D;δ,η)=0$⁠), and $TcReg$ includes all control units (D_i  = 0). For estimation of the $k⋆$ associated with $τS,δη, TtReg$ includes all units in the treatment group inside the interference region (D_i  = 1 and $gi(D;δ,η)=1$⁠), and $TcReg$ includes units in the treatment group outside of the interference region (D_i = 0 and $gi(D;δ,η)=0$⁠). To incorporate covariates, we estimate $TcReg$ with the average predicted values from the linear model for all observations in the treatment group, but with the corresponding treatment indicator set to 0.

The results of the sensitivity analysis are shown in Table 4, where we report, for each of the three values of η combined with δ  = 100 or δ  = 250, the differential registration factor for both $τT,δη$ and $τS,δη$⁠. For each combination of η and δ, the first two columns report $TcReg$ and $TtReg$⁠, the estimated values of the turnout-to-registration shares for each group—the difference between these values are the effects reported in Table 3. The third column reports the differential registration factor associated with each effect. For example, for η  = 1 and δ  = 250, $k⋆$ is 1.447, showing that the rate of registration in the treatment group would have to be 44.7 percentage points higher in the treatment group than in the control group to make the estimated interference free treatment effect $τ^T,δη=−0.069$ (reported in Table 3 and also obtained from Table 4 as 0.155–0.224) consistent with a 0 effect on true turnout rates. A 44.7 percentage point difference in registration rates is a very large effect, unlikely to occur in practice. This large value of $k⋆$ suggests that the interference-free effect is robust: the negative turnout effect would remain even with substantial differential registration between the treated and control groups.