Measuring Perceptions of School Lockdown Drills: Development and Validation of the Lockdown Impact Scale for Caregivers (LIS-C)

Abstract

Understanding the extent to which caregivers perceive active shooter lockdown drills in K–12 schools as physically and emotionally safe for their children is pertinent, yet little empirical evidence exists. Negative parental perceptions of drills can lead to decreased student participation and compliance and amplify public concerns associated with school safety. This article introduces the Lockdown Impact Scale for Caregivers (LIS-C), a novel instrument developed to assess caregiver perceptions of K–12 school lockdown drills. Utilizing a national sample of 999 caregivers, this study validates the LIS-C through exploratory and confirmatory factor analyses, revealing a three-dimensional construct that includes perceived impact, school safety, and expressed concern. The development and validation of the LIS-C provide school-based mental health professionals with a psychometrically sound tool to address caregiver concerns, paving the way for a more informed and intentional implementation of lockdown drills. This research underscores the pertinence of examining caregiver perceptions to foster a safer and more supportive school environment.

Since the mid-1990s, there has been a surge in K–12 school shootings within the United States, fundamentally altering the perception of schools as safe havens to potential sites of harm (Rich & Cox, 2018). In response, substantial changes to school safety protocols have been made, including the implementation of lockdown drills in over 95 percent of public schools (Rich & Cox, 2018). Involving approximately 4.1 million students annually (Musu-Gillette et al., 2018; Sacks & Masiakos, 2020), these drills are designed to prepare students and staff for high-threat scenarios, such as an active shooter. They generally involve practicing a schoolwide lockdown, during which individuals secure themselves in safe locations, close doors and windows, and await confirmation from authorities that the threat has subsided (Schildkraut & Nickerson, 2022).

The notable rise in schools engaging in lockdown drills has sparked national debates on the necessity and safety of these drills (Bonanno et al., 2021). Critiques cite drills as “emotionally traumatic” (Rygg, 2015), while others claim lockdown drills promote feelings of safety and security in students and teachers (Reingle Gonzalez et al., 2016). Additionally, there is a belief that lockdown drills may contribute to heightened public anxieties about school safety, serving as a stark reminder of the risks of violence in educational settings (Worthington et al., 2021). However, despite the increased frequency and concerns associated with lockdown drills, little is known about how K–12 caregivers, inclusive of both parents and legal guardians, perceive these drills in relation to their child’s safety and well-being. While researchers have minimally explored students’ perceptions of lockdown drills (e.g., Bonanno et al., 2021; Huskey & Connell, 2021), research on caregivers’ perceptions is limited, and a validated instrument measuring such perceptions does not exist. As a result, school-based mental health workers are attempting to respond to increasing caregiver concerns without a solid research base (Goodman-Scott & Eckhoff, 2020; Wallace, 2020).

Current Study

In this article, we describe the development and initial validation of the 16-item Lockdown Impact Scale for Caregivers (LIS-C), a brief, internally consistent measure of caregivers’ perceptions of the impact of lockdown drills on their child’s safety and well-being. Based on extant literature on student and parent perceptions of lockdown, we generated an initial item pool. We then conducted exploratory and confirmatory factor analysis in two subsamples of caregivers to provide preliminary evidence of scale reliability and validity. Our purpose was to design a brief, psychometrically sound scale for schools to assess the extent to which caregivers perceive active shooter lockdown drills as physically and emotionally safe for their children and whether these concerns, if any, are externally communicated. “Caregiver” is utilized as an omnibus term inclusive of all individuals who serve as the primary care provider for a child currently enrolled in grades K–12 in a public or private U.S. school. For parents of multiple children, responses for the eldest currently enrolled child were given.

Caregiver Perceptions of Lockdown Drills

Research on the emotional impact of lockdown drills on students and teachers has generated varied and contentious findings. While some studies suggest that drills elicit fear, anxiety, and traumatic stress in students (Bonanno et al., 2021), others claim that well-executed drills reduce anxiety and enhance feelings of safety and preparedness (Huskey & Connell, 2021). Nickerson and Schildkraut (2021) report lower anxiety and improved well-being in high school students after well-coordinated drills, suggesting that some drills may empower students with improved emergency response skills. Others report an inverse relationship between security measures and perceived safety (Booren et al., 2011; Perumean-Chaney & Sutton, 2013). These mixed findings point to the complexity of school safety and preparedness, underscoring the need for a more comprehensive understanding of drill impact on school communities. This necessity leads directly to the significant, yet overlooked, influence of caregivers on student perceptions and reactions to school safety protocols.

The significance of engaging caregivers in students’ academic lives and the crucial role of caregivers in shaping students’ perceptions of school climate are well established (Schueler et al., 2014). Caregivers’ attitudes toward and connectedness with school can influence their child’s ability to tolerate educational stressors and obstacles (Deng et al., 2022). In the context of school safety, while caregivers acknowledge the significance of lockdown drills for safety and readiness, a growing trend reveals heightened parental concerns and communication of these concerns to children and staff (Goodman-Scott & Eckhoff, 2020). It follows that negative caregiver beliefs around drill utility and safety influence students’ involvement and response.

Inconclusive findings on the impact of lockdown drills on student well-being, alongside caregivers’ influential role in shaping student perceptions, highlight the complex nature of school safety. The challenge, best explained by stress crossover theory, is rooted in the direct experiences of students and how these are interpreted and communicated within families.

Stress Crossover Theory

Guided by family systems theory, stress crossover theory posits that caregivers’ and children’s emotional experiences are interconnected and influential (DePasquale, 2020). It emphasizes the impact of interpersonal relationships and social contexts on stress and emotional well-being and that socioemotional support must dually target individuals and their broader social system (Dotterer et al., 2021). In this context, it is unsurprising that negative parental views on drills correlate with reduced student participation and compliance, and reluctance among administrators to communicate lockdown procedures to families (Sacks & Masiakos, 2020). It follows that caregiver concerns, whether intentionally or unintentionally conveyed, may impact how children perceive school lockdowns and potentially influence drill effectiveness (Wallace, 2020). For instance, a caregiver’s expressed anxiety about school safety can inadvertently heighten student fear during a drill. Conversely, constructive communication by caregivers about a drill’s purpose can foster a sense of security and preparedness.

Recognizing caregivers’ internalized conflict of safeguarding their children while also being concerned about potential emotional impacts, national guidelines highlight the importance of schools maintaining clear and consistent communication with families about lockdown drills (National Association of School Psychologists [NASP] et al., 2021; Schildkraut & Nickerson, 2022). Despite calls to better understand the “disconnect between school actions, parent perceptions and student perceptions related to active-shooter prevention” (Wallace, 2020, p. 156), there is a lack of well-validated scales for assessing caregiver perceptions of lockdown drills. Such deficits hinder a comprehensive understanding of caregiver concerns and communication patterns between caregivers and children. Developing such a scale could enable school-based mental health workers with a practical tool to enhance family–school connectedness, lockdown notifications and emergency response communications, and advocacy regarding developmentally and culturally inclusive crisis response protocols.

To bridge the gap between theoretical understanding and practical application, the LIS-C is developed to measure caregiver perceptions of school safety, emotional impact, and expressed concern. Through the lens of stress crossover theory, the LIS-C is not only a measurement tool, but also an intervention guide aimed at reducing negative transference from caregivers to children and foster a safe and supportive educational environment.

Development of the LIS-C Items and Expert Reviews

The development of the LIS-C was grounded in Lambie et al.’s (2017) steps for establishing content-oriented evidence for counseling assessments. First, we conducted a thorough deductive literature analysis, reviewing information pertinent to (a) best practice guidelines and federal reports (e.g., DeVos et al., 2018; NASP et al., 2021); (b) school safety and school climate research (e.g., Booren et al., 2011; Schueler et al., 2014); and (c) literature on the emotional impact and current perceptions of school lockdown drills (e.g., Bonanno et al., 2021; Wallace, 2020). We then developed a conceptualization framework of three core domains necessary to evaluate caregiver perceptions: (1) school safety and preparedness, (2) perceived impact on well-being, and (3) crossover communication. The first two domains intentionally reflect the binary opposition found within the literature, which is that lockdown drills are necessary to keep students safe but may have psychological consequences. The third demonstrates external expression of these concerns.

An initial pool of 32 items was created based on these domains following Kline’s (2005) nine guidelines for item development. This first domain (eight items) included items that prompt caregivers to reflect on the utility of lockdown drills in supporting a sense of safety at school for the caregiver and child. The second domain (16 items) addressed the emotional impact of lockdown drills frequently expressed by caregivers, students, and teachers in the literature. The third domain (eight items) included items associated with caregivers’ comfort in discussing drills with their child and whether they had expressed lockdown concerns to others. We elected to use a five-point Likert-type levels-of-agreement scale ranging from 1 = strongly disagree to 5 = strongly agree.

To evaluate the items and provide evidence of content validity, we recruited eight experts to review and provide feedback on the 32 items and their respective domains. Experts were provided with a description of each construct and theorized domain and rated the relevance of each item using a three-point scale (ranging from 1 = excellent to 3 = poor). Expertise was determined by having related scholarship or expertise in school mental health, school safety, K–12 education, scale development, or having a child in a K–12 school that routinely conducts lockdown drills.

Following three rounds of reviews, our expert panel recommended removing 15 items and combining four into one statement, noting duplicate and poorly worded items (e.g., “Lockdown drills make my child feel worried”) and items falling outside the scope of the expressed domains (e.g., “presence of visible security measures within the school”). Item pretesting for the remaining 18 items used a sample of 24 graduate-level counseling students who had a child currently enrolled in a K–12 school or were employed as teachers. Pretest feedback on item wording, survey structure, and completion time resulted in two items being reworded and one item deemed repetitive and potentially biased removed. Instructions were also reworded to clarify that respondents should answer based on their oldest child. The three domains and five-point Likert scale, with higher scores indicating greater impact, were maintained.

Method

Using a randomly split subsample, we used exploratory factor analysis (EFA) to explore the initial factor structure and confirmatory factor analysis (CFA) to cross-validate the factor structure and assess the psychometric properties of the scale. The three following questions guided our analysis: (1) does the LIS-C factor structure correspond to the theoretical domains generated from current research? (2) Do LIS-C scores demonstrate strong internal consistency? (3) Is the emergent factor structure of the LIS-C confirmed?

Sampling and Data Collection

Following institutional review board approval (#1871323-1), participants with at least one child currently enrolled in grades K–12 in a public or private U.S. school that currently engages in at least one lockdown drill during the academic year were recruited via a Qualtrics research panel for 14 days in April 2022. Quota sampling, a nonprobability sampling method that allows for strata within the sample to be proportional to the study population (Brabham, 2013), ensured the same was demographically comparable to the 2020 U.S. census distributions (± 10 percent) for gender, age, race/ethnicity, and income. See Table 1 for participant characteristics.

Data Analysis

The data were divided into an EFA dataset (N = 499) and a CFA dataset (N = 500), meeting the minimum sample size requirement of 300 for EFA. The Kaiser–Meyer–Olkin statistic of 0.893 and Bartlett’s test of sphericity, χ² (136) = 5,155.635, p < .001, indicated the data were suitable for factor analysis. We used SPSS version 28.0 for Mac to examine the underlying factor structure of the LIS-C. We employed principal axis factoring and direct oblimin rotation with Kaiser normalization due to the potential for correlated factors. The number of factors extracted was determined through an examination of eigenvalues, analysis of the scree plot, and consideration of the factors’ theoretical alignment.

Results

The EFA initially revealed a four-factor structure from the 17-item scale, with eigenvalues exceeding 1.0 for each factor (6.659, 2.903, 1.233, and 1.079, respectively), cumulatively accounting for 69.8 percent of the variance. However, the fourth factor, comprising only two items, raised concerns about its reliability and interpretability within the scale’s conceptual framework. Furthermore, item 12 (“I have never been concerned about my child participating in a lockdown drill at school”) failed to load significantly on any factor, suggesting a misalignment with the underlying structure intended by the LIS-C.

In response to these issues, and aiming for a more coherent factor solution, we excluded item 12 from further analysis. This adjustment led to a reevaluation of the factor structure, resulting in a refined three-factor solution. The removal of item 12 and the reanalysis significantly clarified the scale’s factor structure, with revised eigenvalues (6.569, 2.879, and 1.194) accounting for 66.52 percent of the variance. The scree plot analysis further supported this three-factor solution. During this process, we noted that item 10 (“My child believes they are in real danger during a lockdown drill”) exhibited cross-loadings on Factors 1 and 3, with respective loadings of .531 and –.323. Following established guidelines (Watson, 2017), we retained this item in Factor 1, as the loading difference exceeded the .2 threshold, indicating a stronger association with this factor. See Table 2 for the final three-factor solution.

The final 16-item LIS-C yielded a three-factor solution that accounts for 65.52 percent of the total variance. The three subscales include perceived impact, school safety, and expressed concern. The first factor, perceived impact, accounts for 41.05 percent of the variance and consists of eight items that assess the psychological impact of lockdown drills on caregivers and children. The second, school safety, accounts for 18.00 percent of the variance and consists of five items that assess caregivers’ perception of lockdown protocols in keeping their child safe. The third, expressed concern, accounts for 7.47 percent of the variance and consists of three items that assess caregivers’ communication with others regarding concerns about lockdown drills in their child’s school.

To assess the psychometric properties and confirm the factor structure, we conducted a CFA using the Lavaan package in R. Our dataset (N = 500) met the minimum sample size of 200 for CFA (Kline, 2005). To assess model fit we used chi-square/df, Tucker–Lewis Index (TLI), comparative fit index (CFI), standardized root mean square residual (SRMR), and root mean square error of approximation (RMSEA). The CFA model included the three subscales of the LIS-C as separate factors, specified to test the three-factor model with all items loading on their respective factors. The model was estimated using the diagonally weighted least squares estimator because it is a more robust estimator with ordinal data. Model fit indices were: [χ²(101, N = 500) = 232.337, χ²/df = 2.30; p < .001], CFI = .978, TLI = .974, RMSEA = .051, and SRMR = .068. Factor loadings for all items were above .30 and significant at p < .001, indicating an adequate model fit (see Figure 1). While these results suggest that the LIS-C subscales are separate and distinct, we also used Cronbach’s alpha to test the internal consistency of the full scale and subscales. The LIS-C demonstrated strong internal consistency with α = .872 for the full scale, α = .898 for school safety, α = .843 for perceived impact, and α = .891 for expressed concern. Table 3 includes the final items and standardized factor loadings.

Discussion

The LIS-C factors are consistent with the literature and associated debates, which recognize lockdown drills as a key component of school safety (Factor 2) despite uncertainties associated with the emotional impact on students (Factor 1). While drills have been directly associated with increased perceptions of safety (Huskey & Connell, 2021), other findings indicate drills may lead to a range of adverse mental health outcomes (Bonanno et al., 2021). Consistent with stress crossover, externalized communication was a separate factor (Factor 3).

Notable modifications to how we originally conceptualized the LIS-C were observed. We initially theorized that items associated with the utility of lockdown drills (items 1–6) would cluster under one domain. Likewise, items within the dimensions of emotional impact and parent communications would cluster under direct impact (items 7–13) and crossover communication (items 14–17), respectively. However, discussions with children regarding the intent of lockdown drills (item 14: “I feel comfortable talking with my child about lockdown drills and their intent”) loaded on Factor 2, not Factor 3, and two items associated with drills being a waste of time (item 5: “I think school lockdown drills are a waste of time” and item 6: “My child thinks school lockdown drills are a waste of time”) clustered under Factor 1 rather than Factor 2.

Emergency response drills are based on the idea that practice reinforces what individuals should do in an actual emergency, even if impaired by stress or fear (Schildkraut & Nickerson, 2022). Drills also identify weaknesses in emergency plans and facilitate improvements in school preparedness (DeVos et al., 2018). Thus, all items on Factor 2, including caregivers’ comfort level in discussing drills and their purpose (item 14), are associated with school safety. Similarly, items associated with drills being a waste of time (items 5 and 6) were initially theorized under Factor 2 based on work by Zhe and Nickerson (2007) and Bonanno et al. (2021). However, recent literature reflects that school communities believe broader and more comprehensive safety reforms are needed. While adolescents agree that drills are important, they also feel drills “do nothing to prevent a shooting in the first place” (Moore-Petinak et al., 2020, p. 512). It follows that items associated with the futility of drills would load on Factor 1.

These findings underscore the complexity of caregiver perceptions toward lockdown drills, highlighting the balance between perceived safety benefits and potential emotional impacts on students. This also provides a unique contribution to academic discourse on school safety by offering a pragmatic approach to enhance mental health support in educational contexts. In addition to providing a practical, no-fee instrument for school-based mental health professionals, our results support a more concrete understanding of the extent and nature of caregiver perceptions of lockdown drills in relation to their child’s safety and well-being. The intent is to empower schools to gain a more nuanced understanding of caregiver concerns, to facilitate informed discussions and interventions around school safety.

Applications to Practice

School-based mental health workers play a vital role in creating a supportive and resilient school environment that prioritizes the well-being of students during lockdown drills. Concurrent with Goodman-Scott and Eckhoff, (2020), the development of the LIS-C was intended to support school-based mental health professionals in their “work to ensure that lockdown drills provide more benefit than harm to their school community, including students, school staff, and caregivers/families” (p. 444). Being able to quantify parental support or concern is vital to this process because caregivers influence how students engage with and respond during these exercises, as are recommendations that crisis response protocols are a collaborative communication effort that engages caregivers in all planning phases (Dotterer et al., 2021; Elbedour et al., 2020).

Administration of the LIS-C, whether part of a school climate assessment, a targeted assessment for concerned caregivers, or in response to a recent event, allows for more informed policies that address the whole school community and enhance two-way communication. This information can also be used to better inform crisis response planning for students with a history of trauma, neurodevelopmental or physical disabilities, or other mental health concerns. For example, higher LIS-C scores for caregivers of students with disabilities can prompt information gathering regarding the nature and extent of caregiver concerns. By practicing developmentally and ethically appropriate lockdown drills, schools reduce the likelihood of adverse emotional responses and may even increase response effectiveness (Schildkraut & Nickerson, 2022).

The LIS-C, while focused on lockdown drills, can also contribute to a larger understanding of caregiver views on school safety and preparedness, shedding light on the effectiveness of safety procedures within educational settings. This insight can guide schools in structuring lockdown drills to address specific concerns raised by parents, therefore improving communication and education about these drills. Enhanced understanding supports increased family engagement and trust and provides avenues for caregivers to offer valuable perspectives on optimizing practices. Finally, using the LIS-C better positions school-based mental health workers to address caregiver concerns, support age-appropriate drills, and work with staff to ensure drills are conducted in a culturally responsive manner with adverse effects minimized.

Limitations and Future Research Directions

Study limitations include the use of cross-sectional data, potential response bias from participant self-selection, infeasibility of concurrent validity testing due to the absence of directly comparable scales, and constraints in item generation stemming from limited literature on caregivers' perceptions of lockdown drills, possibly resulting in incomplete coverage of certain aspects and associated concerns. As research begins to expand around lockdown drills, the scale should be revised to capture a broader range of perceptions and experiences, including the influence of family engagement and trust on caregivers’ perceptions of lockdown drills. Furthermore, sociocultural factors were not extensively examined. Future research should address how historically vulnerable and minority populations experience lockdown drills and the perceptions of racially and culturally diverse caregivers. Such work would support more culturally relevant and trauma-informed practices that are essential in addressing the diverse needs of the whole school community.

In addition to these considerations, a revised scale for teachers is strongly recommended. We acknowledge the impact of increased security measures in schools on teacher well-being and perceptions of school safety. Such a scale could assess teachers’ perceptions and the emotional impact of lockdown drills, extending the caregiver-focused approach of the LIS-C to encompass the broader school community. By including teachers’ perspectives, future research could offer a more comprehensive understanding of the effects of lockdown drills, thereby facilitating a more holistic approach to enhancing school safety.

Funding for this study was granted by the College of Education and Human Development, George Mason University, Fairfax, VA, USA.

References