Language tester identity in policy contexts: A narrative analytical approach

Abstract

Increasingly scholars are calling for language test developers and researchers to intervene in policy due to the recognized socio-political use of language tests and the potential ethical issues at stake when tests are used to enforce government policies. Research has shown that policy spaces may be highly demarcated with diverse expectations, stereotypes, and power dynamics. Language experts are just one voice amid an array of actors and interests, yet their perspective can aid in guaranteeing fair and just testing processes. However, insufficient research has examined how language testers actually use their expertise in policy. We sought to explore this novel research area using narrative analysis to examine the identity of language testers. Identity navigation framework and indexicality guided our analysis of four narratives told by language testers of their interactions with policymakers. The findings demonstrate four modes of engagement in relation to decision-making processes—Objective Expert, Critical Activist, Problem-solving Partner, and Issue Advocate. Different roles can complement each other and can contribute to the value of language expertise in policy settings.

Introduction

Within applied linguistics, language testing is a policy-centred field. Test and test scores are used by professional, academic, and governmental authorities to regulate, among other things, the education, employment, and movement of people around the globe. In recent decades, language testing policies have been on the rise due to an increase in test use in immigration and integration policies (van Oers, Ersbøll and Kostakopoulou 2010; Goodman 2014; Rocca, Deygers and Carlsen 2020). Governments and designated language testing institutions collaborate to fulfil policy edicts requiring language proficiency in national languages for migration (Saville 2009). A study conducted in twenty European countries found that many language testers had daily or weekly contact with policymakers regarding test use in migration policies (Schildt, Deygers and Weideman 2024). Research has shown that language testing professionals may be situated, both discursively and in practice, in positions that require them to enforce ‘securitization’ processes of the state (Harding, Brunfaut and Unger 2020). Scholars have highlighted the critical need for language experts and policymakers to work together to ensure that decisions made based on language test scores are fair and fit to the immigration purpose (McNamara and Shohamy 2009).

Despite an at times high level of coordinated action, research on language expertise in the policy context is scarce (Arias, Vessey and Sheyholislami 2022). The interactions between language testers and policymakers have not been documented systematically, meetings and documents often remain closed or confidential (Knoch 2021), and there is a lack of theory to understand and explain the expert-policy relationship. We define policymakers as individuals who regularly seek to influence policy on a given issue or who control the venue wherein policy decisions occur (Weible 2018). In policy settings, language experts enter a discursive space where they must newly negotiate their expert role amidst diverse expectations, stereotypes, and power dynamics (Lo Bianco 2021). Against the backdrop of the increasing use of language tests in policy environments and considering a great deal hangs in the balance for test takers who take migration language tests, the identities of language testers in the policy context are particularly salient. Language experts are only one voice among the collection of stakeholders involved, yet their voice can lend authenticity and prominence to debates about language policies (Bianco 2010) as well as rigor and grounding in professional standards (Harding, Brunfaut and Unger 2020).

In this article, we use the concept of identity as a window to explore how language testers navigate their role within policy contexts. We also examine how applied linguistic expertise is valued and legitimized. Identity is understood as how individuals are positioned and position themselves in speech and actions (Davies and Harré 1990). Identities are formed in and through group membership (Wenger 1998), where common knowledge and goals are shared (Tajfel and Turner 1986). A growing body of research has examined the professional identities of language teachers (Varghese et al. 2005, 2016; Kanno and Stuart 2011; De Costa and Norton 2017). Studies have explored teacher identities in micro-level classrooms (Kayi-Aydar 2015; Derakhshan, Moradi and Nazari 2024) and at the macro level, examining the contextual and structural factors that shape language teacher agency and identity (Priestley, Biesta and Robinson 2015; Kang 2024). Language tester identities, however, may be distinct from language teachers and, particularly in the policy context, have not been explored.

To address this gap in the literature, the present article uses narrative analysis to examine four stories told by language testers of their interactions with policymakers. Individuals claim and express their identities in and through discourse (Johnstone 2018). Language is a tool that can signal similarity to and difference from others. This study’s methodological framework draws on the notion of identity navigation (Bamberg 2011a). Identity navigation refers to the ways individuals manage and adapt their identities in response to different social or professional environments by analysing positioning on three continua: difference/sameness, agency/control, and constancy/change. This article contributes to theoretical advancements in three fundamental ways. Firstly, language testers possess unique professional expertise and experience in language assessment. Studying the identities of language testers can help develop the concept of ‘language tester expertise’ and shed light on its value and legitimacy in the policy context. Secondly, the premise underlying this article is that language testers are key stakeholders in policy contexts, not only as executers of political agendas but also as advisors and consultants in policymaking. Examining language testers’ identities positions them as active agents within wider socio-political contexts where their professional voices matter. Finally, this study has important theoretical implications for conceptualizing the expert-policy relationship and factors that might help or hinder this interface in everyday settings.

Literature review

Research on language testing expertise in policy

The socio-political character of language testing has long been recognized in scholarly literature (Shohamy 2001; McNamara and Shohamy 2008, 2009), and much research has addressed the potential ethical issues of language test use to enforce government policies (Kunnan 2009; Carlsen and Rocca 2021; Khan 2022). Theoretical discussions have focussed on fairness and justice rooted in concern for the consequences of test use (Messick 1989). For example, McNamara and Ryan (2011) proposed a distinction between fairness, referring to test quality (is the test psychometrically defensible?), and justice, referring to test use (should the test be used in the first place?). They deemed fairness considerations alone to be too narrow; hence factoring in justice aspects presented a broader view of test validity. Based on this distinction, scholars have debated the value and legitimacy of language testing expertise in the context of policy-driven tests (McNamara 2006; Carlsen and Rocca 2022). Policy can be understood as the deliberate decisions—actions and nonactions—of a government or equivalent authority toward specific objectives (Weible and Carter 2017), and hence policy contexts are settings around which interactions involving policy happen.

Language testers can intervene in policy in three primary ways: (1) at the macro level of policy formation, (2) at the meso level of policy interpretation and/or evaluating the outcomes of a particular policy, and (3) at the micro level of policy enactment developing or administering tests for the implementation of policy (Lo Bianco and Aliani 2013; Elder 2021). While interventions at any level constitute a form of policymaking (Menken and García 2010; Lo Bianco 2021), language tester expertise may be most commonly applied in the micro-implementation phase of policymaking (Schildt, Deygers and Weideman 2024). Increasingly, however, scholars call for language testers to intervene at the macro level in the early stages of policy formation (Harding, Brunfaut and Unger 2020; Elder 2021; Carlsen and Rocca 2022). Research has shown that policymakers often have misconceptions about language testing and may lack even basic knowledge about language levels referred to in policy documents (Pill and Harding 2013). Nevertheless, policymakers’ beliefs about language assessment can have a direct impact on policy since they have considerable power to make and enforce decisions. Arguably, test developers know best what their tests can measure and what language levels are appropriate for different groups of candidates; thus, they are in an ideal position to engage with policymakers and contribute to public debate.

However, expecting language testers to become actively involved in policymaking processes without adequate training may be unrealistic (McNamara 2006). Scholars have pointed out that formal training for language testers does not adequately equip them to navigate the interface between policy and language testing (McNamara 2009). Lo Bianco (2001, 2014, 2019, 2021) has repeatedly stressed that academics must develop policy literacy, that is, an understanding of the policy context and power relations. Recent calls to action in compilations such as The Routledge Handbook of Language Testing (Harding and Fulcher 2021) recommend more policy literacy; however, they offer limited guidance for developing such competency. Language experts, thus, have few opportunities to prepare themselves for collaboration in the complex and ever-shifting policy environment.

As stated above, in this article we use identity as a lens for examining how language testers navigate their role in policy contexts. Although identity was once seen as an individual, innate sense of self (Mead 1934; Erikson 1968), it is now understood as a collectively formed understanding of self (Tajfel and Turner 1986) resulting from habitus, the context in which we live and work (Bourdieu 1991). Language tester identities may therefore be heavily reliant on the continual ‘discussing, critiquing, and questioning’ among assessment experts who form a ‘knowledge community’ (Inbar-Lourie 2008) and thus refine both individual and socially shared representations. Some scholars focus on a bottom-up approach to explore how individuals agentively guide the process of identity development by, for example, using language to position themselves (Ochs 1993). Others focus on a top-down perspective to examine the environmental factors, for example, political structures, that may shape agency in identity navigation (Gee 2015a). From this perspective, social and political conditions strongly influence how an individual’s identity is constructed (Davies and Harré 1990).

Identity is constructed within complex socio-political spaces involving positioning by the ‘other’ (Pennycook 2004) and hence power relations play a critical role in shaping the extent to which individuals can negotiate their professional identities (Fairclough 1992). This means that the agency of language testers is not only shaped by their professional expertise and community but also by the broader power structures that influence policymaking. Researchers at the Language Testing Research Centre (LTRC) of the University of Melbourne reported on their ongoing relationship with policymakers in the Australian policy context (Elder 2021; Knoch 2021). They underscored that power imbalances, unclear communication channels, and competing agendas mediated their relationship. This reflects research showing that language experts’ ability to provide policy advice is impacted by external factors (Lo Bianco 2021) and that positionality is a major determinant of engaging and having impact in policymaking (Blommaert 1999; Gee 2015b). Therefore, it is important to examine to what extent language testers have autonomy to exercise professional expertise. For example, research using discourse analysis examined official documentation of language testing for migration and found that language testers are positioned, both discursively and in practice, as ‘compliant enforcers’ (Harding, Brunfaut and Unger 2020, p. 682). The authors argue that language testers are instruments of the state securing the borders from the threat of immigration and that language testers should resist this positioning through advocacy at the local/national level.

Therefore, as governmental policies increasingly make use of language testing, scholars have called for language testers to increase their participation in the political sphere (Harding, Brunfaut and Unger 2020; Carlsen and Rocca 2022). Carlsen and Rocca (2022), for example, encouraged language testers to be language test activists based on the concept of intellectual activism (Contu 2018) and language activism (Florey, Penfield and Tucker 2009; Flores and Chaparro 2018). Elder (2021) called for language testers to be policy responsible professionals who view testing as part of a policy web and recognize a duty to assist decision-makers in considering the broader implications of testing activity. Even so, the question of whether or how language testers actually take up activist professional identities has, to date, not been explored. Therefore, it is not well-understood how language testing expertise is constructed and practiced in policy contexts. Professional identities have been extensively studied in the field of applied linguistics concerning teacher identities in traditional language classrooms and programs (Varghese et al. 2005; De Costa and Norton 2017), yet no research to date has explored the identities of language testers in the policy world.

Research questions

The literature review underscored that there is room for a clearer understanding of the nature of language testers’ roles in the policy domain. There have been many calls for language testers to intervene in policy contexts, but few attempts to conceptualize the nature and value of language testing expertise in these spaces. We seek to address this research gap by using narrative analysis to examine how language testing expertise is enacted in policy settings and how language testers construct and negotiate their professional identities. Thus, the research questions for this study are:

1. How is language testing expertise understood and enacted in policy settings?

2. How do language testers negotiate their professional identities in the policy context?

Question one develops the notion of language testing expertise and its value and legitimacy in the policy sphere. Question two addresses the factors that help and hinder language testers in navigating their identities in everyday practice. By integrating these two questions in a holistic analysis we hope to contribute to understanding of how language expertise is constructed, narrated, and performed in policy spaces.

Methodology

Framework of analysis

In this study, we have chosen to analyse narratives. In line with Norrick (2000) and Johnstone (2018), we define narratives as stories consisting of two or more clauses with a temporal progression. In other words, narratives imply the ordering of characters over space and time. Narratives have been shown to function well in their discourse surroundings as a tool for navigating the who-are-you and who-am-I questions of identity (Bamberg 2011a) and for targeting how narrators position themselves and others. To uncover such positionings our analysis was guided by two analytical frameworks, indexicality and identity navigation. The first, indexicality, refers to linguistic forms’ potential to index ‘ideologies, social representations about group membership, [and] social roles’ (De Fina, Schiffrin and Bamberg 2006, p. 4). For example, deictics (e.g. inclusive versus exclusive we-forms) and jargon may function as indexical markers by pointing to certain aspects of socially shared conceptualizations.

The second framework for this study, identity navigation (Bamberg 2011a), addresses how individuals are positioned and position themselves along three continua: difference/sameness, agency/control, and constancy/change. For the purposes of our analysis, we used only the first two continua. The first dimension looks at how individuals align themselves with and/or differentiate themselves from others. The latter dimension relates to how individuals choose linguistic devices that construct the actors in their narrative as more or less influential, powerful, and responsible (Bamberg 2011b). For example, the first-person verb tense is a marker of high agency, which positions the person as the subject, the ‘doer’ constructing the world. In contrast, the passive tense is a low agency marker, positioning the person as the ‘undergoer’ on the receiving end of actions (Bamberg, De Fina and Schiffrin 2011). Therefore, this framework allowed us to analyse how language testers construct their identities in policy contexts and how their professional expertise is positioned in relation to other factors.

Based on these frameworks, we examined how language tester identity and expertise is negotiated in the policy context. Arguably, these four stories of interactions with policymakers may not fully represent the range of experiences language testing professionals have in the policy arena. Nevertheless, we attempted to expand the possibility of comparability across research contexts and the relevance of the results to theory-building by utilizing theoretical sampling based on our analytical frameworks and transparency throughout the research process, including a clear description of the methods and our own positionality (Stake 1995). It is important to emphasize that this article does not document ‘real’ identity performances (e.g. actual audio recordings of interactions between language testers and policymakers), which are difficult to document due to informal or closed meetings and confidentiality (Knoch 2021). We analyse narratives where language testers reflect on and describe such interactions. It is also important to bear in mind that interviews can be characterized by a great deal of reflexivity (Linde 1993) and the identities they contain are versions of a reflected upon and rehearsed self (Georgakopoulou 2006). Interviewees may convey the identities they want to construct for themselves or the identities they think the interviewers expect of them (Van De Mieroop 2011). These reflected upon narratives may not reveal the ‘true’ identities of language testers in policy contexts. We can only draw tentative conclusions about the locally constructed identities of language testers in the specific interactions with policymakers in the narratives under scrutiny.

Participants

The narratives we analysed formed part of a corpus of twenty-eight semi-structured interviews with language testers (Schildt, Deygers and Weideman 2024). We conducted the interviews from October 2021 to March 2022 with professional language testers working in twenty European countries, including Austria, Belgium, Czech Republic, Denmark, Germany, France, Finland, Estonia, Iceland, Italy, Luxemburg, the Netherlands, Spain, Slovenia, Portugal, Poland, Norway, Romania, Switzerland, and the UK. Around half of the twenty-eight participants worked for universities, non-profits, or private companies. The other half worked for governmental or semi-governmental (controlled or financed by the government) institutions. In most cases, the language testers interacted with policymakers who were civil servants, that is, bureaucrats charged with the interpretation or implementation of policy. In some cases, language testers interacted with government officials or ministers involved in policy formation.

Since researcher identities and worldviews impact research decisions and collaborations (De Costa and Norton 2017; King 2024), we see ourselves also as participants in this study. The third author is an established researcher in the field of language testing and has connections that facilitated the interviews that were conducted. This author also has first-hand experience developing and validating tests for government use and interacting with policymakers in the national policy context, experience which was critical in the development of the research objectives. The first author, a doctoral researcher, had previous experience working for language testing institutions. This author interviewed the participants and maintained contact with them, including meeting many of them at conferences and events. Our insider/outsider status as researchers/language testers inspired us to engage with this under-researched topic and seek to understand experiences in policy contexts. The second author’s focus on the discursive analysis of identity guided the methodological choices and the analysis of narratives in this research.

Narrative selection

To select the narratives under scrutiny, we searched the dataset of twenty-eight interviews for all the stories told about interactions between language testers and policymakers. The interview guide for the semi-structured interviews centred on the participants’ interactions with policymakers. An initial search of the data uncovered thirty-eight narratives. As an in-depth analysis of all thirty-eight was not feasible, we used exploratory analysis to position the narrators on the identity navigation framework (Bamberg 2011a) dimensions of difference/sameness and agency/control (Fig. 1). Following this, we chose four narratives that represented emblematic positionings of language testers vis-à-vis policymakers (pseudonyms Christine, Robert, Natalie, and Emily).

The selected fragments came from interviews that were conducted online in English and were video recorded. The participants spoke under the condition of confidentiality and thus specific details about their regional and institutional contexts cannot be provided. The four participants were assigned English pseudonyms, and the identifying place or institution names were removed to respect the anonymity of the participants. Although they were from different countries and institutional settings, all had been language teachers before moving into more senior positions as language testers. Table 1 describes the selected narrators’ pseudonym, the institutional sector where they worked and the narratives length and location in the interview. The narratives under analysis ranged in length from approximately one and a half minutes to two and a half minutes. In preparation for the analysis, each of the four narratives was transcribed using a simplified version of Jefferson’s (2004) transcription conventions, extended by Mondada’s (2016) multimodal transcription conventions to represent relevant embodied semiotic resources (outlined in Appendix  1).

Analysis

Our analysis consisted of watching and listening to the narratives repeatedly to identify linguistic, prosodic, and embodied variation in the narrators’ speech. First, using the analytical concept of indexicality, we took into account a wide variety of discursive elements such as boosters, hedges, lexical choices. Second, based on the identity navigation framework, we considered positionality through the use of particular pronouns and verb tenses. Third, based on insights from conversation analysis (Sacks, Schegloff and Jefferson 1974), we looked at sequential features of how the narratives emerged in the interview interaction. Finally, we relied on multimodal resources such as gestures, eye-gaze, and facial expressions to shed light on the way identities were embodied and enacted in the discourse (Mondada 2016). We analysed the extracts iteratively with the assistance of the ELAN annotation tool for video and audio recordings (Wittenburg et al. 2006).

Results

In this section, we examine how Christine, Robert, Natalie, and Emily enact their language testing expertise and express their professional identities in the narratives they tell about interactions with policymakers. The narratives have been separated into two parts because of the length of the fragments and the way the narratives were told (e.g. first a hypothetical story which leads into a present-day narrative; for an overview of narrative genres, see Van De Mieroop 2021). The title of each section suggests the narrator’s role and the positioning of language expertise.

Christine: Critical activist—Language expertise is challenged

The first fragment we discuss comes from the interview with Christine. The narrative follows a turn by the interviewer who asked how Christine has gained the knowledge and skills required to work with policymakers. Christine initiated a story that recalled her reaction upon learning she would ‘have to cooperate’ with the government.

Excerpt 1a: Christine [C], Interviewer [I]

Christine starts by situating her story in the context of the time when she ‘heard’ about collaboration with the government. She states that this caused her sleepless nights (line 2), of which the frequency is underscored by the use of the extreme adjective ‘countless’ (Edwards 2000). This is also prosodically emphasized highlighting its emotional impact. Christine prefaces this declaration with the discourse marker ‘of course’ (line 1) presenting her feelings as self-evident to the interviewer. In line 4 she introduces both her language testing organization (pronoun ‘we’) and the government body (specific name removed for confidentiality). In this way Christine creates two distinct groups and affiliates herself with the former. By speaking of the government as a general body rather than of an individual person, she depersonalizes ‘them’ allowing her to further distance herself. Moreover, the verb choice ‘have to’ expresses that cooperation with the government goes against her will. Being in a position of obligated cooperation is especially evident from the explanation in lines 6–9 regarding the financial situation of the language testing organization. Christine’s rising intonation and eyebrow lift signal that the issue is emotionally loaded. This is expressed verbally by the re-iteration in line 10 of her insomnia and the ongoing nature of her feelings (‘still don’t feel comfortable’ in line 12). She positions herself—and her organization—as different from, yet highly dependent on the government body. She uses collective pronouns interspersed with her quite personal and emotional perspective. Therefore, from the very beginning of the story it is clear that Christine would not have chosen this collaboration, and it has had a strong emotional impact on her professional identity.

In the next excerpt Christine shifts from recalling a past moment to describing a present habitual situation in her ongoing relationship with the government. She references ‘B2’ which refers to the upper intermediate level of language proficiency on the Common European Framework of Reference for Languages (CEFR).

Excerpt 1b: Christine [C], Interviewer [I]

At the start of this excerpt, Christine again signals her lack of agency using the expression ‘when I have to write the tests’ (line 14). She then self-repairs to a collective perspective (‘when we have to’) thus adopting the organizational perspective but maintaining the passive identity. In the next line she specifically references her language testing expertise by prosodically and repeatedly emphasizing a knowledgeable self (‘I KNOW, I KNOW that this is not B2’) (Rieger 2003). In this way, Christine uses a first-person pronoun to claim a competent identity as a language tester. This statement is embodied by turning away from the camera with eyes closed and is followed by another extreme case formulation (‘at all’) in line 17 (Pomerantz 1986) both serving to underline the legitimacy of her knowledge of what constitutes B2 level.

In line 17, Christine uses a contrastive conjunction ‘but’ (Schiffrin 1987), which sets up the disagreement with the government institution over what constitutes a B2. The story continues from the collective perspective. Christine explains that ‘we’ attempt to lower the language level ‘as much as we can’ and in this way make small changes despite their obligated compliance. In lines 27–29, Christine reveals that these attempts are not successful when they receive the answer that ‘you have to make them harder’. The use of reported speech and gestures while enacting this moment in the narrative (up and down finger movement) seemingly gives the listener ‘direct access’ to the event (Stokoe and Edwards 2007) and reinforces its veracity (Schiffrin 2003). Presumably the exchange happened electronically ‘we send the items’ and ‘we receive the answer’ but use of reported speech envisions it as a live event stressing its strong impact. The reported interaction reinforces the misalignment between the two groups in the story. Language testing expertise, in this case, is being sharply challenged in the policy context. The debate is at the linguistic level over what constitutes a B2; however, the prosodic marking and emotional impact suggest a deeper disagreement. Christine rebels against the strict control. She uses her expertise (B2 level item writing) almost as a symbolic act of resistance.

Finally, in line 30 Christine again shifts to a personal perspective. She closes her eyes and shakes her head (cf. line 16) while sharing a reported thought (Haakana 2006) (‘I should just go and do something else’). She reveals that she has this thought ‘really often’ (line 36) underlining its frequency and hence its seriousness. The comment, prefaced by the use of ‘so’, acts as the story coda (Labov and Waletzky 1966) relating all the past story events to the present (Schiffrin 1987). With this, Christine summarizes the impact of the ongoing experience and reinforces that working with the government creates such inner conflict that she questions whether she should continue in her current position. In sum, Christine constructs a highly knowledgeable professional identity based on her language expertise. As she navigates the implementation of government policy, she portrays a stark ‘us vs. them’ divide. The wide chasm between the two opposing groups may be technical and/or ideological. Either way, language testing expertise is delegitimized leading to feelings of frustration. Christine resists the high level of control of the government institution and in this way attempts to reestablish the value and legitimacy of her language testing expertise.

Robert: Objective expert—Language expertise is neutral

A different story emerges from the second narrator Robert. The excerpt analysed here follows an exchange in which the interviewer asks Robert if he has ever talked with a politician or been to parliament. He responds saying no, that his role is ‘to inform but not debate’. The narrative that follows is told perhaps to exemplify this point.

Excerpt 2a: Robert [R], Interviewer [I]

In this excerpt, Robert tells a hypothetical narrative that begins with the phrase ‘let’s say’ and uses the modal verbs ‘would’ and ‘could’ (Norrick 2000, p. 131). The policymakers are constructed with a pronoun ‘they’ in line 1 but there is no corresponding term of reference to which this pronoun refers. ‘They’ are unnamed. Robert creates a second category, which he calls a ‘group of professionals’ (line 5) and he affiliates himself (and others) with this group (‘we could be a part of that group’). He repeats the category label ‘professionals’ with prosodical emphasis. By this he seems to be highlighting the professional––not political––nature of this group. In this way, Robert is clearly assigning language testing expertise in the policy context a professional role rather than a political one. The professional group is controlled by ‘political demands’ (line 11), which signals dependency on the interests and agendas of political elites. By using the term ‘political demands’, Robert indirectly refers to the group of policymakers from line 1. Referring to them in this depersonalized way further removes the policymakers from sight.

While the first part of the narrative related to a hypothetical situation, Robert goes on to describe an event that he presents as a personal experience. This reinforces the fluid and emergent nature of narratives (Van De Mieroop 2021), where one narrative genre can lead into another.

Excerpt 2b: Robert [R], Interviewer [I]

In the above narrative Robert narrated a recent event where language testers responded to ‘a political wish’ to design new tests. As in the first hypothetical situation, in this story the policymakers are not identified; they are referred to indirectly through their policy edicts (‘a political agreement’). Robert’s long pauses before ‘political’ in both lines 11 and 14 are accompanied by hesitation markers (em) and a reformulation (line 15) which mark these expressions as problematic to articulate. This perhaps points to his reluctance to name the policymakers or hints at a desire to remove the political aspect from his work. Both of these expressions (a political wish, a political demand) suggest a powerful authority and a clear line of command. However, in the lines that follow Robert attempts to carve out a space for language tester influence in the policy context. Robert explains the workplace language tests that the language testers were asked to design and concludes that it was ‘up to us’ (line 30). Nevertheless, he follows this assertion with the hedge ‘more or less’ (Prince, Frader and Bosk 1982), which deemphasizes responsibility. Although Robert claims that the language testers have ‘some influence’ (line 35), nevertheless, he switches the pronoun ‘we’ to the generic ‘you’. This subtle shift may function as an ‘externalizing or distancing device to diffuse the speaker’s personal accountability’ (Stirling and Manderson 2011). As such, it invokes a broader category of ‘people in general’ with whom to share the responsibility. In this way, Robert downplays the influence of language testing expertise in the policy context, which is seen to conform to strict guidelines.

The final five lines of the excerpt act as a coda with the discourse marker ‘so’ (line 33) for the resolution of the narrative (Schiffrin 1987). This discourse marker also serves to take the narrative out of the past and connect it to the present (Labov and Waletzky 1966) accompanied by a switch from past tense to present tense. The coda invokes Robert’s identification with a professional identity (‘we are professionals’) with the same prosodic emphasis used at the beginning of the story. He constructs an identity for the ingroup which is strongly dependent on their test developing expertise. This identity is impersonal and non-ideological, entitling him to speak authoritatively about the design of tests but not about the political realm. In conclusion, in this excerpt, the value of language testing expertise in the policy context rests on its technical, professional character, which is clearly differentiated from that of policymakers. There is no ‘I’ present in Robert’s narrative. Personal beliefs or values are absent and there is limited expression of individual agency. Robert refers to ‘a political wish’ or ‘political demands’, formulations which suggest a high level of control from the outgroup. He refrains from naming the outgroup explicitly and in so doing he distances himself from the decisions policymakers may take.

Natalie: Issue advocate—Language expertise is engaged

The narrator of the third story is Natalie. The story she tells comes after a turn by the interviewer who asked what factors contributed to policymakers taking up her recommendations. Natalie responded that this could be attributed to ‘personal things’ and told the following narrative about being consulted regarding changes in migration policy for family reunification (Figs 2 and 3).

Excerpt 3a: Natalie [N], Interviewer [I]

This first segment of Natalie’s narrative begins with a brief orientation (lines 1–3) which introduces the new government and their priority to change the foreign citizens law. As such, the story follows the typical structure of a narrative of personal experience (Labov and Waletzky 1966). The government is named specifically in line 1 and Natalie introduces herself in the narrative through the pronoun ‘me’ (line 6). She differentiates herself from the government in the evaluation clause about the law being an ‘absolute shock’. This extreme case formulation may help portray a circumstance as ‘unfair, immoral, embarrassing, uncomfortable, or in some other way undesirable and/or intolerable’ (Pomerantz 1986, p. 228). Natalie accompanies this formulation with the discourse marker ‘of course’ signalling that her shock is understandable to the interviewer who takes on the role of sympathetic listener. Natalie states ‘they didn’t ask they just…’ (line 9) followed by the hand movements depicted in Figs 2 and 3, used to communicate that the government moved ahead with plans to change the law without consultation. In line 13 Natalie explains how her advice came to be solicited, ‘it was totally kind of accidental’. The semantically extreme adjective ‘totally’ is followed by the softener ‘kind of’ perhaps to fix a problem of exaggeration (Edwards 2000) or as a filler during her search for the next word ‘accidental’. Natalie narrates the fact of being asked using reported speech (‘ask her’) accompanied by prosodic shifts, which help indicate that she is directly quoting (Holt 1996). In performing the reported speech she gives strength to the words and possibly displays an evaluation of the original speaker (mockery or parody) (Buttny and Williams 2000) or signals her disbelief at the randomness of the consultation.

Natalie’s story continues with another orientation phase (lines 16–18) in which she describes receiving a telephone call from the government worker. She refers to ‘A2’ which corresponds to a elementary or pre-intermediate level of language proficiency on the Common European Framework of Reference for Languages (CEFR).

Excerpt 3b: Natalie [N], Interviewer [I]

In this excerpt, Natalie emphasizes the accuracy of her memory by stating ‘I remember’ (line 16) and pointing her finger upward, which, together with the vivid detail about her location, aids in constructing a credible account (Potter 1996). Reported speech throughout this narrative permits Natalie to document the event as if it is occurring at the moment of telling (Tannen 2007) and gives a real-life feel to her professional identity performance. Natalie uses a plural pronoun ‘they’ to refer to a representative of the group of policymakers, but she then shakes her head and reformulates with ‘this lady’ (line 19) thus making the term of reference closer in proximity and perhaps more accurate as she remembers speaking with a woman. The woman consults Natalie on what level of language to require which indicates that Natalie has some amount of responsibility and choice in the formulation of the law. Therefore, language testing expertise is seen as valuable to the policy formation. The narrative moves to the complicating action (line 25). At this point, Natalie shifts her pronoun for the policymaker from ‘she’ to the plural ‘they’ reintroducing distance and emphasizing the diverging opinions: ‘they wanted A2’. In this way, she differentiates herself from the policymaker. The story reaches its climax in her third-person imperative: ‘I said, please people don’t do that’ (line 26). Through the use of direct reported speech, Natalie ‘shows’ and dramatizes the interaction (Labov 1972; Wierzbicka 1974; Mayes 1990). Although she presumably was speaking on the telephone to one person, Natalie uses the depersonalizing noun ‘people’. Lines 28-33 are resolution clauses (Labov and Waletzky 1966), which release the tension and tell what happened next. Natalie projects herself as the protagonist of the story acting to resolve the conflict. In doing so, she constructs a heroic role for herself. The fact that the woman was thankful and took the advice (line 33) further reinforces Natalie’s agentive identity and influence.

Although her expertise was sought out and was influential in the writing of the law, Natalie seems unsatisfied. She explains that the language testers preference would be ‘don’t give exams, just give them the possibility to learn’ (lines 35 and 36). This seems like a contradictory stance for language testers to take since presumably their work depends on developing and selling exams. Natalie shows her sympathy for test takers by naming them as a group and advocating for free opportunities for them to learn the language. The repetitive coda of her story ‘they don’t listen’ (line 40), links back to a previously repeated phrase ‘they didn’t ask’ (line 9) thus serving a referential and tying function (Halliday and Hansan 1976). The repetition also has an evaluative function (Labov 1972) emphasizing and intensifying the difference between the us/them groups that Natalie has created. It also brings into question the authority of language tester expertise since it appears that sometimes policymakers do not consult language testers. To summarize, Natalie employs prosodic marking and extreme adjectives to construct a professional identity as a language tester. She constructs an agentive role for herself through her multiple uses of direct reported speech and her direct impact on policy formation. She presents herself as knowledgeable about language levels and the needs of test takers, in contrast to the policymakers who are portrayed as being less knowledgeable of language assessment and distant from test takers.

Emily: Problem-solving partner—Language expertise is legitimized

In the fourth narrative, Emily recounts her relationship with policymakers following a question by the interviewer as to whether she had ever made objections to a governmental policy.

Excerpt 4a: Emily [E], Interviewer [I]

In this first excerpt, Emily begins by speaking about both language testers and policymakers together using the indefinite pronoun ‘everyone’ (line 3). She refers to a ‘good spirit’ and presents a harmonious relationship generalizable to ‘everything’ (line 1). In lines 6 and 7 she introduces the language testing institution and the government agency with their official names joined by the conjunction ‘and’ hinting at their active and equal presence in the narrative. In line 9 Emily uses the collective pronoun ‘we’ (Vertommen 2013) and hence she constructs one large group that encompasses both the language testers and policymakers. Extreme case formulations ‘everything’ (line 9) signal the similarity between the two groups. Language expertise appears to be understood and shared between them.

While a placid relationship was described, it is important to keep in mind that the interviewer had specifically asked for ‘objections to a government policy’. In this interactional context, Emily perhaps felt obligated to recall a disagreement between the language testing institution and the government agency. Despite the harmony in the relationship, Emily acknowledges that they may have different points of view (lines 10 and 11). She initiates a hypothetical story with the use of the conditional (Norrick 2000) ‘there may be’ (line 8) and ‘they might be suggesting’ (line 12), which serves as a preface to the narrative that follows her rhetorical question ‘well what was that discussion we just had?’ (line 13).

Excerpt 4b: Emily [E], Interviewer [I]

Emily begins this part of the story in the structure of a typical narrative (Labov and Waletzky 1966) with orientation clauses framing the experience as during the time of Covid (line 16). She also introduces test takers, referring to them with the general reference marker ‘people’ (line 18) and then ‘test takers’ (line 25). As the story unfolds, the harmony between the two main protagonists of the story (the testing agency and the national agency) is put in jeopardy by the test takers who present a problem (they want to cancel their test participation and get their money back). The prosodic markers on ‘cancelling’ and ‘required’ (lines 19 and 20) function as contextualization cues which qualify them as questionable practices (Gumperz 1982). In line 28 Emily uses agentive speech, expressing a clear volition (‘we wanted to change that’). However, she reformulates this assertion using the contrastive conjunction ‘or’, which would suggest that it was actually the governmental agency that wanted to change the practice. This confusion over who wanted to change the practice seems to further highlight the shared agency between them.

Emily acknowledges that they have different ideas about how to resolve the problem, but she downplays the differences with the softeners ‘kind of’ (line 31) and ‘maybe’ and ‘a bit’ (line 34) (Ishihara and Friedrich 2016). In the final line of the story Emily employs present continuous tense to relate the narrative to the present and resolve the tension created by the difference of ideas (‘we are now discussing’). While there are clearly differences of opinion between the two groups and a solution has not yet been reached, the fact that both parties are sitting at the table to discuss it illustrates a low power imbalance between them and shared responsibility. Overall through her linguistic choices of action verbs and voicing devices, Emily positions her group as co-actor with the government agency. She uses the pronoun ‘we’ throughout the story constructing a collective identity that includes both language testers and policymakers. There is no ‘I’ in her narrative, and hence she does not portray an individual professional identity separate from the larger group. Policy advice appears to be of a purely technical nature and void of any personal preferences or values. She provides an example of language testing expertise being acknowledged and legitimized (working together to resolve a problem in a relationship characterized by low power imbalances).

Discussion

Our analysis of language testers narratives demonstrates that the policy context will shape the role of language testers, as will their professional and institutional context. Ownership and control of processes and information can be sharply demarcated (Lo Bianco 2021) which is a factor in the (limited) influence that language testers may have on policy outcomes (Knoch 2021). Despite this, a central argument of the study is that language testers are an important stakeholder in policy decisions (Lo Bianco and Aliani 2013; Elder 2021) and they have choices about how to employ and enact their expertise in decision-making processes.

The findings demonstrate that how language testers conceive of the value of language testing expertise matters. As discussed in the literature review, one understanding of language testing expertise is largely technical in nature. In this view, the legitimacy of expertise for policy is based on providing justification of the quality of tests through test validation procedures. A second conception of language testing expertise is social in nature. In this view, the legitimacy of language testing expertise is based on providing not just a technical rationale, but also on considering the social and political justification for the use of tests and the values inherent in test constructs. These two views of language testing expertise are congruous with what researchers have called the ‘narrow’ and ‘broad’ view of validity and the validation of test score interpretations (Mcnamara 2010; McNamara and Ryan 2011). As illustrated in Fig. 4, combining these narrow and broad views of language testing expertise with a top-down or co-constructive policy context provides a simple theoretical basis for four modes of engagement: Objective Expert, Critical Activist, Problem-solving Partner, and Issue Advocate.

In the top left quadrant is the Objective Expert. The Objective Expert is exemplified by Robert’s identity positioning in a highly controlled, top-down policy environment. Robert saw his responsibility in this context as strictly technical in nature. He distanced himself from decision-makers and described a clear task division; experts = technical work, politicians = political strategy. Robert’s positioning may be an ascription––either consciously or unconsciously––to an ideology of impartiality and upholding the public value of neutrality (Weber 1978), the view that bureaucrats should remain ‘above the fray’ and strive for a value-neutral stance in executing political decisions. The ideology of impartiality hints at the traditional (if critiqued) dichotomy between scientific knowledge (neutral) and policymaking (value-laden) (Deygers et al. 2021). Robert’s positioning demonstrates an identity that relies strongly on the strength of being recognized as an impartial, highly professional language assessment expert. This positioning serves a clear purpose of avoiding normative questions and remaining a step removed from decision-makers. It thus lends a certain objectivity and credibility to language testing expertise.

The top right quadrant is the Critical Activist. For Christine, a lack of ownership or control converted her into an instrument of the state or a ‘compliant enforcer’ (Harding, Brunfaut and Unger 2020, p. 682). Christine experienced both inner and outer conflict as a result of being stripped of her authority to exercise her profession according to the standards to which she aspired. Having little agency called into question her identity as a professional language tester and even raised doubts about continuing in the profession. Without formal channels to influence policy, Christine looked for alternative ways to use professional expertise to influence. Thus her identity may align more specifically with a ‘language test activist’(Carlsen and Rocca 2022). This role describes a person who ‘asks critical questions about whether or not the use of a test has harmful consequences’ and ‘takes action to influence or change policies’, including everything from everyday acts of resistance to protests and demonstrations (p. 604). The Critical Activist role may imply personal or professional risks. Christine’s positioning demonstrates an identity that relies strongly on being viewed as a socially responsible language tester critiquing the policies that tests serve. This positioning serves the purpose of promoting ethical and just testing practices and lends greater accountability and transparency to language testing expertise.

In the bottom left quadrant of Fig. 4 is the Problem-Solving Partner. Emily’s description of language testers and policymakers working together supports a collaborative, problem-solving view of policymaking (Ansell, Sørensen and Torfing 2017). The dialogic or co-constructive policy system facilitates her position as a working partner with policymakers. The combined group is depicted as coordinated in negotiating different viewpoints and interests. This supports an argumentative view of the policy process with researchers and policymakers ‘making sense together’ (Hoppe 1999). Emily’s positioning demonstrates an identity that, like Robert, appears to be more technical than political in nature. The collective identity she presents is focussed on test-based concerns rather than on ideological arguments. Emily’s positioning demonstrates an identity that relies strongly on being seen as an engaged, effective, problem-solving expert. In this sense, the value of language testing expertise in the policy sphere expands to include negotiation and compromise.

The final bottom right quadrant represents the Issue Advocate. This role is exemplified by Natalie’s positioning as an advocate for test takers. Natalie does not aspire to the ideology of impartiality, but rather takes a critical stance and intervenes in politics when necessary to make policymakers aware of the implications of decisions (Hoppe and Jeliazkova 2006). The Issue Advocate is consulted or participates in public discourse at specific moments using promotion and persuasion to influence decision-makers (Spolsky 2022). Such moments can be structured (e.g. advisory committees, expert panels) or unstructured (informal or ad hoc exchanges, e.g. the telephone call that Natalie received). The Issue Advocate perhaps best correlates with Carlsen and Rocca’s description of language testers in Italy who explored various policy options with government officials (Carlsen and Rocca 2022, p. 608). While the Issue Advocate and Critical Activist both employ a broad view of language testing expertise, the distinction between the approaches may lie in the policy context and in the level of risk associated with their engagement. A policy system where language testers are given a voice may involve less risk. Carlsen and Rocca state that advocacy is more risk-free than activism which can come at a high cost (p. 604). In a top-down policy environment such as Christine’s, more may be at stake in the choice to intervene in the public discourse. In sum, Natalie’s positioning demonstrates an identity formed through an understanding of both the test use and policy context. This positioning serves the clear purpose of conferring greater visibility and influence to language testing expertise.

Returning to our original idea that language testers have choices in the policymaking context, it is important to bear in mind that the four modes of engagement described in this study are not definitive, nor is one necessarily better than another. Other modes may exist that are not captured in this analysis. A social constructionist perspective of identity (Schnurr and Chan 2011; Van De Mieroop 2022) implies a plural conceptualization of identities and helps understand how people shift in and out of a variety of not necessarily homogenous and consistent identities. Language testers, in turn, may shift in and out of different positions and the lines between them may be blurred. Scholars have pointed out that, in practice, complete objectivity in the policy context might not be attainable or desirable, and values and interests may always be inherent in expert contributions (Lo Bianco 2019). As is clear from the data, the necessity of one role over another will depend on the circumstances and how language testers conceive of language testing expertise. This study demonstrates that different positions can provide different things. In the end, what contributes to the legitimacy and authority of language testing expertise in policy settings is for values and stance to be clearly communicated and for different approaches to be complemented by each other.

Conclusion

As stated in the introduction, the stakes are high in language assessment, particularly when language tests are used to regulate who is selected and who is rejected for education, employment, and immigration. Therefore, the expertise and voices of those who research, develop, and administer tests truly matter in policy when selecting different courses of action. In the four narratives analysed, the policy context played a major role in granting different degrees of autonomy to language testers, who navigated positions of greater or lesser influence, power, and responsibility. While language testers may not be able to change the policy environment in which they work, they can choose how to engage. The data demonstrated four identities and positions in policy contexts based on different understandings of the value and legitimacy of language testing expertise: Objective Expert, Critical Activist, Problem-solving Partner, and Issue Advocate. Each of these positions has a place in responding to the demands of government test users and in guaranteeing fair and just testing processes.

This study has implications at the theoretical, methodological, and practical levels for applied linguists. Theoretically, scant research exists on language testers identities, hence this article opens up a prime area of research. More broadly, as language professionals of every kind have more and more contact with policymakers, this study demonstrates that there is much to be learned by looking at identity navigation in highly prescriptive environments. At the methodological level, this research is novel in that it uses narrative analytical techniques to explore the value of professional expertise in policy settings. Finally, the study has practical implications for the training and education of language testers. The findings suggest that language testers can cultivate a deeper awareness of the policy context and critically reflect on their own understanding of language testing expertise to better structure engagement strategies.

Notes on Contributors

Laura Schildt is a Doctoral Researcher at Ghent University, Belgium, working on a project entitled, ‘High-stakes Language Tests, Migration Policy, and Ethical Codes’. Her research interests include language assessment in migration policy and the role of language expertise in policy-making.

Dorien Van De Mieroop is a Professor of Linguistics at KU Leuven, Belgium. Her main research interests lie in the discursive analysis of identity in institutional interactions and narratives, about which she published more than 50 articles in international peer-reviewed journals and co-authored and co-edited books and special issues. She is co-editor of the journal Narrative Inquiry.

Bart Deygers is an Assistant Professor of Second Language Acquisition and Language Testing at Ghent University, Belgium. In the field of language testing, his research interests include migration policy, fair and just test score use, policy analysis, and validity theory. In SLA, he focuses on instructed language gains in vulnerable migrant populations.

Acknowledgements

First and foremost, we wish to thank the language testers who participated in this research for their openness and support. Second, with the pressure for rapid and numerous publications in academia, in-depth analysis such as this can sometimes be neglected. We wish to thank Marieke Vanbuel for repeatedly encouraging us to do this study. Finally, we are very grateful to the editors and anonymous reviewers for their dedication and for going the extra mile.

Funding

None declared.

Appendix 1 Transcription conventions

(based on Jefferson 2004; Mondada 2016)

References