Using freedom of information requests to access novel data sources in health professions education research

Abstract

Educators and researchers are reliant upon access to data to drive teaching methods, curricular improvements, and progress in medical education research. However, data are not always accessible, due to resource constraints, institutional policies, and privacy concerns. Researchers have attempted to access novel data sources through surveys, semistructured interviews, and databases; however, these methodologies are limited.

To improve access to data, Freedom of Information (FOI) Acts grant researchers the ability to formally request data that any public institute holds. Researchers have been reluctant to use this tool due to negative perceptions, despite its unique benefits.

To increase awareness of this underutilized methodology, we summarize the process of FOI Act requests, its strengths and weaknesses, and the ways in which health professions education can leverage FOI requests within research. We provide examples of the use of FOI requests as a research method within adjacent fields and nascent use within the field of health professions research. In doing so, we hope to highlight how FOI requests can be a useful tool in health professions education researchers and its potential to increase access to unique data sources.

Introduction

Educators and researchers are reliant upon access to data to drive teaching methods and curricular improvements [1, 2]. Data-driven insights enable educators to identify areas of success and can support education providers in identifying priority groups, monitoring progress, and understanding the effectiveness of interventions to maintain high educational standards [3, 4].

However, data are not always accessible. This is due to resource constraints, institutional policies, and privacy concerns [1]. Organizations may disseminate information through reports, yet, raw datasets are seldom published alongside, which stifles additional analysis and transparency [5].

Increased access to data can increase the depth and breadth of scholarly inquiry [1] and the ability to develop pedagogical practices and policies. Greater access can increase the quality and quantity of research and ‘democratize science’ by reducing inequities between research teams, institutions, and countries. Increased data accessibility encourages researchers and educators to collaborate globally, leading to the exchange of ideas, good practices, and insights, ultimately improving healthcare and education systems [1].

To improve access to data, ~80 countries grant their citizens the right to request access to information held by public authorities through Freedom of Information (FOI) requests [6]. As a consequence, FOI requests are a tool that members of the public can use to request information from institutions. FOI requests have already proved a vital resource within journalism, where news outlets have used the legislation to collect data on patient safety issues [7]. Some academics have embraced FOI requests as a research tool; for example, Savage & Hyde used FOI requests to gather data from 48 national regulators regarding whistle-blowing disclosures in food regulation [7]. Meanwhile, within social sciences, Murray collected data from 152 directors of children's services to influence policy changes regarding looking after children [8]. However, despite some academic interest, there is an enduring lack of awareness of FOI legislation globally amongst academics, with <5% of FOI requests arising from the academic community [9]. Indeed, there are few health professions education studies that have used FOI requests [10, 11].

In this paper, we summarize the process of FOI requests, its strengths and weaknesses, and the ways in which health professions education leverages FOI requests within research. In doing so, we hope to highlight how FOI requests can be a useful tool in health professions education research and its potential to increase access to unique data sources.

Method

In Europe, the USA, and Australia, formal requests for information from public bodies are termed ‘Freedom of Information Act requests’; however, other countries use terminology such as ‘Access to Information Act’, ‘Right to Information Act’ [12], and the ‘Official Information Act’ [13]. Whilst there are differences between each country’s policies [14], each access law shares the same function: to increase the transparency of the public sector and their activities by facilitating access to data.

An understanding of the provisions and jurisdictions of the FOI Act is important to any discussion surrounding its use [15]. We provide an example of the FOI legislature used in the UK in Table 1. We have chosen to use the UK to provide more detail on legislation because it represents a comprehensive framework, illustrating the balance between transparency and data protection. In providing this information in Table 1, we hope to offer insight on who can make an FOI request, what for, and when.

Why use Freedom of Information requests?

FOI requests provide a route to raw information or data sources that may be difficult to access with other methods such as surveys, interviews, or medical education databases. The FOI request can be used to ask for qualitative and quantitative responses to questions or documents and media in their unaltered form such as course handbooks, assessment policies and metrics, protocols, timetables, and curricular information. This access allows researchers to analyse and draw conclusions from the information first-hand and can also be used alongside other methods to validate or triangulate findings.

In Table 2, we present an example of how FOI requests were used to create a robust analysis of teaching by UK medical schools. The data would have been difficult to obtain using other methods. For example, comprehensive curricular datasets are not usually made public, surveys can suffer from low response rates, medical databases (such as the UK Medical Education Database (UKMED)) do not collect curriculum structure and content data, and interviews communicate individuals’ perspectives, rather than comprehensive reporting of curriculum information. The data in the Analysis of Teaching of Medical Schools (AToMs) study were essential for understanding the current practice of medical education in the UK, offering insights that were previously inaccessible. This example shows how FOI requests can increase access to information and support detailed comparative analyses.

Making a successful Freedom of Information request

An in-depth guide to designing FOI Act requests is beyond the scope of this article. For readers who want a full account of FOI Act design and practical guidance, we would recommend ‘Making Freedom of Information Requests: A Guide for Academic Researchers’ by Bourke et al. [16] and to follow Walby & Luscombe’s guidance on good practice in FOI research [17]. Submitting requests can be facilitated through ‘whatdotheyknow.com’ [18], a comprehensive website that also tabulates data from successful FOI Act requests. For a deeper understanding of the legal framework surrounding the act, we encourage readers to identify and explore in depth the relevant information access laws in their country.

Nevertheless, there are key concepts that should be considered at the core of FOI research, and the principles of good FOI research draw on similarities with good practice survey design [19]. Rigour can be achieved in FOI research by having a well-planned, systematic approach with specific attention given to the request and sampling strategy used [17].

A standardized request should be designed with consideration given to existing literature. Clear questions are paramount in FOI requests, and expert validation can improve clarity and ensure relevance to the research topic [19]. It can also be used to help identify potential biases, errors, or methodological issues. The number of questions should be kept to the minimum needed to answer the research question, as this reduces the burden on the institution and research team alike. Other methodologies such as surveys use formal pilot testing or cognitive interviews with respondents to improve validity. In FOI research, this is often not feasible due to the limited number of institutions and the time and cost limits of the FOI Act—as a consequence, this may impact content and response process validity [11]. Where this is a concern, proxies could be used—for example, people who hold similar roles at the researcher’s institution could be asked to review the questions, and cognitive interviews could be completed with them. Rather than using pilot studies, the researcher can sample a small number of institutions initially and review the preliminary data. This can identify issues with understanding and further refining of the questions [7, 17].

Many researchers often want large volumes of data [16], from numerous different sources, resulting in needing to make a series of requests. Therefore, researchers should develop a clear plan for how they will send and track requests and store responses and data. Researchers should ensure they have the capacity to handle the volume of communications and expertise within their team to review, extract, and analyse the requested data.

A note on ethics

As with any research, care should be taken to ensure ethical research design. Walby et al. [20] argue that ethical approval should not be needed for FOI requests. This is because of the existing built-in bureaucratic mechanisms, which already provide a legal framework for institutions to review (and reject) requests. An additional review by an ethics board would be unfair as it can result in ‘double jeopardy’—whereby the request is reviewed twice through similar processes—and this could also infringe upon a citizen’s legal rights [20].

Secondly, the information obtained through FOI requests is considered published material; its sheer presence in the public domain mitigates the potential for harm or privacy concerns from its use within research. Nevertheless, some UK ethical review boards have begun to treat FOI requests as inherently risky and in need of independent ethical scrutiny [20]. This stance, in our experience, is often characterized by a lack of familiarity with FOI research and risks imposing governance in a manner that creates additional barriers to accessing data—contrary to the act’s original intent.

Ethical review processes are a fluid, subjective construction, and, as such, the ethical evaluation of FOI Act research may differ significantly between committees, review boards, and journal editors. Consequently, for peace of mind, researchers may wish to discuss the project with an ethics board and seek an ethics waiver prior to starting any FOI research.

Discussion

To evaluate when and how to appropriately use FOI requests as a research tool, we need to consider the strengths, constraints, and boundaries of its use. Here, we explore the strengths of FOI research including being supported by a legal framework, creating high response rates, and increasing accessibility.

All methods also have inherent weaknesses, and some are more critical than others. We also consider constraints including the perception of FOI as a confrontational tool, unintended costs, poor-quality responses, and delays. Finally, we consider when a project may be better served by an alternative approach, to assist researchers in weighing the options in their ‘toolbox’ of methods.

Strengths of Freedom of Information research

Legal framework

The FOI Act provides both researchers and institutions with a well-defined framework to follow when requesting and handling information. The FOI Act serves as a mediating mechanism that balances the interests of stakeholders to reach an agreeable, legally supported outcome.

Researchers can encounter difficulties when obtaining information from large public authorities, as it is not always clear who has the right to access and disseminate data. For example, Devine et al. previously encountered issues when universities initially voluntarily provided information via interviews but later challenged the authors’ right to publish the material [10, 21]. Any data provided through FOI requests is considered explicitly in the public domain, meaning there should be no subsequent disputes in its use [22].

The FOI Act also protects public authorities, who have a right to protect their service users and by extension their data. The FOI Act, functioning as a robust legal framework, equips institutions with clear, well-supported avenues to reject vexatious requests, protect personal information, monitor the data they release, and reduce unnecessary resource allocation.

High response rate

A high response rate is important as it improves validity, generalizability of conclusions, and the statistical power of quantitative data [23]. Studies that use FOI requests have a very high response rate, for example, two studies that used FOI requests in medical education research both achieved a 100% response rate from the universities they approached [11, 21]. This is in direct contrast to surveys, which are susceptible to low response rates [24], which can introduce a nonresponse bias and affect the validity of conclusions [23]. For example, a similar survey-based study of economics teaching at universities received a response rate of 35% [25]. The legal right to access data is unique to the FOI Act, and this considerably increases response rates.

Accessibility

Research projects often have limited resources to invest in data collection. All data accessible by an FOI request have already been collected by the public authority. The FOI response from the institution simply provides that information to the person requesting it. This reduces duplication of work and FOI requests are therefore more cost-efficient than repeating the data collection process via surveys or interviews. As a consequence, FOI requests have the potential to democratize research by opening opportunities to researchers in low-resource institutions, and countries, where limited resources can act as a barrier to wide-scale research [7].

Freedom of Information requests can be a conversation

Most public organizations in the UK have a designated FOI officer who can serve as a point of contact. The FOI Act includes a right to advice and assistance from the public organization [26], but the researcher would be wise to build this into a positive relationship with the organization rather than inconsiderately insisting on their statutory rights. Clear, specific objectives regarding the information requested not only ease the workload for the FOI officers but also contribute to creating a positive rapport. The more time invested in a constructive dialogue with the FOI team, the more likely they are to understand the purpose of the research and why the data are being requested. In cases where the information sought is not readily accessible, these officers may signpost the researcher to alternative sources. If there is ambiguity with an FOI Act request, the public authority will seek clarification prior to responding, and it is in their interest to ensure that they reply adequately to your request. They do not want to duplicate or create unnecessary work for themselves. Similarly, researchers can perform repeat FOI requests to clarify or obtain more information if anything is unclear in the response from the public authority. This is something that is not possible when using a survey, as once the questions are created and distributed, the quality of the responses is then dependent on the respondents’ interpretation.

Constraints and caveats of Freedom of Information research

Perception of Freedom of Information as a confrontational tool

Arguably, the biggest limitation of the FOI Act methodology is academics’ perception of it, and it is often seen as a confrontational tool, which negates its use [9, 27]. Many esteemed authors do not view FOI requests as an admirable research method. Some academics criticize the FOI Act as trawling for data or associated with political agendas or vexatious requests [6, 27]. This is a common misconception, which will only improve with greater awareness of the method but is currently exacerbated by incorrect application.

Unfortunately, the poor perception of the FOI Act extends to the institutions themselves, with the FOI legislation seen as something to protect against, rather than a resource that can enhance the process and efficiency at which they deal with data requests [9]. Universities offer training to staff members to ensure compliance with the act [9], but little to no teaching is provided to their academics to use the tool to obtain information. The culture perpetuates the FOI Act as a confrontational tool, which academics and researchers may, understandably, believe will jeopardize their future access and positive relationship with an institution [9, 27]. Furthermore, institutions could avoid high volumes of FOI requests by providing greater access to data, through data portals, for example. If the information is already accessible, then responses to FOI requests would simply be signposting the researcher to the data.

Unintended costs

Researchers should consider the appropriateness of their request and explore alternative methods to FOI requests before they choose it as their method of choice. This is because while institutions are protected in terms of maximum cost, there can be a significant cost across multiple institutions (Table 1). For example, Breathnach et al. highlighted that a request made to all 167 acute NHS trusts in England could generate substantial costs of up to £75 000 [27].

Passive method design that can result in poor-quality responses

The FOI Act provides no right to high-quality information, only the information that institutions hold. FOI Act requests are often viewed as passive in nature [15], as they involve the collection of existing publicly available information rather than actively interacting with the sources of information. This passivity differentiates FOI Act requests from more active data collection methods like surveys or semistructured interviews [15, 28]. Consequently, FOI requests cannot gain new information. Researchers are reliant on existing information available and are constrained by the limitations of the institution’s processes. It may not be clear what methods were used to collect the data initially, store it, or extract the requested data. The researcher is placing all responsibility for the data in the institution, and it may not be possible to ascertain whether the data are complete or accurate.

Similar to databases such as UKMED, the FOI Act is constrained in its capacity to access comprehensive metadata regarding curricular structures and content. The challenge of gathering metadata is a broader issue in health professions education research, particularly when dealing with historical data, as curricula, educational practices, methodologies, and metrics evolve over time [29].

Furthermore, data are reliant upon the person responding to the request and their own thoroughness and quality. There are risks of systematic errors where the person may not interpret the request appropriately or random errors if the data are incorrectly transcribed in the response [11]. Indeed, the same standardized request sent to multiple institutions can generate different results due to the human subjectivity of interpretation [7]. If no reason is provided as to why the data requested are required, institutions may not be familiar with the true aim of the research and may interpret the request incorrectly.

As a result, researchers can be provided with overwhelming amounts of data in unhelpful formats such as computer outputs or unstructured data [30]. For example, the data obtained in the AToMs study (Table 2) were a large volume of medical school timetables. Timetables may appear intuitive to those familiar with the institution; however, they can be relatively uninterpretable to an outsider. The AToMs study therefore relied on the recruitment of locally knowledgeable coresearchers who were integral to the success of the study [10]. Researchers should consider the data they are requesting, and the likely format it will be received in, and understand that the FOI Act is the beginning of the research journey. Researchers should therefore factor time into the collation and analysis of the data. Providing the institution with a clear structure or desired format can reduce ambiguity for all parties. It may also transpire that, after all these efforts, the institution does not hold the required data. For example, documents or data points may never have been stored or may have been deleted after a certain period of time.

Bureaucratic delays

FOI legislature often includes a time limit to provide a response in the range of 15–30 days [12, 13, 31, 32]. However, since the introduction of the FOI Act, institutions have become well versed in managing requests, and Roberts notes that institutions have generated ways to act evasively or dilute their effectiveness [33].

Moreover, each time a query or clarification needs to be discussed, the time limit is paused until there is clarity on the request received [34]. Some institutions may provide simple data to the research question posed, stating yes or no without any elaboration. Further delays are incurred whilst the researcher rephrases or redesigns their questions to better access the data they desire. In our own research [11], we found that this can cause significant delays.

If the public body does not reply within this allotted time, there is little that can be done to speed up the process of obtaining the information. As nonresponse is technically against the law, the requester can follow the appropriate channels to investigate—which in the UK is to contact the Information Commissioner’s Office—and this should eventually result in receiving the data.

Limited scope of data and alternative method choice

As we have discussed, FOI requests are useful for collecting a wide range of data types including qualitative and quantitative responses to questions and different media. However, FOI requests cannot answer ‘why’ an organization did something (unless this was explicitly outlined in an existing data source). For these types of questions, surveys or interviews might be more suitable. Similarly, we have found that FOI requests are challenging when answering questions that require quantitative data beyond a few data points. It may not be possible to share a whole database, and each data point has to be requested and extracted. In this case, existing databases may be more suitable and can provide access to raw data that can be investigated thoroughly.

Conclusion

Valuable data sources exist that can influence health professions education, but these sources are locked within institutions. The FOI Act is a unique methodology that is key to unlocking this data and facilitating an open and transparent area of health professions education research. It is a valuable alternative to other methodologies due to its robust legal framework, which not only facilitates researchers accessing data but also protects institutions and individuals. Nevertheless, FOI requests are underutilized within health professions education. This is predominately rooted in its current perception as a confrontational tool both by academics and institutions alike. These criticisms are extrinsic to the method itself and often reflect the poor application of the act rather than an inherent flaw with the methodology. Researchers should therefore understand the strengths and limitations of FOI research and plan their research considerately. High-quality FOI research can be achieved by following systematic processes outlined by good practice guidance.

Acknowledgements

None.

Ethical approval

Not applicable.

Author contributions

S.E.Y., M.E.L.B., and M.H.V.B. were responsible for Writing - original draft; Writing - reviewing and editing, and Methodology. M.H.V.B. was responsible for Conceptualisation and Supervision. S.E.Y was responsible for project administration.

Conflict of interest statement

None declared.

Funding

None declared.

Sources of support

Nil

Author descriptions

Dr Sophie E. Yale is a GP Registrar currently working in the North-East of England.

Dr Megan E.L. Brown is a Senior Research Associate at Newcastle University and co-chair of the NIHR Incubator for Clinical Education Research.

Dr Matthew H.V. Byrne is a DPhil candidate and a Urology Registrar at the University of Oxford, UK. He is the chair of MedEd Collaborative.

Study type

Education and learning.

References