Increasing the utility of epidemiologic studies as key evidence in chemical risk assessment

Abstract

Setting the stage

The Society of Toxicology (SOT) 2024 session, Capturing Unknowns: Increasing Utility of Epidemiologic Studies as Key Evidence in Chemical Risk Assessment, convened a group of risk assessors, epidemiologists, and toxicologists to discuss the importance of the integration of epidemiologic data into the derivation of toxicological reference values (TRVs). Historically, epidemiologic data have been used for hazard identification in chemical risk assessments, and rarely as key evidence for establishing quantitative dose–response relationships. As discussed in the session, risk assessors are now increasingly seeking to use human data as evidence in the dose–response step of chemical risk assessments. Epidemiologic studies can be highly influential when assessing the health effects of chemical exposures and can enhance evidence gathered from toxicological studies. For observational studies, risks, rates, prevalences, and odds ratios are common measures of the frequency of an outcome and are used to describe patterns of diseases or associations with chemical exposures. Because they evaluate the human experience, observational investigations can provide invaluable information for public health and risk-based decision-making.

Epidemiologic studies also have the benefit of real-world exposure scenarios, which may capture exposure intensities and frequencies that are more relevant to the population of interest. The use of epidemiologic studies for quantitative dose–response analysis is attractive to risk assessors, since uncertainties regarding the human relevance of observed effects in animals are removed, as well as interspecies uncertainties in animal to human extrapolation of toxicokinetics and toxicodynamics. Additionally, use of human evidence (including experimental and observational investigations, see Table 1) can reduce or mitigate the need for animal testing (Thomas et al. 2018).

Despite these strengths, challenges in interpreting the findings from observational studies exist, including internal (the extent to which results of a study can be relied on) and external (generalizability) validity. For example, a critical issue related to internal validity is whether the association between the exposure and outcome of interest can be explained by other factors (e.g., confounding) or whether the effect of the health outcome is modified by levels of another factor (e.g., effect modifier or interaction). Epidemiologic studies also often lack information necessary for conducting a dose–response assessment, such as quantified exposure concentrations, analyses on the shape of the dose–response curve, or data about the source-to-intake pathway (Christensen et al. 2015). Different human study designs can be more or less useful for application in dose–response assessments. Table 1 is adapted from authoritative sources and provides a broad description of types of human studies and an overview of the strengths and weaknesses encountered when assessing exposure/outcome relationships and evaluating causal conclusions (Szklo and Nieto 2007; Rooney et al. 2014; TCEQ 2017; NTP 2019).

While guidelines often recommend using the risk of bias (RoB) analyses to address the extent to which study results can accurately identify the relationship between exposure and outcome, there is still debate on how to incorporate studies that are identified as having study limitations or biases. Thus, understanding the utility and the limitations of applying epidemiologic data to dose–response assessment and uncertainty assessments is needed to reduce gaps in knowledge. This workshop summary provides an introduction and overview of the general needs and concerns related to using epidemiologic data in dose–response analysis and recommendations to address the uncertainties.

Current practice

Historically, epidemiologic studies are typically used for the hazard assessment step of TRV derivation. Due to the inherent limitations of observational studies (e.g., lack of randomization and inability to control exposure levels/timing), it is critical to evaluate the potential impact of bias in epidemiologic studies when interpreting the evidence base. Characterizing the potential presence, direction, and magnitude of bias enhances transparency of sources of uncertainty in the conclusions. Several approaches have been developed to evaluate individual studies for RoB (Steenland et al. 2020). A greater challenge, and a key element of synthesizing epidemiological evidence, is determining how RoB impacts certainty in the overall evidence for a health effect of interest; this requires considering RoB across available studies and data streams. Traditionally applied RoB approaches (e.g., ROBINS-E, NTP OHAT, GRADE) (Rooney et al. 2014; Schunemann et al. 2014; NTP 2019; Higgins et al. 2024) provide a systematically applied, but qualitative accounting of potential biases. A qualitative approach can provide an overview of the reliability of the data and the directionality of the biases may be predicted based on whether the bias is expected to be differential. However, when the qualitative RoB approach of the data is used independently, the magnitude of the bias is not described and therefore a lack of confidence in a causal conclusion may remain. Methods for quantitative bias assessment allow for the estimation of the sensitivity of the observed associations attributable to uncontrolled confounding, exposure misclassification, and other selection biases (Lash et al. 2014; Barberio et al. 2021; Fox et al. 2021; Verbeek et al. 2021). However, these post hoc adjustment methods require assumptions regarding the prevalence or the sensitivity and specificity of exposure or outcome ascertainment methods.

Triangulation is a method for evaluating bias during evidence synthesis, wherein causal inferences are strengthened by integrating results from different research approaches, where each approach has different sources of potential bias (Lawlor et al. 2016; NASEM 2022). When evidence synthesis (considering bias as well as factors such as consistency, coherence, effect magnitude, and exposure–response gradient, see Hill 1965; NTP 2019 for definitions), demonstrates that bias is not anticipated to explain the observed effects and the evidence is sufficient to support a hazard conclusion of a likely adverse effect, then epidemiology studies with sufficient dose–response information can be considered as candidates for the quantitative dose–response step of the TRV derivation.

Another solution for addressing uncertainties and bias, as discussed by the panelist from ToxStrategies, is to evaluate causality during the evidence synthesis phase with the use of a blend of causal diagrams, such as directed acyclic graphs (DAGs), together with traditional methods like the Bradford Hill considerations (Hill 1965) and other evidence integration tools (e.g., GRADE or NTP OHAT). Visualization of complex exposure–response relationships through the use of DAGs may support assessments of causality through identification of potentially biasing pathways between confounders and the hypothesized exposure–outcome relationship, among other biases attributable to population selection or misclassification (Digitale et al. 2022; Shimonovich et al. 2022). These tools and frameworks support risk assessors, through systematic integration or triangulation of evidence, to transparently determine whether observed exposure-outcome relationships are likely to be causal. An example of the blended use of Bradford Hill considerations and evidence integration frameworks (NTP 2019), is included in Vincent et al. (2024). Although not explicitly considered in the OHAT (NTP 2019) or GRADE (Schunemann et al. 2014) evidence integration frameworks, biological plausibility plays a crucial role in evaluations of causality. In the absence of a biologically plausible mechanism or mode of action, observed associations with a high RoB (usually due to confounding or poor exposure characterization) are less likely to be causal and are more likely to be a correlational or chance finding. As exemplified in Vincent et al. (2024), currently available mechanistic and mode of action information was used to evaluate the plausibility that inhaled formaldehyde could be systemically distributed to tissues associated with lymphohematopoietic (LHP) outcomes. Low confidence in the observed associations from epidemiological evidence was determined through the integration of epidemiological evidence of associations between inhaled formaldehyde and LHP outcomes, mechanistic information (and lack of biological plausibility for systemic distribution of inhaled concentrations), and the lack of observed LHP outcomes in toxicological evidence. Conversely, if there is a clear mode of action for the exposure–outcome pathway, then observed associations would be given more weight when deciding chemical hazard, even if limited by residual biases or uncertainties in exposure–response.

The panelist from the Texas Commission on Environmental Quality (TCEQ) provided an overview of their process for TRV derivation that includes evaluating epidemiologic studies in combination with experimental evidence from animal models and plausible biological mechanisms. Per TCEQ Guidelines (TCEQ 2015), data from human studies are preferred over animal data for TRV derivation. However, setting safe levels of chemicals in the environment using dose–response relationships from observational data is complex. Therefore, many of the chemical toxicity factors derived by TCEQ are ultimately based on animal data, often because it is the only evidence available with adequate information to conduct a dose–response assessment. Approximately 20% of the TCEQ toxicity factors are based on dose–response information from epidemiologic data.

Risk characterizations based on observational data contain an inherent degree of uncertainty and variability, which should be characterized (TCEQ 2015). The exclusion of an uncertainty analysis from a risk assessment prevents decision-makers from taking well-informed actions in setting health-protective standards for chemicals that cause adverse health effects. Uncertainty analyses are done on a case-by-case basis, and their components may be numerous and variable. For example, the uncertainty may refer to the model, the model parameters, the endpoint selected, the modeling methodology, the exposure estimation, etc. Although such uncertainty is often discussed qualitatively, various methods are being put forward to better quantify such uncertainty in a way that is useful to risk managers who are applying the chemical risk information to real world situations. For example, providing a range of TRVs for a chemical, rather than a single value, can provide important information if there is an exceedance of a TRV. As a default, the lowest value in the range can be used for screening, and if it is exceeded, the range demonstrates how much uncertainty (and conservatism) was present in the screening level and therefore can help with decision-making in the presence of multiple potential sources of risk. Multiple methods of uncertainty analysis are available, varying from relatively uncomplicated (such as using a range of TRVs) all the way to sophisticated quantitative uncertainty analyses (NRC 2009).

Opportunities for improvement

As noted by the panelist from Burns Epidemiology Consulting, LLC, epidemiologic research provides invaluable information for understanding the relationship between environmental exposures and health outcomes. While epidemiology and risk assessment have common goals of understanding and reducing human health impacts associated with exposure to environmental chemicals, each discipline utilizes different terminologies and skill sets. Further, regulatory approaches to evaluating study utility and quality for risk assessment purposes can vary, even within a single organization. This contributes to the challenges epidemiologists face when seeking to develop studies that can be used in chemical risk assessments, and that risk assessors can face when working to include the data from these studies (LaKind et al. 2023). For decades, scientists have recognized that dialogue between risk assessors and epidemiologists is crucial, yet tools that can assist in breaking down silos between the disciplines are limited (Deglin et al. 2021). One of these tools, the Matrix, is designed to enhance cross-discipline communication and to bridge the epidemiology/risk assessment gap. The Matrix includes a description of selected elements that when included in epidemiology design, analysis, and/or reporting enhance the use of epidemiology results for a risk assessment. The Matrix is not intended to supplant best practices for environmental epidemiology or existing risk assessment frameworks on integrating multidisciplinary data. Rather, the goal of the Matrix is to improve understanding and communication between the disciplines. Bridging the gap between epidemiology and risk assessment will enrich both disciplines and enhance public health decision-making (LaKind et al. 2020).

Final thoughts and future considerations

Chemical risk assessments are moving beyond the use of typical animal assays for dose–response assessment. This session provided active conversations between epidemiologists, toxicologists, and risk assessors and summarized recommendations to address inherent uncertainties that are critical for forward progress (Fig. 1). Although existing guidance provides frameworks for the use of epidemiologic assessments in dose–response assessment, there are no overarching, authoritative best practices for its use. Therefore, inconsistencies in the approach and application of epidemiologic research for dose–response assessment are prevalent and may impact confidence in points of departure (POD) derived from epidemiologic literature. Uncertainty assessments are useful tools that help risk assessment professionals understand the sensitivity of PODs to assumptions made during the modeling of epidemiologic evidence, including assumptions regarding uncertainty in exposure estimations and background. Guidance on the incorporation of limitations from confounding, exposure uncertainty, co-exposures, outcome ascertainment, and selection bias in POD derivation is needed to improve confidence in reference values derived from epidemiologic data. In the meantime, the continued use of all data streams (e.g., mechanistic, animal, epidemiologic) for evidence integration is crucial to reduce uncertainties in dose-response models derived from epidemiologic evidence.

Fig. 1.

Panelists’ thoughts on the increasing utility of epidemiologic studies for chemical risk assessment.

Open in new tabDownload slide

Acknowledgments

The authors would like to thank Dr Elizabeth Radke-Farabaugh for her contributions to the overall session and panel discussion. The authors would also like to thank the Society of Toxicology for hosting this session at the 2024 Annual Meeting and providing a platform to allow researchers to engage in open dialogues about current scientific topics.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Conflicts of interest

Opinions expressed are of the authors and do not express the views or opinions of the US Food and Drug Administration.

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References