Evaluating Pathogenicity Scoring system for Missense Variants on real-world data: A comparison between AlphaMissense and SeqOne DiagAI pathogenicity score

Context

Predicting the impact of missense mutations remains a significant and complex challenge in human genetics. Approximately 98% of these amino acid substitutions are classified as having uncertain significance, underscoring the ongoing difficulty in determining their effects on protein function and associated health implications (Cheng et al., 2023).

Here, we provide a real-world comparison between scores that rank mutations according to their predicted pathogenic impact

  • Google AlphaMissense is a novel pathogenicity scoring system rooted in AlphaFold’s predictive capabilities (Cheng et al., 2023) 
  • Established scores REVEL (Ioannidis et al., 2016) and CADD (Rentzsch et al., 2018)
  • SeqOne DiagAI’s Universal Pathogenicity Score (“UPP”) that emulates ACMG guidelines with a machine learning model

Google has recently unveiled AlphaMissense, a novel pathogenicity scoring system rooted in AlphaFold’s predictive capabilities (Cheng et al., 2023). AlphaFold, a cutting-edge AI developed by DeepMind, revolutionizes life sciences by accurately forecasting protein structures from amino acid sequences (Jumper et al., 2021). Unlike traditional ensemble methods that aggregate multiple scores, AlphaMissense leverages AlphaFold’s structural predictions to assess the pathogenic potential of missense variants—mutations that modify the protein’s amino acid composition. This innovative model integrates insights from protein structure and linguistic models used in 3D protein structure prediction.

SeqOne has developed an advanced variant pathogenicity prediction score within its SeqOne DiagAI suite, designed to assist geneticists in diagnosing hereditary diseases. This model leverages a machine learning algorithm trained on a comprehensive dataset comprising over one million genetic variants from ClinVar. To enhance interpretability and closely align with expert evaluations, the model integrates features inspired by ACMG criteria, utilizing data from sources such as ClinVar, gnomAD, and dbNSFP. 

Method

Upon the public introduction of a novel scoring metric, our team at SeqOne undertakes a preliminary scientific benchmark analysis utilizing real-world datasets to ascertain its efficacy and determine its potential integration into our platform and proprietary AI-based predictive models.

We assessed the performance of AlphaMissense in conjunction with established scores such as REVEL (Ioannidis et al., 2016), CADD (Rentzsch et al., 2018), and our proprietary AI-driven pathogenicity score, “UPP” (Universal Pathogenicity Prediction), across 62 Whole Exome Sequencing (WES) datasets identified with diagnostic missense mutations. To ensure the integrity of our evaluation, particularly given that our UPP model is trained on ClinVar data, we meticulously excluded any diagnostic variants previously cataloged in ClinVar. This measure eliminates potential benchmark biases favoring UPP or REVEL. This empirical assessment yields critical insights into a key performance indicator consisting of the percentage of WES analyses with the diagnostic variants being part of the best ranked variants.

Such a real-world benchmarking exercise is instrumental in elucidating the tangible benefits these scoring metrics may offer to biologists in their clinical diagnostics and routine analytical workflows.

Results

Benchmarking pathogenicity scores against the entirety of missense variants identified through Whole Exome Sequencing (WES) offers insights into their respective efficacies. 

REVEL operates as an ensemble method amalgamating numerous scores. A large proportion, typically 60%, of the WES samples have their diagnostic variant well-ranked by REVEL. However, in terms of sensitivity—a measure of the proportion of diagnostic variants that are ranked in the top list of variants—AlphaMissense exhibits superior performance.

CADD, which was previously regarded as the benchmark in this domain, does not surpass AlphaMissense in terms of variant ranking, suggesting a shift in the paradigm of state-of-the-art scoring methods is warranted.

SeqOne’s proprietary “UPP” score surpasses the performance of other evaluated prediction scores. This performance reflects UPP integration of a collection of predictive factors influenced by ACMG guidelines—including prediction scores, Gnomad allele frequencies, Clinvar mutation prevalence, recognition of mutational hotspots. This holistic approach to pathogenicity scoring for missense variants surpasses conventional methods to rank variants.

Further analysis incorporated an initial variant filtration based on Gnomad frequency (population frequency < 0.01), a standard practice in causal variant identification during variant interpretation. Post-Gnomad filtering, all methods enhance their performance significantly, narrowing the performance gap between them. However, SeqOne’s UPP remains superior to all other prediction scores.

Figure: Percentage of analysis with the diagnostic variant, which was found by a geneticist, being in the top-rank list of variants according to different scoring systems: AlphaMissense, CADD, REVEL and SeqOne UPP.

Limitations

A notable limitation of the UPP score stems from its dependency on a range of predictive and annotation metrics customary in clinical contexts. This dependency implies that while the model is proficient in evaluating variants with established clinical significance for medical diagnostics, its capability may diminish when encountering novel variants lacking recognized clinical importance. The continuous evolution of the ClinVar database, marked by regular and significant updates to variant classifications, necessitates an ongoing refinement of our model to remain in sync with the latest developments. The most recent update to our model aligned with the November 2022 release of ClinVar, and we have instituted a regimen of periodic updates to seamlessly integrate new information from ClinVar.

Moreover, our model is designed to identify variants not yet annotated in ClinVar that exhibit features consistent with those identified as pathogenic in the database. However, it’s important to note that while ClinVar categorizes certain variants as pathogenic, our evaluations have flagged some of these as potential “artifacts” or “false positive pathogenic” variants, employing specific criteria for such determinations.

To augment the model’s precision in distinguishing authentic pathogenic variants from potential artifacts and false positives, we have incorporated frequency data from gnomAD into our analyses. This integration aims to bolster the model’s discernment capabilities, thereby enhancing the accuracy and reliability of the diagnostic process.

Discussion

The introduction of AlphaMissense is poised to influence the predictive score landscape profoundly. Its standalone performance rivals that of established benchmarks.
Likely, AlphaMissense will soon be integrated into ensemble approaches, like REVEL and METARNN, augmenting their efficiency in predicting pathogenicity for missense variants.

SeqOne has created UPP, a component of the SeqOne DiagAI suite, tailored for everyday clinical diagnostics, following the principles of explainable AI, as outlined by ACMG guidelines.

SeqOne is dedicated to equipping biologists with a sophisticated tool designed to streamline the decision-making process, thereby enhancing the efficiency of diagnostics and contributing to superior patient outcomes.

SeqOne proprietary AI-driven pathogenicity score, “UPP” (Universal Pathogenicity Prediction) is available as a standalone API and is part of SeqOne clinical genomics cloud platform. To learn more contact science@seqone.com.

Stay in the loop Want to stay on top of latest genomic analysis news? Subscribe to our monthly newsletter.  

References

Cheng, J., Novati, G., Pan, J., Bycroft, C., Žemgulytė, A., Applebaum, T., Pritzel, A., Wong, L. H., Zieliński, M., Sargeant, T., Schneider, R. G., W, A., Senior, Jumper, J., Hassabis, D., Kohli, P., & Avsec, Ž. (2023). Accurate proteome-wide missense variant effect prediction with AlphaMissense. Science, 381(6664). https://doi.org/10.1126/science.adg7492

Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., Bridgland, A., Meyer, C., Kohl, S. a. A., Ballard, A. J., Cowie, A., Romera-Paredes, B., Nikolov, S., Jain, R., Adler, J., . . . Hassabis, D. (2021). Highly accurate protein structure prediction with AlphaFold. Nature, 596(7873), 583–589. https://doi.org/10.1038/s41586-021-03819-2

Ioannidis, N. M., Rothstein, J. H., Pejaver, V., Middha, S., McDonnell, S. K., Baheti, S., Musolf, A. M., Li, Q., Holzinger, E. R., Karyadi, D. M., Cannon-Albright, L., Teerlink, C. C., Stanford, J. L., Isaacs, W. B., Xu, J., Cooney, K. A., Lange, E. M., Schleutker, J., Carpten, J. D., . . . Sieh, W. (2016). REVEL: An ensemble method for predicting the pathogenicity of rare missense variants. American Journal of Human Genetics, 99(4), 877–885. https://doi.org/10.1016/j.ajhg.2016.08.016

Rentzsch, P., Witten, D., Cooper, G. M., Shendure, J., & Kircher, M. (2018). CADD: predicting the deleteriousness of variants throughout the human genome. Nucleic Acids Research, 47(D1), D886–D894. https://doi.org/10.1093/nar/gky1016

SeqOne Genomics appoints Martin Dubuc as Chief Executive Officer

SeqOne Genomics, an award-winning French HealthTech startup specializing in AI-powered clinical genomics, is pleased to announce the appointment of Martin Dubuc as its new Chief Executive Officer. Historic investors include Elaia, IRDI Capital Investissement, Merieux Equity Partners, Omnes, and Software Club.

Mr. Martin Dubuc joined SeqOne’s board of directors in October. He is a seasoned healthcare executive with global experience who most recently headed up Biogen’s Global Digital Health division, driving personalized and digital medicine. He brings over two decades of experience in the pharmaceutical industry at Pfizer, Merck&Co, Biogen, and thought leadership in digital health.

As part of the transition, Nicolas Philippe, PhD, co-founder and until then Chief Executive Officer, will become President and Chief Product & Scientific Officer, heading up research and product development of the proprietary SeqOne clinical genomics platform.

These appointments reflect SeqOne’s market leadership ambition and growth strategy across international markets and the biopharmaceutical industry.

Sacha Loiseau, Ph.D., chairperson of SeqOne Genomics’ Board, stated: “Under Nicolas’ visionary leadership, SeqOne has passed a multitude of important milestones across product development, clinical advancements, regulatory approvals, and commercial successes, notably in the blazingly fast development and launch of a best-in-class HRD test. We are thrilled to have Martin spearhead the next phase of SeqOne’s growth story as we scale up to become a global leader in the rapidly growing next-generation sequencing clinical informatics market. Martin’s strategic acumen as our chief executive, coupled with Nicolas’ deepened focus on our product innovation and science, positions SeqOne at the cusp of transformative success.” “SeqOne’s leading clinical genomics platform empowers genetics labs and healthcare professionals to improve patient care across multiple therapeutic areas. I am excited to join a highly talented and multidisciplinary team at this exciting time and with such an important mission of making personalized medicine a reality at scale, accessible to all”, commented Mr. Dubuc.

About SeqOne Genomics:

Founded in 2017, SeqOne Genomics, headquartered in Montpellier, France, is a fast-growing deep-tech company focused on turning genomic data into medically actionable insights in oncology and rare and inherited diseases. SeqOne clinical genomic analysis platform, powered by explainable AI, provides genomic labs and healthcare professionals with best-in-class accuracy, usability, and automation seamlessly across lab technologies.
By streamlining complex multi-omics analysis workflows, we pave the way for a new era of personalized healthcare at scale, where genetic testing becomes as routine as traditional blood work, radically enhancing health outcomes for all.

The company has won numerous awards, including the iLab award and the ARC Cancer Foundation’s Hélène Stark prize. It has been nominated twice for the prestigious Prix Galien award. Investors include Elaia, IRDI Capital Investissement, Merieux Equity Partners, Omnes, and Software Club.

Web: https://seqone.com
Contact: media@seqone.com



Comment aider le biologiste moléculaire à diagnostiquer les maladies génétiques grâce à l’intelligence artificielle? – Lunch Atelier aux Assises de Génétique

Nous sommes ravis d’annoncer que SeqOne Genomics sera présent aux 12èmes Assises de Génétique Humaine et Médicale et animera un Atelier-Déjeuner le Mardi 9 Janvier 2024 de 12h45 à 13h45.

L’essor du séquençage dans le diagnostic des maladies génétiques a conduit à une augmentation des prescriptions pour l’analyse d’exomes et de génomes, mettant les généticiens face à une charge de travail croissante.

Les Pr. Laurent Mesnard, Dr. Kevin Yauy et Dr. Marine Dancer partageront comment l’utilisation de l’intelligence artificielle se révèle précieuse en triant et réduisant le nombre de variants à analyser, en tenant compte des phénotypes des patients à partir de comptes-rendus de consultations, ce qui améliore l’efficacité du diagnostic moléculaire et simplifie le processus dans la routine clinique.

Inscrivez-vous à notre atelier-déjeuner en remplissant le formulaire suivant. Vous recevrez une confirmation de réservation par email.

Le jour J, soit le Mardi 9 Janvier matin, passez à notre stand (n°22) pour récupérer votre bon de commande ou présentez-vous directement devant la salle Maillot à partir de 12h30.

Réservez votre place pour l’Atelier-Déjeuner :

SeqOne Genomics appoints Martin Dubuc, Senior Pharmaceutical and Digital Health Executive to its Board of Directors

Welcome Martin Dubuc to the SeqOne Genomics board

M. Dubuc will strengthen the Board’s capabilities in the fields of personalized medicine and companion diagnostics

Montpellier, France – Thursday, 26th of October, 2024 – SeqOne Genomics, a French company specialized in artificial intelligence-enhanced genomic analysis solutions, is pleased to announce the appointment of Martin Dubuc to its board of directors.

SeqOne Genomics is at the forefront of genomic data analysis in clinical applications, pioneering innovative software solutions that leverage artificial intelligence enabling more precise and personalized medicine.

Mr. Martin Dubuc, is a seasoned senior executive with Global experience who most recently headed up Biogen’s Global Digital Health division driving personalized and digital medicine. He brings over two decades of experience in the pharmaceutical industry at Pfizer, Merck&Co and Biogen across France, Europe and US combined with thought leadership expertise in digital health.

Sacha Loiseau, Ph.D., chairperson of SeqOne Genomics’ Board, stated: “I am thrilled to welcome Martin Dubuc to SeqOne’s Board. His deep insights into the pharmaceutical industry, combined with his knowledge of digital health technologies, will be
instrumental in taking SeqOne Genomics to the next level and providing the most advanced genomic analysis solutions for the pharmaceutical industry.”

“I was immediately seduced by the combination of people, hard science and technical
expertise underlying SeqOne’s genomic analysis platform, coupled with their focus on developing in-vitro diagnostics and companion diagnostics making personalized medicine a reality for patients and bringing innovative capabilities as a strategic
partner to LifeScience companies across the lifecycle.” commented Mr. Dubuc, “The exponential and converging progress of technologies underlying genomic testing opens a new era of opportunities. SeqOne is well positioned to make a meaningful
difference for patients partnering with LifeScience companies catalyzing precision medicine therapy development, genetic disease diagnosis, personalized medicine, and companion diagnostics”.

Dr. Nicolas Philippe, CEO and co-founder of SeqOne Genomics, concluded: “Martin Dubuc is a very strong addition to our Board of Directors. Martin’s shrewd business acumen and knowledge of the pharmaceutical industry have already proven to add significant value and will continue to help us as we strengthen our pharma offering and expand internationally.”

About SeqOne Genomics:
SeqOne Genomics offers high-performance genomic analysis solutions for healthcare
providers, treating patients suffering from cancer, rare and hereditary diseases as well
as pharmaceutical companies developing new therapies. The solution supports both
short read and long-read nanopore sequencing and leverages advanced machine
learning coupled with the company’s proprietary genomics platform to dramatically
reduce turnaround times and costs while delivering comprehensive and actionable
insights for personalized medicine. The company has won numerous awards including
the iLab award and the ARC cancer foundation’s Hélène Stark prize and has twice been
nominated for the prestigious Prix Galien award. Investors include Elaia, IRDI Capital
Investissement, Merieux Equity Partners, Omnes and Software Club.

Web: https://seqone.com

Contact:
Jean-Marc HOLDER
media@seqone.com
+33 (0) 9 54 40 42 84

Seamlessly transition from Agilent Alissa Interpret genomic data analysis software to SeqOne Platform

Seamless transition from Agilent Alissa Interpret to SeqOne Platform

Discover SeqOne, the ultimate interpretation solution and Alissa Interpret alternative, compatible with Agilent wetlab!

Talk to our experts

Agilent’s genomic data interpretation software, Alissa Interpret, is slated for discontinuation in the future.

We know change can be unsettling, but at SeqOne, we’re here to make sure your transition is effortless. We’ve developed a plan that minimizes disruption to your genomic testing operations , by transitioning your genomic analysis to the proven, high performance SeqOne genomic analysis platform.

Why SeqOne is the right alternative to replace Alissa Interpret:

  • Ease of Deployment: Our user-friendly interface is designed for quick adaptation, saving you time and effort.
  • Data Compatibility: Seamlessly transfer your data from Alissa to SeqOne platform and keep performing your analysis. Your Alissa Interpret annotations data will seamlessly integrate with SeqOne, preserving your valuable work.
  • Comprehensive Variant Identification: SeqOne’s advanced algorithms ensure no critical mutations are overlooked.
  • Rapid Prioritization: Streamline your analysis with our software, providing tools to swiftly identify the most relevant mutations for each patient.
  • Certified Solution: SeqOne is a certified CE-IVD platform, but also HDS, ISO 27001 & ISO 13485, ensuring the highest standards of quality.

Your personalised Migration Plan

You have developed an efficient process for your lab and don’t want to start again from scratch. SeqOne has your back with a migration plan that minimises disruption and allows you to maintain your current laboratory workflow. Our migration plan includes:

Don’t wait – make the leap to SeqOne today for a future of uninterrupted, advanced genomic analysis. Learn more now about our NGS data analysis solution and AI-powered platform for quick variant prioritization and reporting.

Request your free demo

Genomic data interpretation workshop & lunch seminar in Graz

Discover how efficient annotation and pathogenicity classification of rare disease and cancer variants get you to a fast diagnostic result Graz Workshop

We will be hosting a workshop & lunch seminar about genomic data interpretation.

DATE: November 8th 2023 starting at 10:00, with lunch at 12:00

LOCATION: LKH-Univ. Klinikum Graz. Auenbruggerplatz 19. Seminarraum 224/2. Stock. 8036 Graz

Discover how efficient annotation and pathogenicity classification of rare disease and cancer variants get you to a fast diagnostic result!

  • Learn how to easily solve complex disease cases with a software data interpretation solution
  • Get an overview about the IVDR regulation’s impacts on the genomic data analysis
  • Bring your laptop and experience yourself the variant prioritization process on real genomic data with the SeqOne Platform

Fill out the form to register :