Custom-engineered AI speeds drug development when it is confined to tasks whose output can be checked against a primary source, and when every claim carries provenance, an audit trail, and a human sign-off. The published evidence supports AI for literature synthesis, structure prediction, data triage, and drafting. It does not support AI as a substitute for Good Laboratory Practice (GLP) studies, clinical data, or scientific judgment.
What does the published evidence say AI can do in drug development?
The strongest result is protein structure prediction. In the CASP14 blind assessment, the AlphaFold network predicted protein backbones with a median accuracy of 0.96 Å, against 2.8 Å for the next best method [1]. The public database built on it grew from a few hundred thousand predicted structures in 2021 to more than 214 million by 2024 [2].
Clinical evidence is thinner. A 2024 analysis of AI-native biotech pipelines found that AI-discovered molecules passed Phase 1 at a rate of 80 to 90 percent, well above historical norms, while Phase 2 success was about 40 percent, comparable to industry averages, on a small sample [3]. My reading is that AI is producing drug-like molecules but is not yet proving it can pick the right target.
FDA's 2023 discussion paper counted more than 100 drug and biologic submissions referencing AI or machine learning in 2021 alone, most of them in clinical development [4]. By January 2025 the agency reported more than 500 submissions with AI components since 2016 and over 800 public comments on its two 2023 discussion papers [5].
How does FDA expect a sponsor to show that an AI model is credible?
The January 2025 draft guidance, "Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products," is the first FDA guidance on AI in drug development [5]. It applies when AI output supports a regulatory decision about safety, effectiveness, or quality. It asks for evidence of credibility, defined as trust in a model's performance for a specific context of use [6].
The framework has seven steps: define the question of interest, define the context of use, assess model risk, plan how credibility will be established, execute the plan, document results and deviations, and determine whether the model is adequate for that context [6]. Model risk combines model influence, how much the AI evidence drives the decision, with decision consequence, how serious a wrong decision would be [6].
The draft guidance excludes AI used in drug discovery, and AI used for operational efficiencies such as drafting a regulatory submission, when those uses do not affect patient safety, product quality, or study reliability [6]. FDA will not police the rigor of your AI-assisted literature review. You must. The ten principles of good AI practice that FDA and the European Medicines Agency published in January 2026 point the same way: all concern process, not architecture [7].
Where does AI fail, and why should a sponsor care?
The best-documented failure is fabricated references. Walters and Wilder asked two versions of a general-purpose chatbot to write short literature reviews and checked 636 citations. Fifty-five percent of the citations from the older model were fabricated, and 18 percent from the newer one; of the real citations, 43 and 24 percent respectively contained substantive errors [8]. A regulatory document with invented references is a liability, not a time saver.
The second failure is data leakage, where information from the test set contaminates training, so reported accuracy collapses on new data. Kapoor and Narayanan found leakage affecting at least 294 papers across 17 scientific fields [9]. In cheminformatics, Wallach and Heifets showed that performance on seven widely used virtual-screening benchmarks was largely explained by overfitting rather than prospective accuracy [10].
The third is non-reproducibility at scale. Roberts and colleagues screened 2,212 papers on machine learning for COVID-19 diagnosis from chest imaging and reviewed 62 in depth. They concluded that none was of potential clinical use, citing methodological flaws, undocumented model selection, missing external validation, and biased data [11].
Which development tasks suit AI, and what control keeps each one rigorous?
The test I apply is whether a task has a ground truth I can check the output against, and whether a wrong output would be caught before it reaches a decision. Absent from the table is any use of AI as safety or efficacy evidence. GLP toxicology, assay validation, and clinical data come from experiments on the regulator's terms, and FDA's credibility framework then applies in full [6].
| Task | Where AI helps | Control that keeps it rigorous |
|---|---|---|
| Literature synthesis | Screening thousands of abstracts, extracting findings, mapping a competitive landscape | Every extracted claim links to a retrievable primary source; a scientist reads the source before the claim is used |
| Regulatory and grant drafting | First drafts of background, rationale, and summary sections built from source documents | Fully cited drafts with no statement lacking a traceable origin; author-of-record review and sign-off on each section |
| Market assessment | Assembling epidemiology, pricing analogs, and pipeline data into a model | Inputs sourced to registries, agency data, or peer-reviewed studies; assumptions listed and stress-tested |
| Structure and property prediction | Hypothesis generation for target engagement and molecular design | Experimental confirmation before any prediction enters a regulatory document |
| Data triage | Flagging outliers, duplicates, and inconsistencies in large datasets | Flags prompt human review rather than deletion; original data preserved |
What does rigor mean when a machine writes the first draft?
Provenance and an audit trail come first. Every factual statement must point to a primary source a reviewer can open, and a citation that cannot be retrieved is treated as false. The audit trail records what went in, what came out, and what a human changed. FDA's discussion paper asks for the same: recorded steps and decisions, with rationale for deviations, so oversight and auditing are possible [4].
Validation against ground truth and human review complete the set. Before relying on a workflow, I test it on questions where I already know the answer: a literature set I have read, a market model I built by hand. Then a scientist reads every cited source behind every claim that matters. The time saved is in search, extraction, and assembly, not in judgment.
How should a small biotech adopt AI without lowering its evidentiary bar?
Be honest about the tradeoffs. Workflows built this way are slower than unconstrained chatbot use, because verification takes time and requires scientific judgment in the loop, which does not scale without limit. What they buy is speed where speed is safe, and a document you can hand to an FDA reviewer or a grant panel without flinching.
In my own practice I build purpose-built AI workflows in-house for literature synthesis, fully cited regulatory and grant drafting, and market assessment. Each produces an audit trail from source to sentence, and a human reviews every claim before it is used. The tools extend what a small team can cover; they do not replace the judgment I am asked to provide.
If you are deciding where AI belongs in your program and where it does not, I can help due to my extensive experience using AI to expedite/optimize my own drug development programs. Reach me at info@hamiltonatlas.com.
References
- Jumper J, Evans R, et al. Highly accurate protein structure prediction with AlphaFold. Nature. 2021;596:583–589.
- Varadi M, Bertoni D, et al. AlphaFold Protein Structure Database in 2024: providing structure coverage for over 214 million protein sequences. Nucleic Acids Research. 2024;52(D1):D368–D375.
- Jayatunga MKP, Ayers M, et al. How successful are AI-discovered drugs in clinical trials? A first analysis and emerging lessons. Drug Discovery Today. 2024;29(6):104009.
- U.S. Food and Drug Administration. Using Artificial Intelligence & Machine Learning in the Development of Drug & Biological Products: Discussion Paper and Request for Feedback. FDA, 2023.
- U.S. Food and Drug Administration. FDA Proposes Framework to Advance Credibility of AI Models Used for Drug and Biological Product Submissions. FDA, 2025.
- U.S. Food and Drug Administration. Considerations for the Use of Artificial Intelligence To Support Regulatory Decision-Making for Drug and Biological Products. FDA, 2025.
- U.S. Food and Drug Administration. Guiding Principles of Good AI Practice in Drug Development. FDA, 2026.
- Walters WH, Wilder EI. Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports. 2023;13:14045.
- Kapoor S, Narayanan A. Leakage and the reproducibility crisis in machine-learning-based science. Patterns. 2023;4(9):100804.
- Wallach I, Heifets A. Most Ligand-Based Classification Benchmarks Reward Memorization Rather than Generalization. Journal of Chemical Information and Modeling. 2018;58(5):916–932.
- Roberts M, Driggs D, et al. Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans. Nature Machine Intelligence. 2021;3:199–217.