01Chemistry, manufacturing and controls (CMC) is one of the key pillars the FDA regulates for a drug program. The regulatory and technical requirements often result in bottlenecks for clinical development and critical errors causing a program to fail to advance. A review of an executed batch record, carried out before each batch of a drug is released, is the only way to confirm that the intended manufacturing process was executed with the intended quality. A complete batch record contains hundreds to thousands of pages that are manually reviewed by a human reviewer, which limits how quickly an organization can scale and introduces human error. It is not hard to see how large language models could make such reviews faster and more accurate, but they have struggled to maintain accuracy over long source inputs and can state things that the source does not support.
Here we describe the development of and testing of CMC Document Processor, or CDP, a regulatory compliant system that quality checks CMC batch records. On four executed batch records from one clinical campaign (1,061 scanned, largely handwritten pages), the human reviewers raised 11 distinct actionable issues as review comments. CDP raised every one of the 11 in at least one of its runs, no missed human actionable comment was recorded, and no CDP comment cited content absent from the record. About four in five comments from the human reviewers and from CDP needed no action. Single CDP runs using Claude Opus 4.7 added 7 actionable comments that no human reviewer had raised. CDP was also more precise on non-actionable comments with 82% non-actionable comments from human reviewers versus 79% non-actionable comments from CDP. A Quality check of the records with CDP’s comments presented in advance took 9.4-fold less time than unaided manual review
02Large Language Models (LLMs) have made impressive advancements in their ability to ingest data and provide relevant information within context and in a state they can maintain across a task. Their ability to improve has been the development across engineering around the AI models, such as retrieval, evaluation harnesses, structured outputs and agent scaffolds. Despite these advancements, reviewing regulated manufacturing records using language models remains very challenging and fraught with errors. A reviewer must report only what the record supports and must account for every page, and each entry has to be checked against the document that governs it.
Current AI models have documented weaknesses on each of these points. For example, AI models can still produce fluent statements that the source does not support, a failure usually called hallucinations1 which increases as inputs get longer2,3, even when the relevant passage has been retrieved perfectly4. Long documents are therefore often split and read in parts. A reviewer then also needs to know which parts were actually read, because an answer about the parts the AI model saw looks complete whether or not the rest was read.
Manufacturing records are a prime example of hard inputs where these AI models can fail. Many are scanned images with no text layer, filled in by hand, often in multiple languages, with corrections struck through and rewritten. Instrument printouts and trend charts are bound into the record. Published benchmarks show that long scanned documents5, handwriting6 and plotted charts7 remain difficult for multimodal models, and checking a record also needs comparison across documents and arithmetic on extracted values8,9.
Drug manufacturing, also known as Chemistry, Manufacturing and Controls (CMC) is regulated under Good Manufacturing Practice (GMP). For each product, the manufacturer writes a master batch record (MBR), the approved instructions for every step, with the limits that apply to process parameters and to the tests performed during manufacture. Operators fill in a copy of the MBR as they work, and the completed copy is the executed batch record (EBR). A departure from an approved instruction or established standard is a deviation, and each deviation must be documented and explained10. Before a batch is released, a quality unit independent of production reviews the EBR against the MBR, with any deviation and investigation reports, and decides whether to release or reject the batch10,11. This review is the point at which the quality of the manufacturing documentation, and with it the quality of the product, is confirmed before patients receive it. For an active pharmaceutical ingredient, one EBR can run from hundreds to thousands of scanned, handwritten pages.
We built CDP (CMC Document Processor) to review these records with Anthropic’s Claude models. For each record, CDP retrieves the MBR and the associated reference documents such as material specifications, and sends the whole record to the AI model in batches of consecutive pages, so that no page is skipped. Its findings appear as advisory comments on the file, with a notice for any page the AI model did not evaluate, and human reviewers decide if each comment is actionable.
We evaluated CDP on four EBRs from one Phase 2 GMP clinical campaign, 1,061 scanned pages in all, which our own CMC and Quality staff and two external consultants also reviewed. One adjudicator scored every comment from the human reviews and from CDP so that the same standard was applied to every reviewer and to every AI model, and a second person confirmed the scoring of each review comment as actionable or not actionable.
To our knowledge this is the first systematic comparison of a language model’s reviews on a live clinical GMP batch with those of human experts. The focus of this paper is the method by which CDP was built, and the results are reported at the level needed to show its first-run performance on its intended use and to flag the gaps that direct its further development. Alongside the evaluation, we describe CDP’s architecture and its controls for GMP use, and the Supplementary Information gives the engineering layout in enough detail for another team to build a similar system.
03Overview
CDP consists of a database of manufacturing documents known as an electronic content management system (ECM), an orchestration layer to connect the documents to an AI model, a document classifier to determine the types of document and their relationships, a document content extractor, and document analysis tools (Fig. 1a).
Documents required by the FDA in a GMP campaign are defined as particular types, such as Certificates of Analysis, analytical methods, batch records, and specifications. In total there are 27 predefined document types that comprise the regulatory required evidence to support a GMP manufacturing campaign for use in humans.
The ECM is a document storage system that is regulated under 21 CFR Part 1112. As part of this regulation, it is critical that each document stored in the ECM cannot be created, deleted or modified without strict controls and logs that ensure the ability to discern invalid or altered records is maintained. As such, an orchestration layer was developed to maintain the validity of the ECM while being able to analyze each document.
These GMP documents are also interdependent, meaning that the standards required for each operation are located in separate documents from where the batch data is found. As an example, the certificate of analysis contains the results of material testing which must be conducted according to the specifications and procedures determined in the analytical methods document. It was then necessary to build a tool to classify each document for their types and relationships to each other.
To further add to the complicated nature of this endeavor, each record contains written records with complex formatting, such as charts, tables, and handwritten notes, as well as large document files that exceed standard ingestion limits for AI models. In order to solve this, we have also built a document content extractor to then be able to fully capture and analyze the contents in each file.
Once all of the content is extracted, we are able to then analyze each accordingly based on a rubric for grading errors and their criticality and instructions for analysis dictated by parameters set from regulations and internal documents. Finally, the analysis also contains an interface to present the findings connected to each statement in a document. It was critical to develop this feature to maintain the validity of the source documents in the ECM as unaltered and thereby maintain compliance with regulatory standards. This interface also allows the last step of a final human approval (Fig. 1b), which is also required by FDA regulations set forth in guidance documents10,11.
Every document uploaded to the ECM is classified first, so that CDP knows what the document is and which other reference documents it must be reviewed against. This takes three steps (Fig. 2a), and each one is controlled, logged and open to correction by a human reviewer.
Identifying the document type
A document uploaded to CDP triggers a call to a Claude model, which reads the opening pages and assigns one of the 27 document types in a controlled list. The AI model follows a classifier prompt that defines each type and gives rules for distinguishing similar types. For example, a “production record” and a “batch production record” are both classed as an executed batch record (Supplementary Note 7.1).
Extracting the identifiers that link documents
For each document type, a template tells the AI model which identifiers to use, such as document numbers, versions and lot numbers. These identifiers distinguish unique documents and determine the relationship of documents to each other. Code checks are used to ensure a false link cannot be created (Supplementary Note 7.1).
Finding the review reference documents for document reviews
A rule set, also held as a versioned configuration file in the ECM, states which reference documents each type must be reviewed against and how to find them. For example, an executed batch record is reviewed against the associated master batch record, material specification, and deviation records (Fig. 2b). CDP identifies an exact match for the reference document first and then tries fallbacks in a set order, such as another revision of the same document. A looser match lowers the stated confidence of the review’s outcome (Table 1). Where a wrong match would mislead, the reference is flagged as unresolved. A human reviewer has full visibility in CDP to confirm which references were used for each document review.
Table 1 | How each review reference document is found, and the cap each step places on confidence
Each reference tries its steps in order and stops at the first that succeeds. Caps are fixed rules that can only make the advisory outcome more cautious. *AI model-assisted steps are not used for batch-record references. A reference marked strict in the rule set is left unresolved rather than filled by a fallback (Supplementary Note 7.2).
Batch records are difficult documents to read. Most are scanned images with no text layer, filled in by hand and often bilingual, with corrections struck through and include attached tables and instrument charts. These records can be hundreds to thousands of pages, which is more than an AI model can carry in one request. The content extractor addresses this by sending the whole record to the AI model in batches of consecutive pages and verifies that no page is skipped (Fig. 3a).
Reading the page, not extracted text
Much of what matters in a regulatory record is written in paragraph text formats. Limits, results, and signatures sit in tables, where a value’s meaning depends on its row and column. A correction is shown by a strike-through beside the new entry, and process data are plotted as charts. Text extraction keeps the characters but loses this layout. Reading the page visually is the only approach we have found that gives the AI model the text together with its formatting and position, so CDP sends each page of the record to the AI model at full resolution.
Splitting long records without losing pages
A long record is split into windows of consecutive pages. In the configuration evaluated here, a window held at most 40 pages, a limit set by the API’s request size and its output per call. Scanned pages vary widely in size, so windows are packed by the measured size of each page. A window that is still too large is split in two rather than reduced in resolution, because handwritten entries need full resolution to be read. We have not measured how window size affects what the AI model finds. Each window is one logged AI model call. It carries the review prompt, the review reference documents found at classification, the metadata extracted at intake, and a header stating which pages of the record it holds.
Accounting for every page
Coverage is tracked in code rather than left to the AI model. Every page is assigned to exactly one window, and the application records which pages each window held. If a window fails, is cut off or is refused by both the primary and the fallback AI model, CDP posts a notice on the document naming the pages that were not read and why. The AI model that answered each window is recorded. Each window also declares content it saw but did not evaluate, such as instrument trend charts, and CDP posts these as notices with page anchors so that a reviewer knows which pages need a person to read them in more detail. These declarations come from the AI model, so a window that omits them produces no notice. The current implementation does not yet check each window’s answer against the output schema, so an unreadable answer would not be flagged. None occurred in the study (Supplementary Note 11.2).
No conclusions beyond a window’s own pages
A window sees only its own pages, so it may not state that something is missing from the record as a whole. Instead, each window lists what its pages show, such as a test result or a time span, and code stamps each entry with the window’s page range. Absences are judged across the whole record in the reconciliation step (Fig. 3b).
The review applies written rules, not the AI model’s general judgement, so that the same entry is treated the same way on every record. The rules for each document type are held in a versioned prompt in the ECM and are agreed with the CMC functions that own those documents, such as process development, analytical development, manufacturing, and quality. Because the rules decide what is worth raising on a GMP record, the quality unit has the final say on them.
What counts as a finding
For an executed batch record, a value must cross limits specified by the master batch record, the reference document used for the review by CDP. The AI model recomputes and verifies arithmetic for all entries and also checks to ensure regulatory standards are met. For example, CDP performs a check to ensure the good documentation practices are followed. This requires that corrections made in the record do not obscure the original entry, are explained, and include the identity of the person making the correction and the date the change was made14.
How each finding is recorded
Every finding has a title, a triage grade, a one-sentence statement that quotes the recorded value, the page in the record where the finding occurred, and the reference document it was reviewed against. The grades are Blocker, Major, Minor and Informational and are used to provide guidance to the human reviewers on the quality of the document review. When the AI model cannot confirm an entry it flags it as verification required rather than asserting a conclusion.
Checking findings against the whole record
Each window sees only part of the record, so a finding raised on one page may be explained on another. After the windows, a single reconciliation pass reads all the findings together with the mitigations recorded anywhere in the record, such as documented deviations, justifications and corrections (Fig. 3b). The findings are then posted as comments beside the file in the ECM.
The advisory outcome
A final AI model call proposes one of four outcomes: Recommend Release, Needs Investigation, Recommend Reject or Insufficient Data. Fixed rules then cap the proposal so that it can only become more cautious. These outcomes are advisory flags for the human reviewers who determine which action to take to address the CDP findings. After the human review of the CDP comments, the quality unit performs an additional review to determine the batch disposition (Supplementary Note 7.5).
What CDP finds depends on its configuration as much as on its code. That configuration is the review prompts and rules, the reference rule set, the switches that turn features on and off, and the choice of AI model for each task. All of these are held as versioned files in the ECM rather than written into the code. The ECM’s version history records who changed each file and when, so the configuration in force for any review can be identified afterwards. A change can be made, or reversed within minutes, without a software release.
Choosing the AI model for each task
CDP calls an AI model for several kinds of work: classifying documents and extracting their identifiers, capturing results into tables, reviewing each record and drafting correspondence. A settings file, the AI model registry, names the Claude model used for each of these tasks and a fallback AI model that answers if the primary AI model refuses. Because a fallback changes the AI model partway through a review, the AI model that answered each call is logged, and evaluation runs can switch the fallback off. The development runs reported here kept it on, and future validation will switch it off.
Testing before a change goes live
A new prompt, rule or AI model is tested before it is used in production. Testing runs either in a separate sandbox environment or in a capture-only mode that makes every AI model call but writes nothing to the reviewed file or the review logs; only a snapshot of existing comments is still saved. Each candidate is run up to three times on a fixed set of 12 records with known findings, including the four study records, and each finding is marked with how many runs raised it. The known findings come from reviewers’ decisions on earlier comments, so the test set reflects what the CMC and quality reviewers judged worth raising. On the study records, successive prompt revisions cut the number of findings by 54%, 76% and finally 79% from the baseline. These figures measure volume and do not by themselves show that real findings were kept.
Promoting an AI model change
An AI model change is run in the sandbox for about a day while the answering AI model, refusal rate and output are watched. It then requires quality sign-off and an entry in the installation-qualification register before it is applied to production. The change of AI model made during this study followed this path and was evaluated through repeated sandbox runs on the study records.
The code is covered by about 3,000 automated tests, which run each time the software is built.
Under regulatory standards such as Good Manufacturing Practices, reviewing executed records and deciding the fate of each batch are duties of the quality unit and cannot be delegated10,11. The regulations also require that a computer system’s output is checked to a degree that matches the system’s complexity and reliability11. CDP was designed so that this responsibility never moves from humans to the AI models based on the regulatory expectations listed in Table 2.
People decide; the AI model only advises
CDP posts findings as comments and proposes an advisory outcome, and nothing more. A named person, working under their own account, decides on every comment and takes every consequential action, such as accepting a record, sending it back to the manufacturer or releasing a batch. The quality unit’s deviation classification and batch disposition sit outside CDP.
The record is never changed
The ECM remains the system of record. The AI model has no tools, no credentials and no way to write to the ECM. The orchestration layer writes only comments and metadata beside each file, and renaming and moving files are switched off, so the original document and its own audit trail are untouched (Fig. 1a).
Every action can be traced
Each human action is attributed to an individual account, and the application acts under its own service identity. Every AI model call made in review, extraction and data capture is logged with the AI model, the instructions, the response and the token use, in an append-only log stored in the ECM. Configuration changes are recorded in the ECM’s version history. Together these allow any review to be reconstructed afterwards: what CDP was given, which version of its instructions it used, what it returned and what a person decided (Supplementary Note 8.3).
Data stays under control
The application runs in an isolated environment, with separate sandbox and production systems and secrets held in a vault. Each request to the AI model provider carries only the document, its review reference documents and the instructions, and uploaded copies are deleted after each review. Documents are processed under the provider’s standard data-retention terms; we do not have a zero-data-retention agreement.
Some of these controls are enforced in code, some by the hosting platform and some by written procedure, such as testing a configuration change in the sandbox or in capture-only mode before it goes live. Supplementary Note 12 states which is which. CDP is not validated as a GMP computerized system and is used as an advisory aid, however the system incorporates these principles by ensuring testing of releases, maintaining change control and audit logs, and ensuring data integrity and security. The full mapping of CDP’s GMP compliance controls is presented in Supplementary Table 4. Draft EU GMP Annex 22 would exclude large language models from critical GMP applications and, for non-critical use, asks that qualified people remain responsible for each output15. We designed and operate CDP as a non-critical, advisory application.
Table 2 | How CDP’s controls map to GMP expectations
04We asked whether CDP could review an executed batch record to the standard an auditor would hold a human reviewer to (Fig. 4). Two internal review teams composed of a CMC and Quality reviewer each reviewed two executed batch records from a four-record manufacturing campaign and logged their review comments. In addition, two external reviewers who are experienced CMC and quality consultants, jointly reviewed all four records and logged their review comments. Each reviewer received the record, its MBR and a specification, and the human reviewers could also consult earlier-step records, which CDP did not receive.
After the manual reviews were complete, a review of the same four executed batch records was performed by CDP using Claude Opus 4.7. CDP review comments were logged and reviewed by the internal review teams. Each internal review team reviewed CDP comments on the two executed batch records that they did not review manually in a cross-over design to ensure familiarity with a record could not bias their review of CDP comments.
One Quality adjudicator, who did not review any of the records, scored each human and CDP review comment according to whether an auditor reading the same record would raise it. A single adjudicator was used deliberately, so that one standard was applied to the matching of comments across the human reviews and the CDP run. The adjudicated results were then confirmed by a second person, who checked each comment’s label of actionable or not actionable and agreed with the selection. A comment was actionable if an auditor would require a correction, an explanation or a justification. The five labels were Match, Relevant addition, Missed human finding, No action required, and Duplicate (Fig. 4c). A Match meant that CDP and a human reviewer had flagged the same actionable issue, and a Relevant addition, used only for CDP, was an actionable issue that only CDP raised. The scores are descriptive and this analysis is exploratory.
The time to complete the manual human reviews and the time to complete the CDP review, including CDP processing time and the human review of CDP comments, was recorded. AI model-call cost was computed from logged tokens at list prices18 (cache writes at 1.25 times the input rate, US$6.25 per million tokens; cache reads at US$0.50) and excludes development, hosting and human time.
CDP raised every actionable issue that the human reviewers raised
The human reviewers’ actionable comments described 11 distinct actionable issues across the four records. CDP raised every one of the 11 as a review comment in at least one of its runs on these records. No comment was labelled as a Missed human finding by the adjudicator.
Nine of the actionable human findings were direct matches to CDP review comments. Additional explanation was required for two of the matches. For the cross-record weight reconciliations, CDP reported the non-reconciling charge tables as observations within the record but did not trace them to the earlier-step record as discrepancies as those earlier step records were not provided to CDP for review under this study. The match for the reactor-temperature excursions was identified in a finding by CDP as a flag for the temperature spike, but it was not specifically labeled in the review comment as an excursion outside the specified limits. The adjudicator classified both of these as intended matches between the human and CDP reviews.
Most review comments needed no action
Every reviewer, human or CDP, produced more comments than needed to perform a review to the level of identifying issues an FDA auditor would raise. Of the human reviewers’ 96 distinct comments, 79 (82%) needed no action. Of the CDP pass’s 42 distinct comments, 33 (79%) needed no action.
The comments that needed no action differed in kind between reviewers. The human reviewers’ were mostly observations about the design of the MBR, steps the MBR left without a limit, entries the reviewers could not read, corrections that good documentation practice permits, and personal preferences, all legitimate points that fall outside the standard assigned as an actionable comment in this study. CDP’s review comments that did not require action were more narrow and mechanical, such as flagging handwritten informational ranges on steps the MBR left unbounded and transcribed totals rounded under the manufacturer’s stated convention.
CDP added actionable issues that no human reviewer raised
The CDP pass on Claude Opus 4.7 raised 7 Relevant additions, actionable issues that no human reviewer had raised, and the adjudicator verified all 7 as accurate (Fig. 5a). They were detail-level items, each a correction or a question for the manufacturer. A wet-cake rinse soak ran about 8 minutes past its printed window. An IPC was marked Fail with no result entered. A yield was one percentage point above its expected range. A printed temperature set point was struck through and rewritten 10 °C lower without documented confirmation. A solvent charge target in a step narrative was left unchanged when about 30% more solvent was charged, still within the printed range. The last two asked for confirmation that a handwritten cycle-time extension, with the executed cycles inside the printed range, and verification steps marked “do not perform” were intended. CDP posted two of the seven at Minor and five at Informational, and two of these had first been graded Major and were lowered by CDP’s reconciliation pass.
No CDP comment cited content that was not in the record
The adjudicator found no comment in either run that referred to content absent from the record. This measure does not capture misreads that produce no comment. Comments that read an existing entry wrongly were counted separately as misreads, and each run had one, which it posted at Major. The CDP pass misread a struck-through handwritten IPC correction, which it worded as needing verification against source data. With one other comment, it produced the pass’s only Needs Investigation outcome.
A quality check with CDP’s comments presented in advance took 9.4-fold less time than unaided manual review
Manual review of the four records by the internal pairs was measured from session logs (Fig. 6). CDP processing and the reviewers’ Quality check of the records with the CDP pass’s comments presented in advance took 166 minutes in total, against 1,553 minutes (25.9 h) for unaided manual review (9.4-fold less). The external consultancy reported 1,380 minutes for its review, 8.3-fold more than the CDP arm, and no time log was seen. Per page, unaided manual review took 1.46 minutes and the CDP arm 0.16 minutes. In the CDP arm the reviewers worked from CDP’s comments and the pages they cited, so the time saved reflects the errors being presented to the reviewer in advance rather than any reduction in the standard of the check. CDP’s AI model calls for the 1,061 pages cost US$36.97 at list prices, or US$0.035 per page.
Similar results obtained using Claude Opus 5.5
A second run of CDP on the four executed batch records was performed using Claude Opus 5.5 as the AI model. The results using this AI model were similar to those using Claude Opus 4.7, with no missed human findings, no fabrications, and a similar rate of actionable vs. non-actionable comments. The CDP run using Opus 5.5 provided 6 actionable comments that no human reviewer raised, 4 of which were not included in the previous CDP runs. The review time for the CDP run using Claude Opus 5.5 was not recorded and is not presented in this study.
05To approve a drug, the US Food and Drug Administration (FDA) asks whether it is safe, whether it is effective and whether the methods, facilities and controls used to make it are adequate19. The third question is the subject of CMC. A drug developer must describe CMC before the first trial participant is dosed, and in more detail with each clinical phase20. CMC is a common failure mode for developers: of FDA’s first-cycle complete response letters for new drugs in 2020–2024, 74% cited CMC (quality or manufacturing) deficiencies21. In our analysis of FDA’s inspection-observation data, deficient production-record review (21 CFR 211.192) was the second most cited drug observation in every fiscal year from 2021 to 202522. FDA also lists inadequate production-record review among three common citations behind facility-related withholds of approval23.
The study was a demonstration of CDP on a real GMP campaign scored against human subject matter experts, and its results are reported to show a high degree of acceptability of performance and to direct development rather than to rank reviewers. The human reviewers raised 11 distinct actionable issues, CDP raised every one of them in at least one run, it missed none and fabricated nothing, and added 7 actionable issues that the human reviewers had not raised. The additions were detail-level points of execution and documentation. The reviewers found different things, so coverage grew most when a CDP run was added to a human review. In this study CDP added to a human review and did not replace one.
The workload grows with the program. As a program moves from Phase 2 to Phase 3 and then to commercial supply, batches and their records multiply, and manufacturing can become a bottleneck for an accelerated program24. At the rates measured here, running CDP on a 500-page CMC and quality pair and checking its comments costs under US $20 and takes about 1.3 hrs, nearly 10x speedup over the approximately 12 hours required for manual review. It is an important distinction that CDP is precise in raising meaningful issues. Comments that need no action also cost reviewers time.
CDP has been deployed on other active manufacturing campaigns, and its comment-audit ledger records 1,563 production reviews of 462 files. On a preceding campaign of 12 executed batch records (about 2,800 pages), reviewers reported that checking CDP’s output took about 2 hours, against about 30 hours projected for manual review. That campaign was not timed under study conditions. CDP’s capabilities will continue to be tested as more batches are reviewed.
In a review of another program, CDP’s stability evaluator, run retrospectively and unchanged on a one-month draft stability report from a non-GMP demonstration batch, flagged as Major an impurity that had roughly tripled at accelerated conditions. Every impurity in the report was within specification. The report’s human conclusion did not address the trend, and routine review had cleared the report. The trend was acted on, and the impurity was later out of specification at the initial time point of the GMP stability study.
An important distinction to make is that while we demonstrate that AI products, like CDP, are extremely capable and show incredible value, their development requires a cadre of domain expertise to achieve the final product. Each feature of CDP addresses a specific challenge and unique need with little technical overlap. For example the document extractor was an engineering solution necessary because of limitations on document processing, the analysis prompts were specific to CMC knowhow, and the output interface and orchestration layer was built to comply with FDA regulations. Organizations who are best primed for the opportunities that AI can present must consider the cross functional organization of teams. Furthermore, the architectural approach of CDP is not limited to just CMC, but could effectively apply for record review in other highly regulated industries, such as aviation and finance.
06We present CDP, a language-model system for quality review of executed batch records. CDP retrieves the documents that govern each record, sends every page to the AI model and reports any page it did not evaluate. Its findings are advisory comments, and reviewers decide what to do with each one. On four executed batch records, the human reviewers raised 11 distinct actionable issues, and CDP matched every one of them in at least one of its runs, with no missed human finding and no comment that cited content absent from the record. A similar share of its comments needed no action as of the human reviewers’ comments, and added 7 actionable issues that the human reviewers had not raised. A Quality check with CDP’s comments presented in advance took 9.4-fold less time than unaided manual review. Overall, the results of such an approach is a promising development for future use cases in decreasing the time, cost, and error rates in drug development.
07Data availability
The batch records and reference documents are GMP records owned by Enveda and contain proprietary data of the program that cannot be made public. They also contain personal data of manufacturing staff. Supplementary Data will include the adjudicated analysis with reviewer role, triage grade, and rationale. Comment text will be stripped of material, product, lot, document, equipment, manufacturer and personal identifiers, and exact process values will be replaced by magnitudes.
Code availability
We release open-doc-review, an independently written, generic companion implementation, under the MIT license at https://github.com/enveda/open-doc-review. It implements, in generic form, the engine’s deterministic components: the window planner, coverage ledger, downgrade-only reconciliation guardrails, confidence caps, capture-only (dry-run) mode, run merging and consistency metrics. It also contains classification, extraction, reference lookup, a per-call audit log, a cost estimator, example adapters for several content platforms and generic prompt templates, among them the redacted batch-record template in Appendix B of the Supplementary Information. Where it implements a component, the Supplementary Notes describe that component in its terms and name the module. It is not the reference implementation: that system’s production integration code, configured prompts and rule sets are proprietary, and the Supplementary Notes describe it in enough detail to rebuild it.
Competing interests
The authors are employees of Enveda, which developed CDP for its own use. CDP is not a commercial product, and the method is described here so that others can build it themselves. The adjudicator approved the internal assessment that classifies CDP as a low-risk, advisory tool.
08 09