Ayub, Umair

Abstract Background Diagnosis and risk stratification in rare hematologic malignancies such as myeloproliferative neoplasms (MPNs) – polycythemia vera (PV), essential thrombocythemia (ET), and myelofibrosis (MF) – require expert review of longitudinal, heterogeneous clinical records. This process is cognitively demanding, inconsistently applied, and difficult to scale beyond tertiary centers. No automated phenotyping workflow currently exists for hematologic malignancies. Methods A HIPAA-compliant large language model (LLM) framework for phenotyping MPN was developed to integrate (i) rule-based retrieval of bone marrow biopsy reports, clinical notes, and structured laboratory results from the electronic health record (EHR); (ii) zero-shot extraction of diagnostic and prognostic variables from unstructured text using GPT-4 Turbo; (iii) a clinician-informed source-prioritization algorithm to reconcile conflicting multi-source data; (iv) WHO/ICC-criteria–based diagnostic classification; and (v) NCCN-based risk stratification using the conventional risk model for PV, IPSET-thrombosis for ET, and DIPSS, DIPSS-plus, and MIPSS70/MIPSS70+ v2 for MF. Patients were identified via MPN-related ICD-9/10 codes; cases met 2017 WHO criteria or had a hematologist-documented diagnosis, and controls did not. The cohort was split into a prompt-development set (n = 60) and a held-out test set (n = 450; 75 cases and 75 controls per disease). Ground truth was established by independent dual-clinician chart review with consensus adjudication. LLM performance was evaluated against the ground truth: variable-level extraction using accuracy, F1 score, and Cohen’s kappa; patient-level diagnostic classification using sensitivity, specificity, and Cohen’s kappa; and prognostic risk stratification (among confirmed cases) using accuracy, weighted F1 score, and quadratic-weighted Cohen’s kappa. Wilson 95% confidence intervals (CIs) were used for proportions and bootstrap 95% CIs with 500 resamples for F1 scores. Results The held-out test set included 450 patients (PV: 150; ET: 150; MF: 150) with pathology reports and structured laboratory results, and 172 patients (PV: 52; ET: 55; MF: 65) with clinical notes. From pathology reports, overall variable extraction accuracy and F1 score were 99% (95% CI, 98–100) and 1.00 (0.99–1.00) for PV, 100% (99–100) and 0.99 (0.96–1.00) for ET, and 100% (99–100) and 0.99 (0.97–1.00) for MF. From clinical notes, overall accuracy and F1 score were 96% (91–100) and 0.94 (0.85–1.00) for PV, 100% (100–100) and 1.00 (1.00–1.00) for ET, and 100% (99–100) and 0.98 (0.95–1.00) for MF. Diagnostic sensitivity was 100% (95% CI, 95.1–100.0) for PV, ET, and MF; specificity was 98.7% (92.8–99.8) for PV and 100% (95.1–100.0) for both ET and MF, with Cohen’s kappa of 0.99 for PV and 1.00 for ET and MF. Risk stratification accuracy was 100% with weighted F1 score of 1.00 and quadratic-weighted Cohen’s kappa of 1.00 across all three diseases. A pre-specified source-ablation analysis showed that pathology reports alone were sufficient for diagnosis (sensitivity 98.7% for PV, 100% for ET, 96.0% for MF; specificity 100% across all three subtypes) but inadequate for prognostication (accuracy 69.3% for PV, 93.3% for ET, 77.3% for MF). Adding clinical notes to pathology reports recovered full prognostic accuracy of 100% across all three diseases. Conclusions This first-in-class automated framework achieved expert-level performance for MPN diagnosis and risk stratification from real-world EHR data, establishing a foundation for scalable, standardized phenotyping in rare hematologic malignancies. Prospective, multi-site validation is warranted before clinical deployment.

e18607 Background: Progression to myelofibrosis (MF) is associated with worse survival and influences treatment decisions, yet predictors of fibrotic progression in polycythemia vera (PV) and essential thrombocythemia (ET) remain poorly characterized. Methods: Patients with pathologically confirmed PV or ET were identified from a retrospective chart review. Demographics (age, sex, race, ethnicity), clinical features (thrombosis and bleeding history, performance status, symptom burden, organomegaly), bone marrow pathology (cellularity, trilineage hematopoiesis, megakaryocyte proliferation and atypia, fibrosis grade), molecular markers ( JAK2 , CALR , MPL , ASXL1 , EZH2 , TET2 , SRSF2 , IDH1/2 , SF3B1 , TP53 ), cytogenetics, laboratory values, and treatment history (phlebotomy, hydroxyurea, interferon, ruxolitinib) at first bone marrow biopsy were recorded. MF progression was defined as hematologist-confirmed transformation to MF. Kaplan-Meier analyses estimated median MF-free survival (MFS), defined as time to MF progression or death. Univariable followed by multivariable Cox proportional regression were performed to identify candidate MFS predictors. The proportional hazards assumption was assessed using Schoenfeld residuals. Hazard ratios (HR) with 95% confidence intervals (CI) were computed, with p<0.05 indicating statistical significance. Results: A total of 505 PV and 198 ET patients were identified. The majority were White (PV: 92%; ET: 86%). PV patients were predominantly male (54%) whereas ET patients were predominantly female (66%). MF progression occurred in 160 (32%) PV and 32 (16%) ET patients. Death was observed in 139 (28%) PV and 10 (5%) ET patients. Median MFS was 158.5 months (95% CI: 143.0-183.0) for PV and 308.0 months (95% CI: 175.0-NR) for ET. In PV, older age at diagnosis (HR 1.04; 95% CI: 1.02-1.07; p<0.001) and shorter time since PV or ET diagnosis to biopsy (HR 0.995; 95% CI: 0.990-0.999; p=0.048) independently predicted worse MFS. In ET, bone marrow hypercellularity (HR 5.67, 95% CI: 1.40-23.0, p=0.015) independently predicted worse MFS. Conclusions: PV patients demonstrated shorter MFS compared to ET. Older age at diagnosis in PV, whereas bone marrow hypercellularity in ET was significantly associated with fibrotic progression or death. Active surveillance of at-risk patients may facilitate early detection and timely treatment initiation. Disease Variable HR (95%CI) P-value PV Age at diagnosis 1.04 (1.02-1.07) <0.001 Time since PV or ET diagnosis to biopsy 0.995 (0.990-0.999) 0.048 ET Bone marrow hypercellularity 5.67 (1.40-23.0) 0.015

e17005 Background: We previously developed a hybrid framework combining LLMs and rule-based algorithms (RBA) for automated risk stratification in localized prostate cancer (locPCa), achieving 89% accuracy. However, clinical T-stage (cTS) inference from narrative MRI reports remained the primary source of errors. Herein, we evaluated whether ensemble approaches using multiple state-of-the-art LLMs could improve cTS inference and overall NCCN risk stratification accuracy. Methods: This study included patients with locPCa presenting with at least one positive prostate biopsy and MRI report available. Building on our previously validated hybrid framework that extracts structured phenotypic variables (PSA, prostate volume, cTS, Gleason patterns, grade group, positive and examined cores) from unstructured reports using LLMs and applies NCCN-concordant RBA for risk stratification, we focused on improving cTS inference. Four HIPAA-compliant LLMs (GPT-4.1, GPT-5.1, Gemini-2.5-Pro, Gemini-2.5-Flash) were evaluated using structured zero-shot prompts iteratively refined through oncologist feedback. Majority voting was employed to generate consensus cTS predictions. Performance was evaluated using weighted accuracy. Results: A total of 358 patients were included. The median age at diagnosis was 65 years (IQR: 60-69); the majority were White (92%) and non-Hispanic (94%). cTS distribution was T2a (51%), T2c (20%), T3a (16%), T3b (7%), T2b (4%), and T4 (2%). The most prevalent risk category was unfavorable intermediate (38%) followed by favorable intermediate (20%), high risk (19%), very high risk (16%), low (5%), and very low (1%). Individual LLM accuracy for cTS inference ranged from 85-90%, with Gemini-2.5-Pro achieving highest accuracy (90%), followed by GPT-4.1 (88%), GPT-5.1 (87%), and Gemini-2.5-Flash (85%). The four-model ensemble reached consensus in 92% of cases, achieving 93% cTS inference accuracy. When integrated into NCCN risk stratification schema, single-model approaches achieved 89-92% accuracy, while ensemble consensus cases achieved 95% accuracy. Conclusions: The ensemble framework using multiple LLMs significantly improves cTS inference and achieves 95% NCCN risk stratification accuracy in consensus cases (92% of patients), representing a 6-percentage-point improvement over our previous single-model framework (89%). This strategy demonstrates that ensemble methods can overcome individual model limitations for complex clinical tasks and holds promise for reliable AI-assisted risk stratification.

e17003 Background: Accurate ascertainment of prostate cancer (PCa) disease states—biochemical recurrence (BCR), castration-sensitive prostate cancer (CSPC), and castration-resistant prostate cancer (CRPC)—is critical for automated clinical trial enrollment, quality improvement initiatives, and real-world evidence generation. Essential information for disease state ascertainment resides in unstructured EHR data, requiring labor-intensive manual abstraction that is inconsistent and not scalable. We developed an automated hybrid AI framework using large language models to identify PCa disease states from real-world EHR data. Methods: This retrospective study included patients with histopathologically confirmed PCa (2017-2022) and ≥3 years follow-up. Sequential EHR data—including PSA and testosterone levels, clinical notes, pathology and radiology reports—were retrieved across temporal points. BCR was identified using rule-based algorithms (post-radical prostatectomy [RP]: PSA ≥0.2 ng/mL ×2; post-radiation therapy [RT]: nadir + 2 ng/mL ×2). CRPC was determined using androgen deprivation therapy (ADT) timing, PSA kinetics, testosterone < 50 ng/dL, and radiographic progression per PCWG/EAU-ASCO criteria. GPT-4o with structured parametrized prompts extracted metastatic status and radiographic progression. Independent clinicians performed manual annotation for validation. The framework was developed on 10% of data with evaluation on 90%. External validation used a held-out cohort of clinical trial patients. Results: Among 200 PCa patients (median age 66 years [IQR 60-71]; 91% White, 93% non-Hispanic), the framework demonstrated high accuracy: PCa diagnosis (92%), RP dates (94%), RT dates (93%), and ADT initiation (92%). For disease state identification, BCR detection achieved 90% accuracy, castration status 93%, and metastatic state (M0/M1) 87%. External validation on trial-enrolled patients confirmed robust performance: 26/31 (84%) CRPC trial patients and 26/27 (96%) CSPC trial patients were correctly classified, validating the framework’s reliability for clinical trial matching. Conclusions: This hybrid LLM-based framework accurately identifies PCa disease states using real-world EHR data with minimal manual intervention. The system offers a scalable, reproducible approach for automated disease state identification, enabling streamlined clinical trial matching and enhanced real-world data curation in oncology.

e16502 Background: Accurate postoperative risk stratification in localized clear-cell renal cell carcinoma (ccRCC) informs prognosis, surveillance intensity, and eligibility for adjuvant systemic therapy. Although validated prognostic models (Leibovich 2018, SSIGN, and UISS) exist, they are inconsistently applied in routine practice due to workflow and documentation barriers. Using AI assisted tools within the electronic medical record (EMR) may enable rapid, standardized risk assessment and improve clinical use. Methods: We retrospectively analyzed 92 patients with localized ccRCC undergoing radical nephrectomy and postoperative oncology evaluation (December 2021–September 2025) across three Mayo Clinic sites. Pathology reports and oncology notes were extracted via structured SQL queries. Clinical and pathologic variables were parsed using a zero-shot GPT-4o pipeline. Gold-standard prognostic risk classification and adjuvant therapy eligibility were established by an expert oncology clinician using validated prognostic calculators and KEYNOTE-564 eligibility criteria. AI-generated outputs and novice manual abstraction by two first-year fellows were compared against expert assessment using accuracy, precision, recall, and F1 score. Leibovich and SSIGN were evaluated as three-class models (low/intermediate/high), and UISS as binary risk stratification (high vs non-high). Results: Among 92 patients with localized ccRCC (median age 64 years [range 41–78]), AI-generated outputs demonstrated near-perfect agreement with expert assessment for KEYNOTE-564 adjuvant immunotherapy eligibility (accuracy 97%, precision 100%, recall 97%, F1 score 98%). For three-class prognostic stratification, AI achieved perfect agreement for Leibovich disease-free survival risk (accuracy, precision, recall, and F1 all 100%), outperforming novice abstraction. For SSIGN, AI demonstrated strong multi-class performance (accuracy 91%, precision 91%, recall 93%, F1 score 92%), compared with novice abstraction (accuracy 92%, F1 score 90%). For binary UISS risk stratification, AI achieved 81% accuracy, precision 74%, recall 100%, and F1 score 85% showing a tendency towards false positives. Conclusions: This study demonstrates AI can use existing prognostic and systemic therapy eligibility to accurately risk stratify and identify patients eligible for adjuvant systemic immunotherapy post nephrectomy. For our next steps, we plan to develop a multimodal AI platform integrating pathologic, radiologic, and clinical data to improve prognostication for outcomes and predictive capabilities for patients who would benefit from adjuvant systemic therapy for more personalized medicine.

Abstract Background: Accurate staging is essential for treatment selection, prognosis assessment, and trial eligibility. While prior studies attempted automating pathological (p) staging, few address clinical (c) staging and no studies have compared LLM-derived staging with clinician staging during clinical visit and retrospective cancer registry staging. Methods: We developed a multi-agent framework using Gemini-2.0-Flash-001 and GPT-5 to (1) identify reports (2) extract key data, and (3) apply AJCC 8th edition staging criteria. LLM staging was compared with clinician documentation and registry staging in breast cancer patients. Clinician-registry agreement served as the reference standard. Two independent breast oncologists reviewed discordant cases. Results: We analyzed 122 randomly selected breast cancer patients across all three Mayo Clinic sites from 2018 - 2023. LLM performance matched or exceeded human inter-rater agreement for pathological staging (95-99.2% vs. 95-98.4%). Clinical staging was more challenging, with LLM concordance of 73-77.9% for cT and 87-89.3% for cN versus 87.8% and 91.0% clinician-registry agreement. Among 89 patients with clinician-registry concordance, LLM concordance was 98.9% for pT/pN, 79.8% for cT, and 91.0% for cN. Experts favored LLM staging in 27.8% (5/18) of cT and 25.0% (2/8) of cN discordances. Error analysis revealed LLM challenges in handling non-mass enhancements (6 cases), selecting radiologic measurements (2 cases), identifying discrete masses (2 cases), and miscellaneous (3 cases). Conclusion: LLMs achieved human-level performance for pathological staging (98%), supporting human-in-the-loop deployment. For clinical staging (79.8% cT, 91.0% cN), future work must enhance multimodal reasoning and integrate physical examination data. Prospective validation is needed to assess real-world impact. With targeted refinements and oversight, automated staging can transform registry workflows while augmenting clinical decision-making. Citation Format: Arshad Mohammed, Umair Ayub, Pooja Advani, Shakeela W. Bahadur, Amye J. Tevaarwerk, Tufia C. Haddad, Elisabeth I. Heath, Brenda J. Ernst, Ben Zhou, Cui Tao, Sara J. Holton, Karthik V. Giridhar, Irbaz B. Riaz. Automating clinical and pathological staging for breast cancer patients [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2026; Part 1 (Regular Abstracts); 2026 Apr 17-22; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2026;86(7 Suppl):Abstract nr 2744.

567 Background: Selecting adjuvant therapy after nephrectomy for renal cell carcinoma (RCC) requires reviewing scattered electronic health records (EHR) and performing accurate eligibility and risk assessments. Large Language Models (LLMs) may assist physicians by extracting key clinical data and generating evidence-based recommendations between active surveillance and adjuvant pembrolizumab. Methods: We selected RCC patients who underwent nephrectomy at Mayo Clinic in Arizona, Florida, and Minnesota between December 2021 and September 2025. Data extraction from EHRs was performed by a HIPAA-compliant LLM (GPT-5 and Gemini 2.0-Flash-001) and compared to human labels. Data on patients’ demographics, medical history, pathology, laboratory results, and imaging were collected. GPT-5 generated treatment recommendations using a structured prompt that incorporated patients’ features and relevant clinical information that is available to clinicians. Concordance between the human labels and LLM was evaluated for extraction. For treatment recommendations, agreement between the LLM’s output and physician’s documented recommendation in clinical notes was assessed, with disagreements reviewed independently by two blinded genitourinary oncologists. Results: The dataset included 85 patients, median (range) age of 65 (34-84) years; 24.7% were females. Patients who declined adjuvant treatment for personal reasons were excluded. LLM achieved an extraction accuracy of 95% for presenting symptoms, 97% for BMI and ECOG, and > 99% for pathology variables. Concordance between LLM and physician treatment recommendations was 84.7% (72/85). Among 13 discrepant cases, one oncologist agreed with the LLM in 9 cases and the other in 8. The main reasons for disagreement between the physicians and the LLM were that the LLM favored active surveillance for patients with declining renal function, whereas physicians still considered adjuvant pembrolizumab appropriate in such cases. Additionally, the LLM demonstrated hesitancy to recommend pembrolizumab for patients with low performance status. Overall, the 13 discrepant cases were nuanced and challenging, with disagreement observed between the two oncologists themselves in three instances. Conclusions: LLMs demonstrated high accuracy in extracting clinically relevant variables and generating objective, evidence-based adjuvant treatment recommendations for RCC after nephrectomy. Importantly, LLM did not generate any inappropriate or potentially harmful recommendations. Our pipeline can serve as a supportive decision tool to enhance consistency in post-nephrectomy management and assist physicians in choosing between active surveillance and adjuvant pembrolizumab. However, final treatment decisions should remain the responsibility of the treating physician after comprehensive patient evaluation.

405 Background: Clinical trials are critical to advancing prostate cancer treatment, yet patient enrollment remains a major bottleneck. Despite strong interest in AI for trial matching, real-world use remains limited due to reasons such as complex infrastructure requirements. To address this, we developed a lightweight, scalable AI framework leveraging the Google Healthcare Search API—requiring minimal technical expertise—to automate trial eligibility screening. Methods: We used a pre-trained GPT-5 large language model (LLM) to decompose the eligibility criteria of 13 prostate cancer clinical trials into 269 YES/NO questions, such that for an eligible patient of a given trial, the chart review would yield YES for inclusion and NO for exclusion questions. From 348 patients enrolled in one of the trials, we built a dataset linking each eligibility question to each patient, resulting in 4,947 patient–question pairs (2,034 inclusion pairs with YES labels and 2,913 exclusion pairs with NO labels). The task was framed as binary classification. During evaluation, for each patient–question pair, the LLM extracted search phrases from the question to retrieve relevant EHR information via the Google Healthcare Search API, then used the question and retrieved information to predict YES or NO responses. Performance was measured using accuracy, precision, recall, specificity, and F1 score. Results: Across 4,947 question–patient pairs, our framework achieved 0.81 accuracy, 0.85 precision, 0.65 recall, 0.92 specificity and a F1 score of 0.74. Results for different phases of trials is reported in Table 1, which demonstrates the approach’s robustness across trial phases. Predictions for exclusion criteria were highly precise, while inclusion criteria were more challenging. Most errors were observed in criteria involving complex nested logical conditions, interpretable clinical definitions and data not explicitly measurable or recorded in the EHR (e.g., planned treatments or clinician-assessed scores). Conclusions: This work demonstrates a practical AI framework to support clinical trial matching. The system can identify complex disease states (biochemical recurrence vs radiographic metastasis), castration status (sensitive, resistant) and genomics. Future work should focus on developing human-in-the-loop supervision and improving question decomposition to reduce errors in complex eligibility criteria. Screening performance of the proposed framework across different phases of trials. Trial Phase #Patient-Question Pairs #YES/NO labels Accuracy Precision Recall Specificity F1 score I, II, I/II 4,227 Yes: 1,735 No: 2,492 0.80 0.84 0.66 0.92 0.74 III 720 Yes: 299 No: 421 0.82 0.90 0.63 0.95 0.74 All 4,947 Yes: 2,034 No: 2,913 0.81 0.85 0.65 0.92 0.74 #YES/NO labels: The number of patient-question pairs that are labeled YES/NO among total patient-question pairs of the same row.

103 Background: Clinicians spend significant time extracting and synthesizing information to risk-stratify and make adjuvant chemotherapy (AC) recommendations for CRC. By automating parts of this process, LLMs could help reduce documentation burden and variability in decision-making. Methods: We selected 144 stage III, pMMR CRC patients treated at Mayo Clinic (2017–2021). HIPAA-compliant Gemini-2.0-Flash-001 and GPT-4o extracted clinical variables (age, neuropathy grade, pathological stage) from EHRs and generated AC recommendations. Two approaches were compared: (1) a rule-based algorithm with nested if/else statements translating variables into NCCN-based recommendations, and (2) a dynamic LLM agent generating recommendations using a structured prompt informed by NCCN guidelines. Concordance with treating physicians was assessed, with disagreements reviewed by a blinded GI oncologist. Results: The dataset included 144 patients with median age 67 (IQR 66–68), 48.6% female. The LLM pipeline extracted neuropathy grade with 100% accuracy and pathological T/N stages with 99.3% accuracy. Concordance with physician recommendations was 93.1% for the rule-based algorithm and 62.5% for the LLM agent. In discordant cases, blinded adjudication by a GI oncologist favored the algorithm and LLM agent over physicians in most cases (Table 1). Conclusions: This exploratory work shows LLM agents can reliably extract variables and generate evidence-based treatment recommendations for stage III CRC. The high rate of expert preference for LLM recommendations in discordant cases is intriguing but limited by small sample size and possible adjudicator bias. Discordant physician recommendations reflect the complexity of decision-making, including patient preferences, performance status, comorbidities, and subtle findings not fully captured in EHRs. Rather than replacing judgment, we envision LLMs as decision support tools—analogous to an advanced trainee reviewing charts and presenting evidence-based recommendations to the attending physician with ultimate authority. Larger multi-institutional studies with multiple adjudicators are needed to validate these findings and clarify AI’s role in oncology. Patient characteristics, LLM extraction accuracy, and concordance with physician recommendations. Metric Result Total patients 144 Median age (IQR) 67 (66-68) Female (%) 48.6% LLM extraction accuracy – Neuropathy grade 100% LLM extraction accuracy – Pathological T and N stages 99.3% Primary Concordance – Rule-based algorithm vs. physician 93.1% Adjudicator preference of rule-based algorithm over treating physician for discordant cases 100% Concordance – LLM agent vs. physician 62.5% Adjudicator preference of LLM agent over treating physician for discordant cases 82.0%

Abstract Background Accurate diagnosis and risk stratification in myeloproliferative neoplasms (MPNs) rely on expert interpretation of complex clinical records. Outside major academic centers, specialist expertise is scarce, and time-consuming chart reviews extend clinic visits and delay treatment decisions, underscoring the need for automated, point-of-care phenotyping for MPN patients. Methods Patients with the ICD codes of PV, ET and MF were identified from the electronic health record. Cases were defined as patients meeting the 2017 WHO diagnostic criteria for PV, ET or MF or those with a definitive MPN diagnosis determined by a hematologist. Controls were the patients who met neither the diagnostic criteria nor had a hematologist-assigned diagnosis. Two clinicians extracted demographic, clinical (symptoms, thrombotic history), laboratory (complete blood count, serum chemistry), genetic (mutations, karyotype), bone marrow morphology, diagnostic mentions and prognostic score information from unstructured clinical notes and pathology reports. Prognostic scores included the conventional risk model for PV, the International Prognostic Score of Thrombosis (IPSET) for ET and the Dynamic International Prognostic Scoring System (DIPSS), DIPSS-plus or Mutation-enhanced International Prognostic Scoring System (MIPSS-70) for MF. Discrepancies in data extraction were resolved by a consensus. A random sample of 510 patients (PV: 85, ET: 85, MF: 85, controls: 255) was split into prompt development (n=60) and test (n=450) sets for iterative workflow engineering and evaluation. The workflow included: (i) a rule-based algorithm for automated retrieval of demographic details, the first bone marrow biopsy report, the first clinical note after biopsy and the latest laboratory results within 90 days preceding biopsy (ii) a HIPAA-compliant LLM (GPT-4 turbo) for data extraction from the clinical notes and pathology reports (iii) a regular expressions-based algorithm for post-processing extracted data into structured tables, (iv) a clinician-informed source prioritization algorithm to select data from preferred sources, (v) a modified WHO-informed diagnostic characterization tool incorporating hematologist-assigned diagnoses to distinguish cases from controls and; (vi) an NCCN guidelines-based algorithm for prognostic tool selection and risk-stratification based on available data. Structured prompts were used to extract demographic, clinical, laboratory, genetic and pathological data in a zero-shot deterministic setting. The performance of automated data extraction, diagnostic characterization and prognostication was compared against the manually curated data. Accuracy and F1 scores were computed for data extraction and prognostication tasks while sensitivity and specificity were calculated to assess diagnostic performance. Results A total of 450 (PV: 150, ET: 150, MF: 150) pathology reports and 50 (PV: 52, ET: 55; MF: 65) clinical notes were identified for 450 test set patients. The test set data extraction accuracy was 95% (reports: 97%, notes: 92%) for PV, 93% (reports: 92%, notes: 94%) for ET and 98% (reports: 99%, notes 97%) for MF. The test set F1 score was 0.95 (reports: 0.96, notes: 0.93) for PV, 0.95 (reports: 0.95, notes: 0.96) for ET and 0.98 (reports: 0.99, notes: 0.97) for MF. PV patients were diagnosed with 100% sensitivity and 98% specificity whereas ET and MF patients were diagnosed with 100% sensitivity and specificity. The accuracy of risk stratification was 100% for PV and ET and 94% for MF. The F1 score for risk stratification was 1.00 for PV and ET and 0.94 for MF. Conclusions The first-in-class automated framework successfully identified pertinent clinical information, performed accurate data extraction from unstructured records and enabled clinician-guided source prioritization for accurate diagnosis and risk stratification of MPN patients. Prospective validation of the framework is required for widespread clinical implementation.