Automating the segmentation, date extraction, and classification of multi-report PDFs in outside medical records using optical character recognition and generative artificial intelligence
Abstract Objectives Patients referred for specialized care often arrive with outside medical records (OMRs) compiled into multi-report PDFs that include imaging, pathology, and clinical notes in unstructured formats. Reviewing these records is time consuming and mentally taxing, increasing the risk of delayed care, clinician frustration, and missed information affecting quality of care. This study aimed to automate the segmentation, classification, and date extraction of scanned OMRs, with a focus on records relevant to breast cancer care. Materials and Methods We used optical character recognition (OCR) to extract machine-readable text from 1303 scanned PDF documents from 116 distinct external institutions. Gemini 1.5, a large language model (LLM), was then used to segment multi-report files into individual documents, classify them into clinically meaningful categories such as mammograms and pathology reports, and extract study dates to build diagnostic timelines. Document categories were informed by clinical workflows in a breast cancer center. Results The system achieved an F1 score of 0.95 for segmentation, 0.96 for classification, and 0.90 for date extraction. In a pilot of 45 records reviewed by clinicians, only 2 classification errors and 1 date error were reported. Clinicians estimated that the tool reduced OMR review time by 40%, improved workflow efficiency, and increased satisfaction. Discussion Our findings demonstrate that combining OCR with LLMs can significantly enhance the processing of unstructured medical records, reducing manual burden and supporting timely clinical decision-making. Conclusion This study demonstrates the successful application of OCR and LLMs for organizing scanned OMRs within a specialty clinic. By automating a previously manual process, the approach supports scalable review of incoming outside records and has potential for adaptation to other clinical workflows. Future work will focus on evaluating the system across additional specialties and institutions.
Challenges in renally eliminated medication use: Evaluating cystatin C and serum creatinine eGFR discordance
AbstractBackgroundAccurately estimating glomerular filtration rate (GFR) is crucial for dosing medications in hospitalized patients. Due to limitations of serum creatinine for GFR estimation, serum cystatin C (CysC) has been explored as an alternative functional kidney biomarker. This study assessed discordance between eGFRCr and eGFRCysC in a large sample of hospitalized patients and examined the frequency of renally eliminated medications affected by eGFR discordance.MethodsThis multisite historical study included adults hospitalized between 2011 and 2023 with CysC and serum creatinine reported within 24 h of each other. The first concurrent biomarker pair for each patient was analyzed. eGFR discordance and use of renally eliminated medications were described.Results17,718 hospitalized patients with concurrent creatinine and CysC assessments were included. The median eGFRCr was 65 mL/min, and the eGFRCysC was 46 mL/min. The median absolute difference of eGFRCr−eGFRCysC was 15 mL/min, and 7972 patients (45%) had a > 30% absolute difference. There was a significantly greater percentage of patients with an eGFR <30 mL/min based on eGFRCysC (26%) compared to eGFRCr (15%) (p < 0.001). Patients were prescribed an average of 20 medications in the 24 h surrounding the concurrent biomarker assessment. Renally eliminated medications accounted for 39% ± 13% of medication orders, and 80% of patients with eGFR discordance were prescribed five or more renally eliminated medications.ConclusionSubstantial eGFR discordance between eGFRCysC and eGFRCr was observed in hospitalized patients, which directly affects the dosing of renally eliminated medications. Further research is needed to optimize the pharmacotherapy of renally eliminated medications with discordant GFR assessments to improve medication safety and effectiveness.
Gastrointestinal Endoscopic Image Classification using a Novel Wavelet Decomposition Based Deep Learning Algorithm
2023 IEEE International Conference on Electro Information Technology (eIT) | July 25, 2023 | Volume: No volume available | Issue: No issue available | Pages: 616-621
Abstract 14758: Novel Deep Learning Algorithm to Detect Heart Failure With Preserved Ejection Fraction (hfpef) Using Computer Aided Auscultation of Phonocardiogram
Introduction: Heart failure with preserved ejection fraction (HFpEF) is an increasingly common clinical syndrome with diagnostic challenges and no effective treatment. The high heterogeneity of HFpEF clinical syndrome demands novel technologies to distinguish from heart failure with reduced ejection Fraction (HFrEF). Currently, invasive hemodynamic exercise testing is performed to evaluate for HFpEF with right heart catheterization. Hypothesis: We hypothesize that a deep learning model can noninvasively detect patients with HFpEF using phonocardiogram (PCG). Methods: Eligible patients undergoing right heart catheterization were recruited after IRB approval. Eko duo stethoscope was used to record simultaneous ECG & PCG at the mitral area on the patients prior to right heart catheterization at baseline. Each recording was 30 seconds in length with a frequency range between 250 - 5000 Hz. R-wave peak detection was performed & beat to beat PCG annotations were performed. The annotated PCG signals were converted into Mel-frequency cepstral coefficients (MFCC) and padded to account for the difference in lengths. Deep Learning model was developed using a Sequential Neural Network with 80% data to train and validate the model and 20% hold out data was used for testing accuracy of the model. Receiver operating curve (ROC) was obtained to estimate area under the curve (AUC) for performance analysis. Results: 42 patients were enrolled with 25 patients diagnosed as HFpEF by right heart catheterization and 17 were reported as controls. 2544 & 1771 individual PCG annotations for HFpEF and controls were used for model training/validation and testing. The model performed reasonably well with accuracy of 0.89, F1 score of 0.88, precision of 0.80 and recall of 0.97 on the test data. ROC was obtained with AUC of 0.94. Conclusions: The results demonstrate the ability of the deep learning model to noninvasively detect HFpEF using PCG. Larger study is needed to validate these findings.
ON THE DESIGN OF A NOVEL PHONOENTEROGRAM SENSING DEVICE USING AI ASSISTED COMPUTER-AIDED AUSCULTATION
Abstract Bowel sounds have been previously used to study intestinal motility and overall digestive health in various clinical settings. However, the blurred definition of bowel sounds and their subtypes, limited resources for interpretation, poor sensitivity, and low positive predictive value led to their restricted utility. Recent advances in artificial intelligence and machine learning have steered interest in developing unique tools using the phonoenterogram to analyze diverse bowel sounds. In our study, bowel sounds were recorded from eight healthy volunteers using the Eko Duo stethoscope. A novel deep-learning algorithm was designed to classify the recordings into baseline or prominent bowel sounds. A total of 11,210 data points (5,605 balanced sounds) were used to train and test the model to yield an accuracy of 0.895, a precision of 0.890, and a recall of 0.854 reflecting successful segregation of these sounds into respective groups. More extensive studies enrolling healthy and diseased subjects with a device specifically tailored to record bowel sounds are needed to generalize these results and determine their application in the patient population.
ITINERARY PREDICTIVE ANALYTICS: AI BASED SOFTWARE AS A MEDICAL DEVICE TO PREDICT PATIENTS’ FIRST VISIT ITINERARY FOR HEALTHCARE ADMINISTRATION
Abstract Majority of hospitals still utilize manual methods for patient scheduling and predicting future appointments, resulting in longer wait times, hospital burnout and inadequate use of resources. A variety of avenues have been explored, including priority patient routing, tele-health, neural networks for improving ER efficiency, predicting no-shows, consultation duration variations, and optimizing operating room utilization. Addressing this issue, a study was conducted using 700 pre-visit notes of pancreatic patients to determine the requirement of endoscopic or biliary procedure. Through natural language processing and traditional or transfer learning algorithms, data could directly be sent to EPIC for nurses to assess in further decision making. Performance of the models was above average with the transfer learning method outperforming the traditional method. Although limited by less dataset and fewer circumstances to test the models on, the results exposed potential for future development with the possibility of patients reporting their chief concerns, in turn analyzed by algorithms, ultimately creating a smooth and effective patient itinerary.
COUGH AUDIO SENTIMENT ANALYTICS FOR SOFTWARE AS A MEDICAL DEVICE APPLICATIONS
Abstract Chronic cough is not only one of the leading causes of seeking healthcare all over the world but also a huge emotional drain on the affected patient population. In this study, we used 24-hour cough recordings to analyze the intervening conversations for sentiment analyses to better diagnose, guide, and manage treatment in such patients. We surveyed a cough clinic and selected four subjects with active cough complaints using relevant ICD-10 codes. Subjects were given and instructed to wear a device to record cough for 24 hours and the recordings were collected at weeks 0, 4, 8, and 12 of the treatment. The collected data was preprocessed to eliminate sections with no data (sleep, silence) and the number of coughs was counted. Google search API calls were used to transcribe the audio files and NLTK’s VADER analyzer was used to classify sentiments on a scale of 0 to 1. Finally, average scores were calculated and plotted over a graph to interpret any trends. 12 weeks of cough treatment had varied results on the four subjects. We categorized the exhibited sentiments into negative, neutral, positive, and compound and noted that they also showed no general trends. Among these, the compound sentiment displayed the most erratic patterns, and the obtained results could not generate a steady trend. Further studies are required with a large cohort to collect data over a longer duration to accurately analyze the sentiments associated with chronic cough.
Deep Learning Based Classiftcation of Normal and Hepatic Fibrosis Mouse Model Using Digital Pathology Images
2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) | January 2, 2023 | Volume: No volume available | Issue: No issue available | Pages: 2308-2313