Announcement
Thank you for attending THSNA 2026. The virtual meeting is now closed.
Thank you for attending THSNA 2026. The virtual meeting is now closed.
Presentation Details
| Multimodal Deep Learning for Pulmonary Embolism Screening Using Chest X-Ray and Electrocardiogram Data Awan Afiaz1, 2, Stephen Salerno2, Somin M Lee3, Asher Mendelson4, Barret Rush5, Matthew Samuel5, Max Tang6, Takeshi Tohyama7, Jennifer Ziegler8, Leo A Celi9, 10, 11, Jeffrey T Leek1, 2, Barbara Lam12. 1Department of Biostatistics, University of Washington, Seattle, WA, USA.2Public Health Sciences, Biostatistics, Fred Hutchinson Cancer Center, Seattle, WA, USA.3University of Toronto, Toronto, ON, Canada.4Section of Critical Care, Department of Medicine, University of Manitoba, Winnipeg, MB, Canada.5Massachusetts Institute of Technology, Cambridge, MA, USA.6University of Waterloo, Waterloo, ON, Canada.7Institute for Medical Engineering & Science, Massachusetts Institute of Technology, Cambridge, MA, USA.8Department of Medicine, University of Manitoba, Winnipeg, MB, Canada.9Laboratory for Computational Physiology, Massachusetts Institute of Technology, Cambridge, MA, USA.10Division of Pulmonary, Critical Care and Sleep Medicine, Beth Israel Deaconess Medical Center, Boston, MA, USA.11Department of Biostatistics, Harvard T.H.Chan School of Public Health, Boston, MA, USA.12Division of Hematology/Oncology, Department of Medicine, Fred Hutchinson Cancer Center, Seattle, WA, USA |
Abstract
Background: Pulmonary embolism (PE) is a life-threatening condition requiring rapid diagnosis. Current clinical decision rules such as the Wells and Geneva scores rely on subjective variables (e.g., clinician suspicion for PE) and past medical history, which are prone to delay and error. A risk prediction tool that uses objective, routinely collected data such as electrocardiograms (EKGs) and chest X-rays (CXRs) offers a promising alternative for identifying high-risk patients early in their presentation to the emergency room (ER), thus accelerating PE diagnosis and reducing mortality. Objectives: To assess whether routinely collected imaging and physiological data, specifically CXRs and EKGs, contain discriminative signals for PE by developing and evaluating a multimodal convolutional neural network (CNN). Methods: We included all adult patients in the MIMIC-IV dataset who underwent Computed Tomography Pulmonary Angiography (CTPA) imaging for suspected PE and extracted paired EKG and CXR studies that were done within 48 hours prior to CTPA. For each patient, we selected the CXR and EKG closest to the CTPA. Our final cohort included 886 patients (212 PE-positive with 23.9% prevalence). PE status was determined by CTPA, the clinical reference standard. CXR data were represented as 1,376-dimensional embeddings extracted from an EfficientNet-L2 model pre-trained on chest radiographs. EKG data were represented as raw 12-lead EKG waveforms (5,000 samples per lead, 500 Hz). We developed a multimodal fusion CNN architecture with separate convolutional branches for each modality, combining learned representations through a fully connected classification head. Models were implemented using `scorcher`, open-source software for deep learning (in R) that we are developing alongside this project. Models were trained using stratified 70/15/15 train/test/validation splits with early stopping. We report performance on a held-out validation set with 95% Wilson confidence intervals. Results: For a screening application, we prioritized high sensitivity to minimize missed PE cases. Our proof-of-concept fusion model achieved a sensitivity of 93.9% (95% CI: 80.4-98.3%) and specificity of 22.5% (95% CI: 15.5-31.6%), with negative predictive value of 92.0% (95% CI: 75.0-97.8%), an area under the receiver operating characteristic curve (AUROC) of 0.68, and area under the precision-recall curve (AUPRC) of 0.48, approximately twice the baseline prevalence of 23.9%, indicating the model provides meaningful discrimination beyond random chance. Our model achieves comparable sensitivity to existing clinical decision rules (the reported Revised Geneva score achieves 91% sensitivity and 37% specificity), using only CXRs and EKGs. Conclusions: Multimodal deep learning using EKG and CXR data can achieve PE detection sensitivity comparable to existing clinical decision rules. This approach could potentially enable automated screening, similar to existing sepsis alert systems, to flag high-risk patients presenting to the ER. Our ongoing work evaluates how vital signs and other electronic health record data might improve the prediction framework. Future directions include evaluating how such automated screening tools can complement existing clinical workflows to accelerate PE diagnosis.
No part of this publication may be reproduced, distributed, or transmitted in any form or by any means, including photocopying, recording, or other electronic or mechanical methods, without the prior written permission of the author.
No part of this publication may be reproduced, distributed, or transmitted in any form or by any means, including photocopying, recording, or other electronic or mechanical methods, without the prior written permission of the author.