Abstract Details

Title MsLesionLLM: A Tool to Extract Key Radiological Metrics from Real-world Multiple Sclerosis Datasets

Topic Multiple Sclerosis

Presentation(s) P2 - Poster Session 2 (8:00 AM-9:00 AM)

Poster/Presentation
Number 1-002

Objective To (1) develop and validate msLesionLLM, an AI-powered prompt to extract information about new multiple sclerosis (MS)-related inflammatory activity from MRI reports and (2) apply the prompt to a real-world use case: MRI activity after starting B-cell depleting therapy.

Background Neuroimaging is routinely used to monitor disease activity in multiple sclerosis (MS). Artificial Intelligence (AI)-enabled large language models could be applied to efficiently analyze imaging reports and understand real-world effects of treatments.

Design/Methods In this retrospective observational study, an LLM model (Versa AI ecosystem that securely connects healthcare data with ChatGPT4) was applied to clinical MRI reports for adults with MS in a single center. The discovery phase involved iterative refinement of a prompt using 5 annotated datasets, to detect new T2-weighted lesions and contrast-enhancing lesions. The validation phase involved applying the prompt to MRI reports from adults with MS initiating B-cell depleting therapy.

Results The validation phase included 1262 notes (536 patients: 70.4% female, median age 40.4, IQR 33.4-51.1). Prompt performance in the validation cohort for the detection of new T2-weighted lesions was: 97.0% accuracy /97.4% sensitivity /95.9% specificity /98.3% positive predictive value /93.6% negative predictive value. Performance for detection of contrast enhancing lesions was: 96.8% accuracy / 96.8% sensitivity / 91.2% specificity / 97.3% positive predictive value / and 95.4% negative predictive value. When applied to the clinical use case, after the first 6 months on B-cell depleting therapy, 97.9% of all MRI reports revealed no enhancing lesions; after the first “rebaselining” scan, 97.5% of all reports revealed no new lesions.

Conclusions AI-enabled large language models can efficiently extract accurate information from unstructured imaging reports. Tools such as msLesionLLM could be applied in many other real-world, large clinical settings to answer questions relating to disease evolution and treatment response.

Authors/Disclosures
Shane Poole (UCSF) PRESENTER	Shane Poole has nothing to disclose.
Kanishka Koshal	No disclosure on file
Nikki Sisodia (University of California San Francisco)	Nikki Sisodia has nothing to disclose.
Kyra Henderson (UCSF)	Kyra Henderson has nothing to disclose.
Jaeleene Wijangco	Jaeleene Wijangco has nothing to disclose.
Danelvis Paredes, MD	Dr. Paredes has nothing to disclose.
Chelsea S. Chen	Ms. Chen has nothing to disclose.
William Rowles	Mr. Rowles has nothing to disclose.
Amit Akula (University of California, San Francisco)	Amit Akula has nothing to disclose.
Jens Wuerfel, MD (Hoffmann-LaRoche)	Dr. Wuerfel has received personal compensation for serving as an employee of MIAC AG. The institution of Dr. Wuerfel has received personal compensation in the range of $500-$4,999 for serving as a Consultant for Actelion. The institution of Dr. Wuerfel has received personal compensation in the range of $500-$4,999 for serving on a Scientific Advisory or Data Safety Monitoring board for Sanofi. The institution of Dr. Wuerfel has received personal compensation in the range of $500-$4,999 for serving on a Scientific Advisory or Data Safety Monitoring board for Roche.
Vishakha Sharma, PhD	Dr. Sharma has received personal compensation for serving as an employee of Roche Diagonostics.
Andreas Rauschecker, MD, PhD	Dr. Rauschecker has received personal compensation in the range of $0-$499 for serving as a Consultant for MRIMatch. Dr. Rauschecker has received personal compensation in the range of $500-$4,999 for serving as a Consultant for Arterys, Inc. Dr. Rauschecker has received personal compensation in the range of $500-$4,999 for serving as an Editor, Associate Editor, or Editorial Advisory Board Member for American Journal of Neuroradiology. Dr. Rauschecker has stock in MRIMatch.
Roland G. Henry, PhD (University of California, San Francisco)	Dr. Henry has received personal compensation in the range of $500-$4,999 for serving as a Consultant for MEDDAY. Dr. Henry has received personal compensation in the range of $5,000-$9,999 for serving on a Scientific Advisory or Data Safety Monitoring board for Novartis. Dr. Henry has received personal compensation in the range of $5,000-$9,999 for serving on a Scientific Advisory or Data Safety Monitoring board for Roche. Dr. Henry has received personal compensation in the range of $500-$4,999 for serving on a Speakers Bureau for Sanofi/Genzyme.
Riley Bove, MD, FAAN (University of California, San Francisco)	Dr. Bove has received personal compensation in the range of $5,000-$9,999 for serving as a Consultant for Alexion. Dr. Bove has received personal compensation in the range of $5,000-$9,999 for serving as a Consultant for Amgen. Dr. Bove has received personal compensation in the range of $5,000-$9,999 for serving as a Consultant for Genzyme-Sanofi. Dr. Bove has received personal compensation in the range of $10,000-$49,999 for serving as a Consultant for TG Therapeutics. Dr. Bove has received personal compensation in the range of $10,000-$49,999 for serving as a Consultant for EMD-Serono. Dr. Bove has received personal compensation in the range of $500-$4,999 for serving on a Scientific Advisory or Data Safety Monitoring board for Cadenza. The institution of Dr. Bove has received research support from Biogen. The institution of Dr. Bove has received research support from Eli Lilly. The institution of Dr. Bove has received research support from Novartis. The institution of Dr. Bove has received research support from Roche Genentech.

��ɫ��

��ɫ��