Completed Biomedical ML Research · Pejaver Lab

Rare Disease Prediction From Clinical Notes

A Mount Sinai research project using clinical notes, rare-disease ontology mappings, and BERT-based models to identify patients with rare diseases from MIMIC-IV.

GitHub repository View presentation

Data

1M+ clinical notes

Model

ClinicalBERT pipeline

Result

about 85% AUC

Lab

Pejaver Lab, Icahn School of Medicine at Mount Sinai

Professor Vikas Pejaver

Mentor

Mohammed Ghouse Syed

Dates

May 2025 - Aug 2025

Pipeline

1Filter Orphanet diseases with exact ICD mappings

2Find first diagnosis dates from MIMIC-IV ICD records

3Keep only clinical notes before diagnosis to reduce leakage

4Match controls by age, gender, and race

5Fine-tune ClinicalBERT-style models over note windows

6Evaluate AUC and inspect highlighted evidence tokens

Project Summary

Rare diseases are difficult to detect early because useful evidence is often scattered through long, messy clinical notes. The project turned unstructured MIMIC-IV notes into a structured patient-level dataset by using Orphanet disease information, ICD mappings, diagnosis timing, and matched controls.

I filtered notes to include only text written before the rare disease diagnosis, then fine-tuned ClinicalBERT-style models using overlapping token windows to handle long notes. The final BERT-based pipeline delivered roughly 85% AUC in identifying rare-disease patients from MIMIC-derived experiments.

Beyond prediction, the project explored interpretability: extracting high-attention tokens, mapping them back to words in notes, and producing highlighted evidence lists that a clinical collaborator could inspect.

Core Components

Clinical-note dataset

Processed 1M+ notes from MIMIC-IV and grouped pre-diagnosis notes into patient-level text blocks for modeling.

Rare-disease mapping

Used Orphanet rare-disease metadata and exact ICD-10 to ICD-9 mappings to construct a structured disease cohort.

BERT pipeline

Fine-tuned ClinicalBERT with sliding 512-token windows, focal loss, and consistent train/validation/test splits.

Interpretability

Built attention-based note highlighting to surface the words and phrases influencing rare-disease predictions.

Clinical NLP

The hard part was making noisy notes usable without leaking the diagnosis.

The project was not just a model fine-tuning exercise. A large part of the work was cohort definition: mapping rare diseases, choosing exact ICD matches, identifying the first diagnosis date, and ensuring the model only saw notes from before that point.

Data source: MIMIC-IV clinical notes and diagnosis records, with patient-level aggregation before first diagnosis.
Disease mapping: Orphanet rare-disease metadata, exact ICD-10 mappings, and exact ICD-10 to ICD-9 General Equivalence Mappings.
Modeling: ClinicalBERT fine-tuning with 512-token sliding windows, 256-token stride, focal loss, and early stopping.
Evaluation: Consistent splits and cross-validation experiments, with AUC used as the main discrimination metric.

Presentation

The project presentation is embedded below and available as a direct PDF.