Project · 2025
← All workSMP2DIAG
Clinical NLP classifier predicting 22 medical diagnoses from patient symptom descriptions (GretelAI dataset, 1,065 samples). Fine-tuned transformer models: BERT/ClinicalBERT (97% acc, F1=0.97) and Flan-T5 (94% on validation), outperforming classical baselines.
Overview
SMP2DIAG is a natural language processing system that predicts medical diagnoses from patient symptom descriptions. Built using the GretelAI dataset with 1,065 samples, the system fine-tunes transformer models to achieve high accuracy in medical diagnosis classification.
The problem
- Working with limited medical dataset (1,065 samples)
- Handling unstructured symptom descriptions
- Achieving high accuracy across 22 different diagnoses
What we built
- Applied text preprocessing: tokenization, lemmatization, and embeddings
- Fine-tuned BERT and ClinicalBERT models for medical domain
- Fine-tuned Flan-T5 as a text-to-text classifier
Results
- Achieved 97% accuracy with BERT/ClinicalBERT (F1=0.97)
- Reached 94% accuracy with Flan-T5 on a validation split
- Outperformed classical baselines (PPMI 90%, GloVe 85%)
How it works
The data is the GretelAI symptom-to-diagnosis set: 1,065 patient descriptions across 22 diagnoses, split 853 for training and 212 for testing. Text is cleaned, tokenized and lemmatized before the classical baselines.
Two classical baselines set the bar: PPMI co-occurrence features and pretrained GloVe vectors, each with logistic regression. BERT and ClinicalBERT are fine-tuned as classifiers, and Flan-T5 is fine-tuned to generate the diagnosis as text.
PPMI, GloVe, BERT, ClinicalBERT and a prompted LLM are all scored on the same 212-case test split, with per-class reports and confusion matrices; Flan-T5 was checked on a validation split carved from the training data. ClinicalBERT scored 97% accuracy with F1 0.97. The prompted LLM, with no fine-tuning, landed at 67%.