Medical Imaging Classifiers

ClinicalPilot integrates 4 specialized medical imaging classifiers powered by deep learning, each deployed as a standalone Streamlit app and embedded via iframe or direct link.

Available Classifiers

Lung Disease Classifier

AI-powered lung disease classification from chest X-rays

Chest Disease Classifier

AI chest disease detection and analysis

AI Retina Analyser

Diabetic retinopathy detection from retinal images

Skin Cancer Detector

Skin lesion classification for melanoma screening

Classifier Kit Architecture

All classifiers are external Streamlit apps hosted independently. ClinicalPilot embeds them via:

Iframe Embedding (primary method)
Direct Link Redirect (fallback if embedding blocked)

Why External? Each classifier is a full deep learning model (50-500MB). Hosting them as separate Streamlit apps allows:

Independent scaling
GPU acceleration (Streamlit Cloud GPU)
No ClinicalPilot backend dependencies

API Integration

List Available Classifiers

cURL

curl -X GET https://api.clinicalpilot.ai/api/classifiers

{
  "classifiers": [
    {
      "name": "Lung Disease Classifier",
      "description": "AI-powered lung disease classification from chest X-rays",
      "url": "https://lung-disease-classification.streamlit.app/",
      "icon": "🫁",
      "embed": true
    },
    {
      "name": "Chest Disease Classifier",
      "description": "AI chest disease detection and analysis",
      "url": "https://caryaai.streamlit.app/",
      "icon": "🫀",
      "embed": true
    },
    {
      "name": "AI Retina Analyser",
      "description": "Diabetic retinopathy detection from retinal images",
      "url": "https://retinopathy-detection.streamlit.app/",
      "icon": "👁️",
      "embed": true
    },
    {
      "name": "Skin Cancer Detector",
      "description": "Skin lesion classification for melanoma screening",
      "url": "https://skincancer-detection.streamlit.app/",
      "icon": "🔬",
      "embed": true
    }
  ]
}

Backend Implementation

backend/main.py:327

@app.get("/api/classifiers")
async def get_classifiers():
    """List available medical image classifier tools."""
    return {
        "classifiers": [
            {
                "name": "Lung Disease Classifier",
                "description": "AI-powered lung disease classification from chest X-rays",
                "url": "https://lung-disease-classification.streamlit.app/",
                "icon": "🫁",
                "embed": True,
            },
            {
                "name": "Chest Disease Classifier",
                "description": "AI chest disease detection and analysis",
                "url": "https://caryaai.streamlit.app/",
                "icon": "🫀",
                "embed": True,
            },
            {
                "name": "AI Retina Analyser",
                "description": "Diabetic retinopathy detection from retinal images",
                "url": "https://retinopathy-detection.streamlit.app/",
                "icon": "👁️",
                "embed": True,
            },
            {
                "name": "Skin Cancer Detector",
                "description": "Skin lesion classification for melanoma screening",
                "url": "https://skincancer-detection.streamlit.app/",
                "icon": "🔬",
                "embed": True,
            },
        ]
    }

Classifier Details

Lung Disease Classifier
Chest Disease Classifier
AI Retina Analyser
Skin Cancer Detector

Lung Disease Classifier

URL: https://lung-disease-classification.streamlit.app/Model: Convolutional Neural Network (CNN) trained on chest X-ray datasetsDetects:

Pneumonia (bacterial, viral)
COVID-19
Tuberculosis
Lung cancer
Normal lung

Input: Chest X-ray (PA or AP view)Output:

Classification label
Confidence score (%)
Heatmap overlay (Grad-CAM)

Usage Workflow

Upload Chest X-Ray

Drag-and-drop DICOM, PNG, or JPEG file

Model Inference

CNN processes image in ~2-5 seconds

Review Results

Label: e.g., “Pneumonia (Bacterial)”
Confidence: e.g., 87%
Heatmap: Shows which lung regions influenced the prediction

Clinical Correlation

AI is not diagnostic. Always correlate with clinical presentation, labs, and radiologist review.

Example Output

Classification: Pneumonia (Bacterial)
Confidence: 87%
Region of Interest: Right lower lobe (Grad-CAM highlights)

Recommendations:
- Correlate with clinical symptoms (fever, cough, sputum)
- Check WBC, CRP, procalcitonin
- Consider sputum culture
- Initiate empiric antibiotics per local guidelines

AI Retina Analyser

URL: https://retinopathy-detection.streamlit.app/Model: ResNet-50 trained on Kaggle Diabetic Retinopathy dataset (88,000+ images)Detects:

No Diabetic Retinopathy
Mild NPDR
Moderate NPDR
Severe NPDR
Proliferative Diabetic Retinopathy (PDR)

Input: Fundus photography (color retinal image)Output:

Severity grade (0-4)
Confidence score
Referral recommendation

Clinical Integration

Diabetic Patient Screening

Annual retinopathy screening for all diabetics (ADA guidelines)

Upload Fundus Image

Taken with fundus camera or smartphone adapter

AI Grading

Grade: Moderate NPDR (Grade 2)
Confidence: 91%
Findings: Microaneurysms, dot-blot hemorrhages, hard exudates

Referral Decision

No DR / Mild NPDR: Annual screening
Moderate NPDR: Ophthalmology referral within 3 months
Severe NPDR / PDR: Urgent ophthalmology referral (1-2 weeks)

Sensitivity: 87% for referable diabetic retinopathy (moderate NPDR or worse)

Skin Cancer Detector

URL: https://skincancer-detection.streamlit.app/Model: EfficientNet-B3 trained on HAM10000 dataset (10,000+ dermatoscopic images)Detects:

Melanoma
Basal Cell Carcinoma (BCC)
Squamous Cell Carcinoma (SCC)
Actinic Keratosis
Seborrheic Keratosis
Benign nevi
Dermatofibroma

Input: Dermoscopy image or smartphone photo (close-up, good lighting)Output:

Classification (7 categories)
Malignancy risk (high/medium/low)
Biopsy recommendation

Workflow Example

Patient Presents with Suspicious Lesion

Chief complaint: “New mole on back, growing”

Capture Dermoscopy Image

Use dermatoscope or smartphone with macro lens

AI Classification

Classification: Melanoma (Malignant)
Confidence: 78%
Risk: HIGH
ABCDE Criteria:
- Asymmetry: Present
- Border irregularity: Yes
- Color variation: Multiple shades
- Diameter: 8mm
- Evolution: Growing per patient

Clinical Action

Biopsy Indicated: Excisional biopsy with 2mm margins. Refer to dermatology or surgical oncology within 1 week.

ABCDE Melanoma Criteria

Criterion	Description	AI Detection
Asymmetry	One half unlike the other	Geometric analysis
Border	Irregular, scalloped, notched	Edge detection
Color	Multiple colors (tan, brown, black, red, white, blue)	Color histogram
Diameter	>6mm (pencil eraser size)	Bounding box measurement
Evolution	Changing size, shape, color	Requires serial images

Frontend Integration

The ClinicalPilot frontend displays classifiers in a modal with iframe embedding:

frontend/index.html (excerpt)

// Fetch classifiers
fetch('/api/classifiers')
  .then(res => res.json())
  .then(data => {
    const classifiers = data.classifiers;
    // Render as cards
    classifiers.forEach(c => {
      const card = document.createElement('div');
      card.innerHTML = `
        <div class="classifier-card" onclick="openClassifier('${c.url}')">
          <span>${c.icon}</span>
          <h3>${c.name}</h3>
          <p>${c.description}</p>
        </div>
      `;
      container.appendChild(card);
    });
  });

// Open classifier in modal
function openClassifier(url) {
  const modal = document.getElementById('classifier-modal');
  const iframe = document.getElementById('classifier-iframe');
  
  // Try iframe embedding
  iframe.src = url;
  modal.style.display = 'block';
  
  // Fallback: if iframe blocked, redirect
  iframe.onerror = () => {
    window.open(url, '_blank');
    modal.style.display = 'none';
  };
}

Some Streamlit apps set X-Frame-Options: DENY, blocking iframe embedding. The frontend automatically falls back to opening the classifier in a new tab.

Limitations & Disclaimers

Medical AI Limitations:

Not Diagnostic: Classifiers provide decision support, not definitive diagnosis.
Requires Clinical Correlation: Always interpret results in context of patient history, exam, and labs.
Radiologist Review: Imaging AI does not replace radiologist interpretation.
Dataset Bias: Models trained on public datasets may underperform on populations underrepresented in training data.
Liability: ClinicalPilot and classifier authors are not liable for clinical decisions based on AI output.

FDA/Regulatory Status

These classifiers are research/educational tools, not FDA-cleared medical devices. Do not use for clinical decision-making without radiologist/dermatologist confirmation.

Best Practices

High-Quality Images

Use proper positioning, lighting, and resolution. Blurry or poorly exposed images reduce accuracy.

Clinical Context

Provide patient history to the interpreting physician (e.g., “AI flagged pneumonia, patient has fever and cough”).

Serial Comparison

Compare AI results to prior imaging when available. Evolution over time is critical.

Document AI Use

Note in the medical record: “AI classifier suggested [finding]. Confirmed by radiology review.”

Adding Custom Classifiers

You can extend the classifier kit by:

Deploy your Streamlit app (Streamlit Cloud, AWS, etc.)
Add to backend list (backend/main.py:327):

{
    "name": "Brain MRI Tumor Detector",
    "description": "Glioblastoma detection from T1-weighted MRI",
    "url": "https://your-streamlit-app.streamlit.app/",
    "icon": "🧠",
    "embed": True,
}

Restart backend — classifier now appears in frontend

Performance & Scaling

Classifier Latency

Classifier	Model Size	Inference Time	Hosting
Lung Disease	120 MB	2-5s	Streamlit Cloud (CPU)
Chest Disease	180 MB	3-6s	Streamlit Cloud (CPU)
Retina Analyser	90 MB	2-4s	Streamlit Cloud (CPU)
Skin Cancer	50 MB	1-3s	Streamlit Cloud (CPU)

For production with high volume, consider:

GPU hosting (Streamlit Cloud Pro, AWS EC2 G5 instances)
Model optimization (TensorRT, ONNX Runtime)
Batch processing for multiple images

Next Steps

Full Analysis

Run multi-agent clinical assessment after imaging review

Human-in-the-Loop

Radiologist feedback workflow for re-analysis

Get Started

Core Concepts

Guides

Medical Imaging Classifiers

Available Classifiers

Lung Disease Classifier

Chest Disease Classifier

AI Retina Analyser

Skin Cancer Detector

Classifier Kit Architecture

API Integration

List Available Classifiers

Backend Implementation

Classifier Details

Lung Disease Classifier

Usage Workflow

Example Output

Chest Disease Classifier (CaryaAI)

Use Cases

AI Retina Analyser

Clinical Integration

Skin Cancer Detector

Workflow Example

ABCDE Melanoma Criteria

Frontend Integration

Limitations & Disclaimers

FDA/Regulatory Status

Best Practices

High-Quality Images

Clinical Context

Serial Comparison

Document AI Use

Adding Custom Classifiers

Performance & Scaling

Classifier Latency

Next Steps

Full Analysis

Human-in-the-Loop

Build docs developers (and LLMs) love

Get Started

Core Concepts

Guides

​Available Classifiers

Lung Disease Classifier

Chest Disease Classifier

AI Retina Analyser

Skin Cancer Detector

​Classifier Kit Architecture

​API Integration

​List Available Classifiers

​Backend Implementation

​Classifier Details

​Lung Disease Classifier

​Usage Workflow

​Example Output

​Chest Disease Classifier (CaryaAI)

​Use Cases

​AI Retina Analyser

​Clinical Integration

​Skin Cancer Detector

​Workflow Example

​ABCDE Melanoma Criteria

​Frontend Integration

​Limitations & Disclaimers

​FDA/Regulatory Status

​Best Practices

High-Quality Images

Clinical Context

Serial Comparison

Document AI Use

​Adding Custom Classifiers

​Performance & Scaling

​Classifier Latency

​Next Steps

Full Analysis

Human-in-the-Loop

Build docs developers (and LLMs) love

Available Classifiers

Classifier Kit Architecture

API Integration

List Available Classifiers

Backend Implementation

Classifier Details

Lung Disease Classifier

Usage Workflow

Example Output

Chest Disease Classifier (CaryaAI)

Use Cases

AI Retina Analyser

Clinical Integration

Skin Cancer Detector

Workflow Example

ABCDE Melanoma Criteria

Frontend Integration

Limitations & Disclaimers

FDA/Regulatory Status

Best Practices

Adding Custom Classifiers

Performance & Scaling

Classifier Latency

Next Steps