Clinical RAG • Verifiable Citations • Safety Guardrails • Lab Interpretation • PDF Intelligence
The LLM draft is not the final answer.
MedFlow retrieves evidence, generates under evidence constraints, verifies claims and citations, checks high-risk numeric details, then decides whether to answer, caution, abstain, or redirect.
- Overview
- Why MedFlow
- Core Features
- System Architecture
- Retrieval Architecture
- Grounded Generation
- Safety Architecture
- Clinical Workspace
- Evaluation & Benchmarks
- Technology Stack
- Project Structure
- Quick Start
- API
- Future Roadmap
- Team
- Disclaimer
MedFlow is a specialized thyroid clinical AI platform designed around one principle:
A fluent medical answer is not automatically a safe or evidence-supported answer.
The platform combines:
- Retrieval-Augmented Generation (RAG)
- source-grounded clinical responses
- page-level and chunk-level citation traceability
- thyroid-specific knowledge workflows
- patient-context-aware lab interpretation
- semantic PDF search and ingestion
- structured generation
- claim-level evidence validation
- numeric / dosage safety checks
- prompt-injection and out-of-scope guardrails
MedFlow is intentionally not an unrestricted general medical chatbot. It is a domain-focused clinical AI prototype for thyroid evidence retrieval and decision-support workflows.
Traditional LLM applications often follow:
Question → LLM → Answer
MedFlow uses a controlled clinical pipeline:
Question
↓
Risk Classification
↓
Evidence Retrieval
↓
Evidence Sufficiency
↓
Grounded Generation
↓
Citation Resolution
↓
Claim Validation
↓
Numeric Safety Validation
↓
Final Safety Policy
↓
ANSWER / CAUTION / ABSTAIN / REDIRECT
A valid-looking citation does not automatically mean the claim is supported.
MedFlow separates:
Citation ID validity
≠
Citation metadata resolution
≠
Claim-to-evidence support
≠
Clinical correctness
That separation is one of the strongest architectural decisions in the project.
| Feature | What it does |
|---|---|
| Evidence-Grounded RAG Chat | Answers thyroid questions using retrieved clinical evidence |
| Page-Level Citations | Traces responses to document, page, section, and chunk |
| Source Verification Panel | Lets users inspect the exact retrieved evidence |
| Knowledge Base | Structured summaries of major thyroid diseases |
| Disease Matrix | Side-by-side clinical comparison of thyroid conditions |
| Lab Interpreter | Context-aware interpretation for different thyroid states |
| PDF Vector Search | Semantic search across curated and uploaded documents |
| Dynamic PDF Ingestion | Upload, process, index, and search new medical PDFs |
| Safety Guardrails | Detects unsafe, out-of-scope, and adversarial requests |
| Claim Validator | Labels claims as SUPPORTED / UNSUPPORTED / REVIEW_REQUIRED |
| Numeric Safety Layer | Checks doses, units, thresholds, durations, and numeric claims |
| Interactive RAG Architecture View | Shows how evidence flows through the system |
| 3D Project Portfolio | Judge/management-facing project showcase under docs/ |
flowchart TB
U[User / Clinician] --> UI[MedFlow Web UI]
UI --> API[FastAPI Backend]
API --> IG[Input Guardrail]
IG --> RC{Risk Class}
RC -->|REFUSE_REDIRECT| SR[Safe Redirect / Refusal]
RC -->|ALLOWED / NEEDS_CAUTION| QP[Query Processing]
QP --> RET[Retrieval Layer]
subgraph Retrieval
DENSE[Dense Retrieval<br/>BGE + ChromaDB]
SPARSE[Sparse Retrieval<br/>BM25]
FUSION[RRF Fusion]
DENSE --> FUSION
SPARSE --> FUSION
end
RET --> DENSE
RET --> SPARSE
FUSION --> EG[Evidence Sufficiency Gate]
EG -->|BLOCK| AB[Abstain]
EG -->|PASS / DOWNGRADE| PACK[Evidence Packaging<br/>E1..E4]
PACK --> LLM[Grounded Generation<br/>Groq / GPT-OSS]
LLM --> JSON[Structured JSON Parser]
JSON --> CE[Citation Engine]
JSON --> CV[Claim Support Validator]
JSON --> NV[Numeric / Dosage Validator]
CE --> FP[Final Safety Policy]
CV --> FP
NV --> FP
FP -->|Safe| ANS[Answer]
FP -->|Caution| CAU[Answer With Caution]
FP -->|Insufficient| AB
FP -->|Unsafe| SR
flowchart LR
PDFs[Medical PDFs] --> EXT[Text Extraction]
EXT --> CLEAN[Cleaning]
CLEAN --> CHUNK[Token-Aware Chunking]
CHUNK --> META[Metadata Enrichment]
META --> EMB[BGE Embeddings]
EMB --> DB[(ChromaDB)]
Each indexed evidence unit can preserve:
Document Name
Section
Page Number
Chunk ID
Source Tier
Retrieval Rank
Retrieval Score
flowchart LR
LABS[TSH / FT4 / FT3 + Context] --> STATE{Patient Thyroid State}
STATE --> F[Functioning Thyroid]
STATE --> P[Post-Thyroidectomy / Ablated]
STATE --> C[Congenital / Pediatric]
STATE --> U[Unknown]
F --> INT[Context-Aware Interpretation]
P --> INT
C --> INT
U --> INT
INT --> REF[Guideline-Grounded Reference]
MedFlow contains two related retrieval views and they are deliberately documented separately.
This is the configuration used for the canonical measured retrieval metrics:
Embedding Model : BAAI/bge-small-en-v1.5
Dimensions : 384
Normalization : Yes
Similarity : Cosine
Chunk Size : 200 tokens
Overlap : 0
Top-K : 4
Indexed Chunks : 1,470
Reranker : Disabled
Query instruction:
Represent this sentence for searching relevant passages:
The product/web architecture also exposes a hybrid retrieval pipeline:
Dense Retrieval (BGE + ChromaDB)
+
Sparse Retrieval (BM25)
↓
Reciprocal Rank Fusion (RRF)
↓
Unified Candidate Ranking
The hybrid path is part of the product architecture.
Important: the frozen Day 2 metrics below belong to the benchmarked retrieval configuration and should not be presented as hybrid-RRF benchmark results unless the hybrid stack is separately evaluated.
| Model | Dimensions | Outcome |
|---|---|---|
all-MiniLM-L6-v2 |
384 | Fast baseline |
all-mpnet-base-v2 |
768 | Strong but slower |
BAAI/bge-small-en-v1.5 |
384 | Selected |
BGE-small was selected because it gave a strong overall balance across:
- Hit@1
- MRR
- passage retrieval quality
- compact embedding size
- practical CPU query latency
| Configuration | Outcome |
|---|---|
| 200 tokens / 0 overlap | Selected |
| 400 tokens / 50 overlap | Lower passage precision |
| 600 tokens / 100 overlap | More context noise |
| 500-character naive chunks | Baseline |
| K | Precision@K | Hit@K |
|---|---|---|
| 3 | ~54.17% | ~81.25% |
| 4 | 53.12% | 87.50% |
| 5 | 50.00% | 87.50% |
Why K=4?
K=4 increased evidence coverage compared with K=3, while K=5 introduced additional context noise without improving Hit@K.
Tested:
cross-encoder/ms-marco-MiniLM-L-6-v2
The reranker was not retained because the tested configuration reduced retrieval quality and added significant latency.
MedFlow includes components because measurement shows they help — not because they sound sophisticated.
Provider : Groq Cloud API
Model : openai/gpt-oss-120b
Temperature : 0.0
Retrieved evidence is packaged deterministically:
[E1]
[E2]
[E3]
[E4]
The generation model is instructed to treat retrieved evidence as the clinical source of truth.
{
"answer": "Grounded answer [E1]",
"claims": [
{
"text": "Clinical claim",
"evidence_ids": ["E1"]
}
],
"evidence_status": "HIGH",
"limitations": "..."
}The LLM is not responsible for inventing final citation metadata.
The citation engine resolves evidence IDs back to:
Document
Section
Page
Chunk ID
ALLOWED
NEEDS_CAUTION
REFUSE_REDIRECT
PASS
DOWNGRADE
BLOCK
N/A_INTERCEPTED
SUPPORTED
UNSUPPORTED
REVIEW_REQUIRED
ANSWER
ANSWER_WITH_CAUTION
ABSTAIN
REDIRECT
| Layer | Responsibility |
|---|---|
| Input Guardrail | Detects injection, unsafe dosage intent, OOS queries, and patient-specific context |
| Risk Classifier | Routes requests into allowed, caution, or refusal behavior |
| Evidence Gate | Checks whether retrieval is sufficient for the requested level of detail |
| Citation Engine | Resolves evidence IDs to deterministic source metadata |
| Claim Validator | Checks whether each claim is supported by cited evidence |
| Numeric Validator | Adds stricter checking for units, doses, thresholds, and numbers |
| Safety Policy | Decides whether to answer, caution, abstain, or redirect |
Medical numbers are treated as high-risk details.
Examples:
7.2 ≠ 7
20 mg ≠ 20%
5 weeks ≠ 5 mg
1.5 mg/kg ≠ 1.5 mg
The validator is designed to inspect:
- medication quantities
- dosage units
- percentages
- ages
- durations
- lab thresholds
- ranges
- numeric claims
The web application includes:
| Module | Purpose |
|---|---|
| Home | Project overview and fast entry into MedFlow |
| RAG Chat | Evidence-grounded thyroid Q&A |
| Knowledge Base | Structured thyroid disease summaries |
| Disease Matrix | Side-by-side condition comparison |
| Lab Interpreter | Context-aware thyroid lab assessment |
| PDF Search | Semantic search across indexed documents |
| RAG Architecture | Transparent visual explanation of the pipeline |
The current evidence corpus and evaluation focus on thyroid medicine.
Supported areas include:
- Hypothyroidism
- Hashimoto's thyroiditis
- Hyperthyroidism
- Graves' disease
- Thyroid nodules
- Differentiated thyroid cancer
- Thyroid evaluation
- Thyroid surveillance
- General thyroid guideline questions
MedFlow is intentionally not an unrestricted general medical chatbot.
The canonical benchmark uses strict passage-level relevance:
Correct document
+
Expected page / passage region
Results:
| Metric | Result |
|---|---|
| Precision@4 | 53.12% |
| Hit@4 | 87.50% |
| MRR | ≈ 0.7031 |
A secondary relaxed analysis measured:
| Metric | Result |
|---|---|
| Document-Level Precision@4 | 85.94% |
| Document-Level Hit@4 | 100% |
The document-level metric is a relaxed source-level measure and is not the canonical Precision@4.
Live Groq benchmark:
7 / 7 tests executed successfully
Categories included:
- Graves treatment
- Hypothyroidism diagnosis
- Thyroid nodule evaluation
- DTC surveillance
- Insufficient evidence
- Out-of-scope query
- Unsupported pediatric dosage
Internal automated results:
| Metric | Result |
|---|---|
| Citation ID Validity | 100% |
| Citation Metadata Resolution | 100% |
| Claim Citation Coverage | 100% |
| Unknown Evidence IDs | 0 |
| Fabricated Citations | 0 |
| Malformed Structured Outputs | 0 |
| Negative-case Abstention | 3 / 3 |
These are engineering evaluation metrics, not proof of clinical accuracy.
21 / 28 passed = 75.0%
17 / 28 passed = 60.7%
Risk Classification Accuracy = 78.57%
Safe Action Accuracy = 89.29%
Automated Faithfulness = 89.88%
Mean E2E Latency = 9,076.36 ms
The regression is intentionally preserved instead of being hidden.
2 / 15 passed = 13.3%
Risk Classification Accuracy = 20.0%
Safe Action Accuracy = 53.3%
Prompt Injection Resistance = 75.0%
Unsupported Dose Blocking = 0.0%
8 / 15 passed = 53.3%
Risk Classification Accuracy = 93.33%
Safe Action Accuracy = 80.00%
Prompt Injection Resistance = 87.50%
Unsupported Dose Blocking = 50.00%
Improvement:
2/15 → 8/15
13.3% → 53.3%
+40.0 percentage points
After the first execution, the original 15-case held-out set is no longer considered unseen.
Later runs are correctly labeled:
Regression After Generalization
not:
New Unseen Held-Out
MedFlow deliberately preserves failures.
flowchart LR
A[Development Benchmark] --> B[Independent Audit]
B --> C[Held-Out Failure]
C --> D[Root-Cause Analysis]
D --> E[Generalized Fixes]
E --> F[Regression Evaluation]
This process exposed:
- phrase-specific injection rules
- patient-context detection misses
- pediatric dosage misses
- spelling / synonym gaps
- overly permissive support heuristics
- metric naming issues
- over-refusal trade-offs
This failure history is part of the engineering evidence of the project.
- Python
- Sentence Transformers
BAAI/bge-small-en-v1.5- ChromaDB
- BM25
- Reciprocal Rank Fusion (product path)
- Cross-Encoder experimentation
- Groq Cloud API
openai/gpt-oss-120b
- FastAPI
- REST APIs
- PDF ingestion services
- environment-based configuration
- authentication / application services
- HTML5
- Vanilla JavaScript
- responsive SPA
- evidence side panel
- clinical workflow screens
- interactive architecture page
- 3D project portfolio under
docs/
- Precision@K
- Hit@K
- MRR
- citation validation
- citation resolution
- claim support validation
- abstention testing
- risk classification
- safe-action evaluation
- prompt-injection testing
- held-out regression
- latency measurement
medflow/
│
├── backend/
│ ├── services/
│ ├── analytics.py
│ ├── auth.py
│ ├── database.py
│ ├── main.py
│ ├── medical_data.py
│ ├── models.py
│ ├── pdf_processor.py
│ ├── rag_engine.py
│ └── schemas.py
│
├── data/
│ └── curated thyroid medical PDFs
│
├── day1/
│ └── ingestion & baseline retrieval
│
├── day2/
│ └── retrieval optimization & benchmarking
│
├── day3/
│ └── grounded generation & citation validation
│
├── day4/
│ └── safety, guardrails & evaluation
│
├── evaluation/
│ └── benchmark datasets
│
├── results/
│ └── evaluation artifacts
│
├── docs/
│ ├── index.html
│ ├── styles.css
│ ├── app.js
│ └── assets/
│ └── 3D project portfolio assets
│
├── index.html
├── run_app.py
├── requirements.txt
├── Dockerfile
├── docker-compose.yml
├── render.yaml
└── README.md
git clone https://github.com/nadaakhatab/medflow.git
cd medflowFor a fork:
git clone https://github.com/YOUR_USERNAME/MedFlow.git
cd MedFlowpython -m venv .venv
.venv\Scripts\activatepip install -r requirements.txtCreate a local .env file:
GROQ_API_KEY=your_groq_api_key_here
GROQ_MODEL=openai/gpt-oss-120b
GENERATION_PROVIDER=groq
PORT=7860Never commit
.envor real credentials.
start_medflow.bat
or:
.\start_medflow.ps1python run_app.pyWhen the application is running:
http://127.0.0.1:7860/docs
opens the interactive OpenAPI documentation.
Typical routes:
| Method | Endpoint | Purpose |
|---|---|---|
GET |
/ |
Serve the MedFlow UI |
GET |
/health |
Backend health check |
POST |
/api/v1/query |
Clinical RAG query |
POST |
/api/v1/interpret-labs |
Thyroid lab interpretation |
GET |
/api/v1/search-docs |
Search indexed documents |
POST |
/api/v1/upload-pdf |
Upload and index a PDF |
GET |
/api/v1/imported-documents |
List imported documents |
| Stage | Status |
|---|---|
| Day 1 — Research / Ingestion | ✅ Complete |
| Day 2 — Retrieval Optimization | ✅ Complete |
| Day 3 — Grounded Generation & Citation | ✅ Complete |
| Day 4 — Safety Architecture | ✅ Implemented |
| Day 4 — Generalization Refinement | 🚧 Final refinement |
| Interactive Clinical Workspace | ✅ Implemented |
| 3D Project Portfolio | ✅ Implemented |
| Production Clinical Validation | ❌ Not claimed |
Benchmark the hybrid product path rigorously:
Dense Retrieval
+
Sparse Retrieval
+
RRF Fusion
against the frozen Day 2 baseline.
Use small child chunks for retrieval precision, then expand parent / neighboring context for generation.
Potential ranking signals:
- guideline authority
- section title
- publication year
- document type
- disease family
Add a dedicated entailment layer for:
- contradiction
- partial support
- unsupported inference
- numeric mismatch
Create a new independently authored benchmark that is:
- frozen before execution
- not tuned against
- broader in wording
- more adversarial
- clinically reviewed where possible
APPROVE
EDIT
FLAG
ESCALATE
Future deployment options:
- Ollama
- local LLMs
- private cloud
- on-premise inference
while preserving the same evidence and safety architecture.
- Adham Elsayed
- Nada Khatabb
- Nourhan Adel
- Magdy Elbassiouny
- Sandy Hisham
MedFlow does not claim:
- 100% medical accuracy
- autonomous diagnosis
- autonomous prescribing
- replacement of physicians
- production clinical readiness
- regulatory approval
Internal retrieval, citation, grounding, and safety metrics are engineering evaluation results — not clinical validation.
MedFlow is an experimental clinical decision-support and hackathon prototype.
It is not a medical device and must not be used as a substitute for professional medical advice, diagnosis, prescribing, emergency assessment, or individualized clinical judgment.