💬 Student Data Chatbot

A web-based chatbot that answers questions about a dataset of 5,000 university students using Python (Flask), scikit-learn (TF-IDF + Logistic Regression), and pandas — a natural-language interface to student score data.

✨ Features

• Question answering — ask in natural language about student counts, gender distribution, age, attendance, participation, and every score type
• Averages & extremes — "What is the average quiz score?", "Who has the highest total score?"
• Grade analytics — grade counts, pass percentage, and the full grade distribution
• Dataset statistics — one-command descriptive statistics of the whole dataset
• Robust NLP — TF-IDF + Logistic Regression intent classifier with cross-validated hyperparameters and a confidence threshold that rejects out-of-scope or gibberish input
• Clean & secure — user input is capped at 500 characters and never evaluated as code; the frontend renders messages with textContent (no HTML injection)
• Responsive UI — mobile-friendly chat with quick-reply suggestions and typing indicator

🛠️ Built With

• Flask — web framework
• scikit-learn — TF-IDF vectorizer & Logistic Regression classifier
• pandas / NumPy — data handling
• Vanilla HTML / CSS / JavaScript — chat interface

🎮 How to Use

Type a question into the chat, or tap one of the quick-reply suggestions. The bot understands phrasings like:

• "How many students are there?"
• "Who has the highest total score?"
• "What is the average quiz score?"
• "Who has the lowest attendance?"
• "How many students got grade A?"
• "What percentage of students passed?"
• "Show the grade distribution"
• "Show dataset statistics"

If the bot does not understand a question, it will say so and suggest what it can answer — it never guesses.

🚀 Getting Started

Run it locally

1. Create and activate a virtual environment

   python -m venv venv
   venv\Scripts\activate        # Windows
   # source venv/bin/activate   # Linux / macOS

2. Install dependencies

   pip install -r requirements.txt

3. Start the server

   python app.py

4. Open http://localhost:5000 in your browser

The intent model is stored in models/intent_classifier.pkl. If it is ever missing or corrupted, the app retrains it automatically on startup.

📂 Project Structure

StudentDataChatbot/
├── assets/               # Screenshots (chat demo, journal cover)
├── app.py                # Flask web server (entry point)
├── chatbot/
│   ├── __init__.py
│   ├── engine.py         # intent -> slots -> dataset query engine
│   └── model.py          # TF-IDF + LR training, CV, evaluation
├── data/
│   ├── students.csv      # cleaned dataset (5,000 students, 12 columns)
│   ├── intents.json      # 34 intents, ~900 patterns, responses
│   └── raw/              # original raw dataset export
├── models/
│   └── intent_classifier.pkl   # trained model (auto-regenerated if missing)
├── paper/                # the published journal paper (PDF)
├── scripts/
│   ├── preprocess_dataset.py   # raw -> cleaned dataset
│   ├── train_model.py          # cross-validate + train + evaluate
│   └── test_chatbot.py         # end-to-end query tests
├── static/
│   ├── css/style.css
│   ├── js/app.js
│   └── img/              # bot & user avatars
├── templates/
│   └── index.html
├── LICENSE
├── README.md
└── requirements.txt

📚 Journal & Improvements

This project is the implementation of the published paper "Pengembangan Chatbot Analisis Data Mahasiswa dengan TF-IDF dan Logistic Regression", extended and improved.

Citation: Regina Hillary, Aliya Cahyanti Wijaya, Melvin Wijaya Susanto, Kurniawan Sutanto, Marta Lenah Haryanti. "Pengembangan Chatbot Analisis Data Mahasiswa dengan TF-IDF dan Logistic Regression". Jurnal Algoritma, Logika dan Komputasi, Vol. IX No. 01, 2026, pp. 875–885.

This repository improves on the paper in several ways:

Aspect	Paper (2026)	This repository
Interface	Desktop GUI (Tkinter)	Responsive web app, deployable to PythonAnywhere
Intent accuracy	83%	~85% cross-validated, ~89% held-out
Hyperparameter tuning	Fixed/manual	GridSearchCV with stratified 5-fold CV
Evaluation	Not reported	Honest CV + held-out metrics + automated tests
Confidence handling	—	Rejects out-of-scope/gibberish input below a threshold
Grade extraction	—	Handles English articles correctly (e.g. "a B" → grade B)
Dataset	Raw export, inconsistent	Cleaned & validated (see below)

🗄️ Dataset

data/students.csv holds 5,000 students × 12 columns: Email, Gender, Age, Attendance (%), Midterm_Score, Final_Score, Assignments_Avg, Quizzes_Avg, Participation_Score, Projects_Score, Total_Score, Grade.

The raw export (data/raw/students_raw.csv) had three problems, all fixed by scripts/preprocess_dataset.py:

1. Participation scale — Participation_Score used a 0–10 scale while every other score was 0–100. Normalized to 0–100 (× 10).
2. Total_Score formula — the raw Total_Score had ~0 correlation with its components. Recomputed from a documented weighted formula: Total = 0.20·Midterm + 0.25·Final + 0.20·Assignments + 0.15·Quizzes + 0.10·Projects + 0.10·Participation.
3. Grade assignment — raw grades were randomly assigned (only ~21% consistent with the total). Derived from Total_Score using the standard Indonesian university scale: A ≥ 80, B ≥ 70, C ≥ 60, D ≥ 50, F < 50.

Checks built into the script: all scores in 0–100, no NaN rows, no duplicate rows.

🧪 Testing

The engine ships with an end-to-end test suite that sends real questions and asserts the expected facts:

python scripts/train_model.py    # cross-validate + train + evaluate
python scripts/test_chatbot.py   # 46 assertions over all intents

📝 License

This project is licensed under the MIT License — see the LICENSE file for details.

👤 Author

Melvin (@CodeMelvin)