nl2sql-agent

A work-in-progress Python project for building a natural-language-to-SQL agent with LangGraph.

Current status

The project is currently in the offline database-understanding stage. It includes:

• configuration loading through .env → config/application.yaml → AppConfig;
• an asynchronous MySQL client that owns the engine and session factory;
• configurable MySQL connection-pool settings and semantic-catalog output paths;
• structured console/file logging with sensitive-field redaction;
• read-only extraction of MySQL tables, columns, indexes, foreign keys and bounded samples;
• sample-scoped data profiling and format candidates for each column;
• a generated, reviewable semantic catalog draft with table purpose/domain/grain, grouped indexes and key metadata, field storage-shape templates, physical and candidate logical relationships, relation validation summaries, and coverage counts.

The online LangGraph conversation workflow, SQL generation/validation and FastAPI layer are planned next. Profile metrics are based on bounded samples and must not be treated as exact whole-table statistics.

Requirements

• Python 3.13 or newer
• uv

Setup

Install the project dependencies with:

uv sync

Run

Start the current example with:

uv run python main.py

Collect the current MySQL schema and bounded samples with:

uv run python -m app.modules.database_intelligence.extraction

Profile the latest extracted snapshot with:

uv run python -m app.modules.database_intelligence.profiling

Generate a draft semantic catalog from the latest snapshot with:

uv run python -m app.modules.database_intelligence.catalog

The draft is written to config/database_catalog/generated_catalog.yaml, and a safe human-readable review view is written to config/database_catalog/database_dictionary.md. The Markdown file is derived from the YAML on every run and must not be edited as a facts source.

The catalog keeps physical facts tied to the snapshot fingerprint, records table/column semantics and logical joins derived from comments, schema and bounded samples, distinguishes scalar from denormalized identifier fields, records primary/unique/normal indexes, and excludes raw sample values. Field data_shape entries contain templates such as dates, inconsistent date strings, years, relative storage paths, integers and CSV identifiers rather than literal sensitive values. Entries with insufficient evidence remain marked for review. queryable=null/policy=review_required means the table must be approved before online NL2SQL can select it; high-sensitivity columns are blocked and medium-sensitivity columns require masking. Logical relationship entries also expose bounded-sample overlap status and review flags; even an inferred relationship with no/low sample overlap must be verified before it is used for an unrestricted JOIN.

The derived profile.json is written next to the selected snapshot under artifacts/database_analysis/, which is intentionally ignored by Git because it can contain development-database sample values.