From a134f976c88039c32413a20af52bef741ff77ec8 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 1 May 2026 18:16:50 +0000 Subject: [PATCH] docs: add ARCHITECTURE.md with high-level and component diagrams Two Mermaid diagrams: a high-level data-flow view showing the two storage backends (sink-postgres and VictoriaMetrics), and a detailed component view of the monitoring stack, CLI, MCP, and cloud surfaces. --- ARCHITECTURE.md | 177 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 177 insertions(+) create mode 100644 ARCHITECTURE.md diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 0000000..df0b7ad --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,177 @@ +# Architecture + +Two views of the `postgresai` system: a high-level data-flow view showing **who talks to what**, and a more detailed view showing the individual components inside the monitoring stack. + +Both diagrams use Mermaid and render directly on GitHub/GitLab. + +--- + +## 1. High-level view + +Shows the main actors, the two storage backends, and how data flows between them. + +```mermaid +flowchart LR + User["👤 User
(human or AI agent)"] + CLI["postgresai CLI
checkup · mon · auth · mcp"] + Target[("🐘 Target
PostgreSQL
(the DB you monitor)")] + Stack["📦 Monitoring stack
pgwatch · Grafana · reporter
(Docker or Helm)"] + SinkPG[("Postgres sink
detailed measurements")] + SinkVM[("VictoriaMetrics sink
time-series metrics")] + Cloud["☁️ console.postgres.ai
Issues · AI analysis · auth"] + + User -->|commands| CLI + User <-->|browser / MCP| Cloud + User -->|browser| Stack + + CLI -->|read-only
diagnostic queries| Target + CLI -->|installs &
manages| Stack + CLI -->|uploads
checkup reports| Cloud + + Stack -->|scrapes metrics| Target + Stack <-->|read / write| SinkPG + Stack <-->|read / write| SinkVM + Stack -->|periodic JSON
reports| Cloud + + classDef storage fill:#fff7e6,stroke:#d48806,color:#000 + classDef cloud fill:#e6f4ff,stroke:#1677ff,color:#000 + classDef user fill:#f6ffed,stroke:#389e0d,color:#000 + class SinkPG,SinkVM storage + class Cloud cloud + class User user +``` + +**Key points** + +- One CLI is the entry point for both modes: **express checkup** (one-shot) and **full monitoring** (continuous). +- The stack uses **two storages** with different shapes: + - **Postgres sink** — stores rich measurement rows (used by the Flask API and some Grafana panels). + - **VictoriaMetrics** — Prometheus-compatible time-series store (used by most Grafana dashboards and the reporter). +- The cloud (`console.postgres.ai`) is **optional**: express checkup supports `--no-upload`, and the monitoring stack runs without an API key. + +--- + +## 2. Component view + +Same system, expanded to show the individual containers/services and how they wire up. + +```mermaid +flowchart TB + %% --- USER SIDE --- + subgraph UserSide["👤 User side"] + direction TB + Dev["Developer / AI agent"] + IDE["AI IDE
Claude Code · Cursor · Windsurf"] + Plugin["Claude Code plugin
+ MCP server"] + CLI["postgresai CLI
(Node/Bun)
checkup · mon · auth · mcp · prepare-db"] + end + + %% --- TARGETS --- + subgraph TargetSide["🐘 Target databases (PG 14–18)"] + direction TB + TargetPG[("target Postgres
+ pg_stat_statements
+ postgres_ai schema")] + Standby[("optional standby")] + end + + %% --- MONITORING STACK --- + subgraph Stack["📦 Monitoring stack — Docker Compose / Helm"] + direction TB + + subgraph Collect["Collection"] + direction LR + PgwPG["pgwatch-postgres
detailed metrics"] + PgwVM["pgwatch-prometheus
time-series metrics"] + end + + subgraph Storage["Storage"] + direction LR + SinkPG[("sink-postgres
measurements DB")] + SinkVM[("sink-prometheus
VictoriaMetrics")] + end + + subgraph Serve["Serving / consumption"] + direction LR + Grafana["Grafana
13+ expert dashboards"] + Flask["monitoring_flask_backend
pg_stat_statements API"] + Reporter["postgres-reports
(reporter, Python)
periodic JSON reports"] + end + + subgraph Self["Self-monitoring"] + direction LR + CAdv["cAdvisor"] + Node["node-exporter"] + PGExp["postgres-exporter"] + end + end + + %% --- CLOUD --- + Cloud["☁️ console.postgres.ai
Issues · AI analysis · auth · report storage"] + + %% --- EDGES: user side --- + Dev --> CLI + Dev --> IDE + IDE --> Plugin + Plugin -->|MCP / HTTPS| Cloud + CLI -->|OAuth PKCE| Cloud + + %% --- EDGES: checkup path --- + CLI -->|"read-only
diagnostic SQL"| TargetPG + CLI -->|"upload checkup JSON
(optional)"| Cloud + CLI -->|installs · starts · upgrades| Stack + + %% --- EDGES: collection --- + PgwPG -->|scrape ~60s| TargetPG + PgwVM -->|scrape ~60s| TargetPG + PgwPG -->|write| SinkPG + PgwVM -->|remote_write| SinkVM + + %% --- EDGES: serving --- + Grafana -->|SQL| SinkPG + Grafana -->|PromQL| SinkVM + Flask -->|SQL| SinkPG + Flask -->|PromQL| SinkVM + Reporter -->|PromQL| SinkVM + Reporter -->|"upload JSON
(optional)"| Cloud + + %% --- EDGES: self-monitoring --- + CAdv -->|metrics| SinkVM + Node -->|metrics| SinkVM + PGExp -->|metrics| SinkVM + PGExp -.->|scrapes| SinkPG + + %% --- EDGES: humans hitting the stack --- + Dev -->|browser :3000| Grafana + Dev -.->|HTTP API| Flask + + %% --- styling --- + classDef storage fill:#fff7e6,stroke:#d48806,color:#000 + classDef cloud fill:#e6f4ff,stroke:#1677ff,color:#000 + classDef target fill:#f9f0ff,stroke:#722ed1,color:#000 + class SinkPG,SinkVM storage + class Cloud cloud + class TargetPG,Standby target +``` + +### Component cheat sheet + +| Component | Role | +|---|---| +| **postgresai CLI** | Entry point. Runs checkups, installs the stack, manages targets, MCP install, auth. | +| **Target Postgres** | The database being observed. `prepare-db` creates a read-only `postgres_ai_mon` user and the `postgres_ai` helper schema. | +| **pgwatch-postgres** | pgwatch v3 instance writing detailed measurement rows into the Postgres sink. | +| **pgwatch-prometheus** | pgwatch v3 instance exposing time-series metrics scraped into VictoriaMetrics. | +| **sink-postgres** | Postgres database (`measurements`) holding rich, structured metric rows. | +| **sink-prometheus** | VictoriaMetrics — Prometheus-compatible TSDB. Default 14-day retention. | +| **Grafana** | 13+ expert dashboards (Node Overview, Query Analysis, Wait Events, Indexes, Tables, Replication, …). Reads from both sinks. | +| **monitoring_flask_backend** | HTTP API for `pg_stat_statements`-derived views consumed by Grafana panels. | +| **postgres-reports (reporter)** | Python service generating periodic JSON health reports; can upload to `console.postgres.ai`. | +| **Self-monitoring** | cAdvisor (containers), node-exporter (host), postgres-exporter (sink-postgres) → VictoriaMetrics. | +| **console.postgres.ai** | Managed cloud: Issues workflow, AI analysis, history, MCP backend. Optional. | +| **MCP server** | Bridges AI IDEs (Claude Code, Cursor, Windsurf) to Issues and action items in the cloud. | + +### Two storages — why both? + +- **VictoriaMetrics** is optimized for high-cardinality time-series and PromQL — the right shape for dashboards and trend reports. +- **sink-postgres** keeps richer, structured rows (e.g. full `pg_stat_statements` snapshots) that are awkward to model as time series and are easier to query with SQL from the Flask API. + +Both are populated independently by their own pgwatch instance, so one can be disabled without affecting the other.