Skip to content

Extract multitenancy into a SharedTables decorator with safe withTenant scoping#11

Open
loks0n wants to merge 3 commits into
mainfrom
feat/tenant-per-row-and-construct-config
Open

Extract multitenancy into a SharedTables decorator with safe withTenant scoping#11
loks0n wants to merge 3 commits into
mainfrom
feat/tenant-per-row-and-construct-config

Conversation

@loks0n

@loks0n loks0n commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Why

The ClickHouse adapter baked multi-tenancy in via mutable setNamespace/setSharedTables/setTenant setters — and each flushed the buffer. setTenant per message (multi-tenant workers) meant one ClickHouse round-trip per message, which in Appwrite Cloud's stats-usage worker capped throughput at ~1 message per cross-region round-trip and OOM'd the queue.

This pulls multitenancy out of the core adapter entirely so the failure pattern isn't even expressible, and gives reads a tenant scope that's safe under concurrency.

Design

Base ClickHouse is now strictly single-tenant — no tenant column, no tenant SQL. Multi-tenancy lives in a SharedTables subclass that controls SQL generation through protected seams on the base:

seam role
tenantColumnDefs() tenant column on every table
keyPrefix() tenant as the leading ORDER BY / projection / GROUP BY key
extraQueryableColumns() tenant accepted as a query attribute
scopeOnlyAttributes() tenant scopes but never narrows a daily purge
scopeQueries() inject the active tenant filter on reads/purges
decorateRow() stamp the per-row tenant on writes

Tenant is no longer adapter state, nor a parameter on every read method. Instead:

  • Reads/purges scope through SharedTables::withTenant($tenant, fn($scoped) => …), which hands the callback a tenant-bound clone — no shared mutable state, safe under concurrent coroutines, and the scope can't leak past the closure. Usage::withTenant mirrors it for the wrapper.
  • Writes keep the per-row $tenant on collect(), so one buffer still batches many tenants in a single flush (the throughput win).
// write: many tenants, one buffer, one flush
$usage->collect('network.requests', 5, Usage::TYPE_EVENT, $tags, tenant: $projectId);

// read: scoped, safe, auto-cleared
$total = $usage->withTenant($projectId, fn (Usage $u) =>
    $u->getTotal('network.requests', [], Usage::TYPE_EVENT));

Notable

  • Daily-purge safety fix: tenant is treated as a scope, not a narrowing filter, so a path-only purge can't fall through to DELETE … WHERE tenant=X and wipe a tenant's unrelated daily rows.
  • Database adapter reverted to no tenant params (keeps its own backend tenant handling).

Migration (breaking)

  • new ClickHouse(host, user, pass, port, secure)new SharedTables(host, user, pass, port, secure, namespace) for multi-tenant; base ClickHouse is single-tenant.
  • ->setNamespace()/->setSharedTables()/->setTenant() removed; namespace is a constructor arg.
  • Reads: scope via withTenant(). Writes: collect(..., tenant: $t).

Tests

247 tests green against ClickHouse + MariaDB (docker-compose); PHPStan level max and Pint clean on src and changed tests. Added explicit withTenant override + cross-tenant isolation tests. Shared-mode tests pin a tenant via a ScopedClickHouse extends SharedTables helper, still exercising the real tenant SQL.

🤖 Generated with Claude Code

loks0n and others added 2 commits June 22, 2026 12:39
…xplicitly

Namespace and shared-tables mode are deployment-fixed, yet were mutable
setters on the adapter — and each setter flushed the buffer. Tenant was
likewise adapter state, so writing one project's metrics required a
setTenant()+flush() per message (one ClickHouse round-trip each).

- ClickHouse adapter takes namespace + sharedTables as constructor args;
  setNamespace/setSharedTables/setTenant setters and their getters removed,
  along with all adapter tenant state.
- collect() gains an optional per-row $tenant, stored on the buffer entry
  and folded into the dedup key, so one buffer holds many tenants and
  flushes without per-tenant round-trips.
- Read/purge methods take an explicit ?string $tenant, applied as a normal
  query filter (parseQueries already supports the 'tenant' attribute) — no
  hidden adapter state. Base Adapter + Database adapter signatures updated
  to match; Database honours it via its existing setTenant.

PHPStan max clean. Test migration to the new signatures follows in the
next commit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ily purge

- ClickHouse tests construct with namespace/sharedTables and use a
  ScopedClickHouse test helper that supplies a default tenant to writes
  (per-row) and reads (per-call), so the existing call sites exercise the
  real tenant code paths without threading tenant through each one.
- Rewrote the per-row tenant-override test to use the new API directly
  (addBatch with $tenant on the row, find/purge with an explicit tenant).
- Reflection helpers target the parent ClickHouse class so they resolve
  private members through the test subclass.

Also fixes a purge regression surfaced by the migration: with tenant now an
explicit query filter, purgeDaily treated a tenant scope as a daily-safe
narrowing filter, so a path-only purge fell through to
`DELETE FROM daily WHERE tenant=X` and wiped the tenant's unrelated daily
rows. Tenant is now excluded from the compatibility / no-op decision and
re-attached only to the final scoped delete.

246 tests green, PHPStan max + Pint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Jun 22, 2026

Copy link
Copy Markdown

Greptile Summary

This PR extracts multi-tenancy out of the core ClickHouse adapter and into a new SharedTables subclass, fixing a critical throughput issue where setTenant() flushed the buffer on every call. Tenant state is no longer mutable adapter state; reads scope via a transient clone from withTenant(), and writes stamp the per-row tenant on collect(), letting a single buffer batch many tenants.

  • ClickHouse becomes strictly single-tenant with six protected seam methods (tenantColumnDefs, keyPrefix, extraQueryableColumns, scopeOnlyAttributes, scopeQueries, decorateRow) that SharedTables overrides to add all tenant SQL in one place.
  • SharedTables::withTenant hands the callback a tenant-bound shallow clone so the scope cannot leak across coroutines or past the closure; Usage::withTenant mirrors this at the wrapper layer and degrades gracefully on single-tenant adapters.
  • purgeDaily now correctly separates scope-only (tenant) queries from narrowing queries so a path-only purge cannot silently fall through to a tenant-wide daily delete.

Confidence Score: 5/5

Safe to merge; the core refactoring is architecturally sound and the concurrency guarantee (scoped clone, no shared mutable tenant state) is correctly implemented.

The tenant-scoping logic is correct across all public read paths. The purge/daily-purge split properly separates scope-only and narrowing queries. The three issues noted are design polish rather than data correctness problems: redundant tenant filters on the daily routing path produce SQL that is logically equivalent to a single filter, the Nullable(String) type hint mismatch only matters for null bindings which never occur in the active scope path, and the shared HTTP client concern only applies to coroutine runtimes not yet in scope.

ClickHouse.php deserves a second look at the interaction between sum()/getTotal() and sumDaily() in the daily routing path, where scopeQueries runs twice on the same query set.

Important Files Changed

Filename Overview
src/Usage/Adapter/SharedTables.php New class implementing multi-tenancy via decorator seams; clean design with scoped clone pattern in withTenant()
src/Usage/Adapter/ClickHouse.php Multi-tenancy extracted into overridable seam methods; scopeQueries() is called at all public entry points but sumDaily/findDaily/sumDailyBatch are also called internally by routedSum, causing the tenant filter to be injected twice in the daily routing path
src/Usage/Usage.php withTenant() delegates to adapter clone when available; collect() correctly includes tenant in the buffer key so a single buffer holds many tenants
src/Usage/Adapter/Database.php Reverted to no tenant params on read methods; setTenant() still persists state on the underlying db object, which means tenant scope leaks across calls when passed null (previously flagged in this PR)
src/Usage/Adapter.php Doc-comment formatting cleanup only; no functional changes
tests/Usage/Adapter/ScopedClickHouse.php New test helper that pins a tenant via SharedTables subclass for exercising real tenant SQL in tests

Reviews (2): Last reviewed commit: "refactor: move multitenancy into a Share..." | Re-trigger Greptile

Comment thread src/Usage/Adapter/Database.php
Comment thread src/Usage/Usage.php Outdated
…ithTenant

The base ClickHouse adapter is now strictly single-tenant — no tenant column,
no tenant SQL. Multi-tenancy lives entirely in a SharedTables subclass that
controls SQL generation through a handful of protected seams on the base:

  tenantColumnDefs()      tenant column on every table
  keyPrefix()             tenant as the leading ORDER BY / projection / GROUP BY key
  extraQueryableColumns() tenant accepted as a query attribute
  scopeOnlyAttributes()   tenant scopes but never narrows a daily purge
  scopeQueries()          inject the active tenant filter on reads/purges
  decorateRow()           stamp the per-row tenant on writes

Tenant is no longer adapter state and no longer a parameter on every read
method (reverted). Instead:

  - Reads/purges scope through SharedTables::withTenant($tenant, $fn), which
    hands the callback a tenant-bound *clone* — no shared mutable state, so it
    is safe under concurrent coroutines, and the scope can't leak past the
    closure. Usage::withTenant mirrors it for the wrapper.
  - Writes keep the per-row $tenant on collect(), so one buffer still batches
    many tenants in a single flush.

Tests pin a tenant via a ScopedClickHouse(extends SharedTables) helper; added
explicit withTenant override + isolation tests. 247 tests green, PHPStan max
and Pint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@loks0n loks0n changed the title Fix namespace/sharedTables at construction; thread tenant explicitly Extract multitenancy into a SharedTables decorator with safe withTenant scoping Jun 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant