configdb: targeted class-relation lookups instead of full closures

[AlexGodbehere]

Review this first

The is_class() guard in class_subclasses and class_has_subclass. all_subclass only contains the reflexive pair (x, x) when x is in all_class, so the seed of both walks has to reproduce that condition exactly. class_members deliberately has no guard, on the reasoning that a non-class has no members so the join drops the seed. Confirm that holds.

This is a schema migration. It adds functions and indexes and does not alter or drop existing objects. The views are untouched and still available.

What this does

Replaces full class-closure scans with graph walks from a single starting point, and adds the indexes the schema was missing.

Why

all_subclass and all_membership are non-materialised views over a recursive CTE (sql/v8.sql). Two consequences:

1. PostgreSQL cannot push a qualifier into a recursive CTE, so where k.class = $1 is applied after the whole closure is built.
2. The seed, all_class, includes select class from membership, a sequential scan of the entire table.

So looking up one leaf class with 333 members reads and hashes every row in membership and every row in object. At 100,000 objects:

HashAggregate (actual time=78.832..79.028 rows=333)
  -> Hash Join (actual rows=333)
       -> CTE Scan on sc (actual rows=1)
            Filter: (class = 22)
            Rows Removed by Filter: 940
            CTE sc -> Recursive Union (actual rows=941)
                 -> Append (actual rows=100960)
                      -> Seq Scan on membership (rows=100320)
       -> Hash (actual time=62.416..62.417 rows=100320)

CDBNotify.class_watch re-runs one of these per watcher on every class change, and acs-auth is several of those watchers. These are the two queries seen running repeatedly on fpd-ago while PostgreSQL was saturated:

select distinct o.uuid from all_membership k join object o on o.id = k.id where k.class = $1
select distinct o.uuid from all_subclass  k join object o on o.id = k.id where k.class = $1

Changes

Schema v14 adds four functions:

Function	Direction
class_subclasses(class)	walks down
class_members(class)	walks down
class_has_subclass(class, obj)	walks up from the object
class_has_member(class, obj)	walks up from the object

Walking up means an existence test never materialises a class's membership.

model.js uses them in _class_lookup, _class_has, class_has, config_class_list, config_search, and the redundant-superclass delete in class_add_subclass.

Indexes added. subclass and membership had only unique(class, id), so any lookup by object was a sequential scan. config had only unique(app, object), which cannot answer a lookup by object. object.class and object.owner are self-references with no index at all.

Numbers

PostgreSQL 16.14, real ConfigDB schema, best of three, warm cache. Class tree of 1 root, 20 mid, 300 leaf.

Objects	Operation	Before	After
1,000,337	is this object a member?	186.64 ms	0.45 ms
1,000,337	subclasses of the root class	360.92 ms	0.54 ms
1,000,337	members of a leaf class	199.61 ms	11.97 ms
1,000,337	members of the root class	1398.23 ms	745.10 ms
1,000,337	delete one object, end to end	0.112 s	0.015 s

Existence tests are flat in object count: 0.42 / 0.46 / 0.46 / 0.45 ms at 1,647 / 10,337 / 100,337 / 1,000,337 objects.

Listing the members of a class that genuinely holds a million objects is still about a second. That is the cost of returning a million uuids, not a query problem.

Testing

acs-configdb has no test suite, so this is differential testing against the views on a live PostgreSQL running the real migration chain, v6 through v14.

Level	Comparisons	Mismatches
SQL, layered tree	every object, both lookups, config_class_list, the class_add_subclass delete	0
SQL, random DAG	1,126 subclass edges, 12,896 closure pairs, 55,586 all_membership rows	0
Application, through Model	50,708	0

migrate.sql runs clean from an empty server to v14, and applies v14 on top of an existing v13 database.

There is no regression net after this lands, because the service has no test suite.

What this does not fix

acs-auth expands a class-scoped grant into one ACL entry per object and returns the whole list from GET /v2/acl/:principal. On fpd-ago that was 8,897 entries from 149 stored ACEs over 1,310 datasets.

That grows linearly with object count. At 1M datasets it is roughly 6.8M entries and 690 MB of JSON per principal, rebuilt and resent whole on every relevant change, with no delta protocol. There is no targeted "may P do X to O" endpoint.

That needs an API change, not an index, and is written up separately.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MtATkkjtiFF8L6z98NFD8A