perf: index the PostgreSQL pushes hot paths and slim list projections - #1692
Open
dcoric wants to merge 3 commits into
Open
perf: index the PostgreSQL pushes hot paths and slim list projections#1692dcoric wants to merge 3 commits into
dcoric wants to merge 3 commits into
Conversation
Push rows carry the full diff inside steps, so queries that read the data JSONB detoast very large rows. Three hot paths paid for that: - the repo activity rollup scanned and detoasted every push row on every repos page; a covering expression index makes it an index-only scan that never touches the heap - the default dashboard query gets a matching partial index, and the user profile predicates get expression indexes - list queries now select data minus the steps key, matching the mongo backend's list projection, which also excludes steps; the push detail view still returns the full document Indexes ship as schema migration v6. Versions 4 and 5 are reserved by the repo_users normalisation branch, and the runner applies in version order, so the gap is harmless.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Indexes the PostgreSQL pushes hot paths and aligns list projections with the mongo backend. Extension of #1532 and #1581, stacked on the schema migrations branch; prompted by the review remark on #1532 that the pushes table is the scale concern.
Push rows carry the full diff inside
steps, so any query reading thedataJSONB detoasts very large rows. This PR:pushes_hot_path_indexes: a covering expression index that turns the repo activity rollup into an index-only scan (it previously detoasted every push row on every repos page), a partial index matching the default pending-review dashboard query, and expression indexes for the user profile predicatesgetPushes,getPushesForUserProfile) to selectdata - 'steps', matching the mongo backend's list projection, which also excludessteps; the push detail view (getPush) still returns the full documentrepo_usersnormalisation branch (feat: normalise PostgreSQL repo permissions into a repo_users join table #1590), documented in the registry; the runner applies by version order so the gap is harmlessThe wider question of shrinking the stored push document itself (the diff is currently stored twice per push, in
steps[].logsandsteps[].content, across all backends) is deliberately out of scope here and tracked in #1691.Like the other stacked PRs, the diff shown includes the parent branches until they merge into
feat/postgres; this PR's own change is the last commit.Resolves #1690