Skip to content

spock_apply: bound per-message memory; add 022 regression guard#501

Open
ibrarahmad wants to merge 1 commit into
mainfrom
MEM_CONSUM_6
Open

spock_apply: bound per-message memory; add 022 regression guard#501
ibrarahmad wants to merge 1 commit into
mainfrom
MEM_CONSUM_6

Conversation

@ibrarahmad

Copy link
Copy Markdown
Contributor

Run the exception-log body in ApplyOperationContext and reset that context once per dispatched message in replication_handler so JSON and tuple allocations no longer accumulate in TopTransactionContext for the outer apply transaction. 022 samples worker RSS via ps during use_try_block and asserts growth is bounded.

Run the exception-log body in ApplyOperationContext and reset that
context once per dispatched message in replication_handler so JSON and
tuple allocations no longer accumulate in TopTransactionContext for the
outer apply transaction.  022 samples worker RSS via ps during
use_try_block and asserts growth is bounded.
@coderabbitai

coderabbitai Bot commented Jun 12, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

This PR fixes a memory-context leak in the apply worker by scoping exception-handling allocations to ApplyOperationContext and centralizing per-operation cleanup. A new TAP regression test validates the fix by triggering a multi-row transaction, detecting the apply worker mid-apply, and asserting that RSS growth stays below a configured threshold.

Changes

Apply Worker Memory Context Leak Fix

Layer / File(s) Summary
Apply worker memory context isolation
src/spock_apply.c
log_insert_exception() now switches into ApplyOperationContext before building/updating exception-log entries and restores context on exit. replication_handler() adds centralized MemoryContextReset(ApplyOperationContext) after error-context stack handling to clean up per-message allocations in one place.
Regression test cluster setup and warm-up
tests/tap/t/022_apply_mem_context.pl
TAP test imports modules, documents the per-row context leak scenario, configures environment-driven sizing/timeouts, spins up a 2-node cluster with replicated table and unique indexes, and performs a warm-up insert to confirm replication.
Leak detection and RSS assertion
tests/tap/t/022_apply_mem_context.pl
Test runs a large single-transaction INSERT, disables subscription mid-apply to interrupt the commit state, re-enables subscription, detects active apply worker via pg_stat_activity, samples process RSS over time, and asserts growth stays below configured threshold in a SKIP-guarded block.
Memory context diagnostics and test completion
tests/tap/t/022_apply_mem_context.pl, tests/tap/schedule
Test invokes pg_log_backend_memory_contexts(), parses server log to extract "used" bytes for three named memory contexts, prints consolidated diagnostic report with RSS and context metrics, tears down cluster, finishes TAP run, and test is added to PR schedule.

Poem

A rabbit hops through memory's maze,
Where contexts leak in apply's haze—
Now scoped so tight, so clean, so bound,
With metrics sampled, truth we've found:
The leak is fixed, no more it grows! 🐰

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely summarizes the main change: bounding per-message memory in spock_apply and adding a regression test guard.
Description check ✅ Passed The description directly relates to the changeset, explaining the memory context improvements and the new regression test that validates the fix.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch MEM_CONSUM_6

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@ibrarahmad ibrarahmad requested a review from mason-sharp June 12, 2026 15:19
@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 0 duplication

Metric Results
Duplication 0

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/tap/t/022_apply_mem_context.pl`:
- Line 40: Remove the unused variable $THRESHOLD: delete the declaration "my
$THRESHOLD = $ENV{SPOCK_LEAK_THRESHOLD} // 5_000_000;" and update or remove the
stale comment referencing "5 MB" so the test reflects that the RSS growth limit
is actually controlled by $RSS_GROWTH (the active variable used later in the
test); ensure no remaining references to $THRESHOLD remain and that the comment
accurately describes $RSS_GROWTH's default.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: c664da9f-d2cb-49d5-aeb7-ad19af24defc

📥 Commits

Reviewing files that changed from the base of the PR and between 922ea93 and 23b1e08.

📒 Files selected for processing (3)
  • src/spock_apply.c
  • tests/tap/schedule
  • tests/tap/t/022_apply_mem_context.pl

Comment thread tests/tap/t/022_apply_mem_context.pl
@ibrarahmad ibrarahmad self-assigned this Jun 12, 2026
@ibrarahmad ibrarahmad added the enhancement New feature or request label Jun 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant