Skip to content

gc: priority scheduling with dual watermarks and cross-scan quota#436

Open
xiaoxichen wants to merge 1 commit into
eBay:stable/v4.xfrom
xiaoxichen:gc-sort
Open

gc: priority scheduling with dual watermarks and cross-scan quota#436
xiaoxichen wants to merge 1 commit into
eBay:stable/v4.xfrom
xiaoxichen:gc-sort

Conversation

@xiaoxichen

Copy link
Copy Markdown
Collaborator
  • Sort eligible chunks by garbage ratio (desc) before submission so the most garbage-heavy chunks are always GC'd first
  • Add gc_garbage_rate_threshold_low (default 30%) as a low watermark; chunks between the two watermarks consume at most half the quota
  • Track m_pending_normal_gc_task_count in pdev_gc_actor to reflect tasks queued or running in m_gc_executor across scan cycles; previous code only capped submissions per scan, allowing unbounded queue growth
  • scan_chunks_for_gc now skips a pdev entirely when already at quota, and derives low_tier_cap proportionally from remaining_capacity
  • Add ADR docs/adr/gc-priority-scheduling.md

- Sort eligible chunks by garbage ratio (desc) before submission so the
  most garbage-heavy chunks are always GC'd first
- Add gc_garbage_rate_threshold_low (default 30%) as a low watermark;
  chunks between the two watermarks consume at most half the quota
- Track m_pending_normal_gc_task_count in pdev_gc_actor to reflect tasks
  queued or running in m_gc_executor across scan cycles; previous code
  only capped submissions per scan, allowing unbounded queue growth
- scan_chunks_for_gc now skips a pdev entirely when already at quota,
  and derives low_tier_cap proportionally from remaining_capacity
- Add ADR docs/adr/gc-priority-scheduling.md

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Xiaoxi Chen <xiaoxchen@ebay.com>
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 71.79487% with 11 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (stable/v4.x@e1c23e1). Learn more about missing BASE report.

Files with missing lines Patch % Lines
src/lib/homestore_backend/gc_manager.cpp 70.27% 8 Missing and 3 partials ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@              Coverage Diff               @@
##             stable/v4.x     #436   +/-   ##
==============================================
  Coverage               ?   52.77%           
==============================================
  Files                  ?       36           
  Lines                  ?     5434           
  Branches               ?      683           
==============================================
  Hits                   ?     2868           
  Misses                 ?     2269           
  Partials               ?      297           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants