feat: add safe client metadata rebuild for master high availability. - #3
Open
ShuweiShen772 wants to merge 9 commits into
Open
Conversation
Enable the master to rebuild its metadata from client-side state after a restart, so that a fresh master process can recover segment and object location information without recomputation. - mooncake-store: add rebuild flow across client_service, master_service, master_client, rpc_service, segment and allocator - add rebuild_types.h defining the rebuild request/response types - add client_metadata_rebuild unit test and ha live-recovery test harness Co-Authored-By: Claude <noreply@anthropic.com>
Strip references to internal design/impl docs (section numbers like "impl doc §4.1" and internal .md filenames) from the HA rebuild code comments, so the comments are self-contained for public review. Comment-only change; no code logic modified. Co-Authored-By: Claude <noreply@anthropic.com>
Translate the remaining Chinese comments and assertion messages in the HA rebuild unit test to English, and replace the draft-era changelog header with a concise English summary of the seven tests, so the file is ready for public review. Also apply clang-format (ColumnLimit 80) across the HA rebuild files to satisfy the format check. Comment, message, and whitespace only; no test or code logic changed, and all seven unit tests still pass. Co-Authored-By: Claude <noreply@anthropic.com>
Add a standalone scale benchmark for the HA metadata rebuild feature that measures rebuild latency, scale across a key-count sweep, and recovery-window read latency and availability. Includes the benchmark client, its driver script, and a report of the measured results. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Author
|
@Kang-Meng Could you please review this PR? This replaces #1 and includes the completed HA rebuild hardening and validation. Thank you. |
Kang-Meng
self-requested a review
September 1, 2026 09:48
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
--enable_ha_rebuild_gate=falseas an emergency rollbackValidation
ha_rebuild_gate_test: 7 tests passed, including exact roster, stale epoch, full RPC gating, timeout, and completion/timeout racesrebuild_retry_test: 9 tests passed, including all five recovery failure points, whole-transaction retry, stale epoch, and shutdown interruptionclient_metadata_rebuild_test: 7 tests passedmaster_service_test: passedBase
Rebased on the latest
xLLM-AI/Mooncake:v0.3.12_xllm_patchcommit (dd44a522). This PR replaces #2.🤖 Generated with Claude Code