HDDS-15866. Design Doc for Manual Safe Mode in SCM#10775
Open
priyeshkaratha wants to merge 1 commit into
Open
Conversation
priyeshkaratha
force-pushed
the
HDDS-15866
branch
from
July 15, 2026 16:23
a91a756 to
3220f16
Compare
priyeshkaratha
changed the title
priyeshkaratha
force-pushed
the
HDDS-15866
branch
from
July 15, 2026 16:27
3220f16 to
e66f403
Compare
priyeshkaratha
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This design proposes a Manual Safe Mode for the Storage Container Manager (SCM), giving administrators an HDFS-style on-demand safe mode for maintenance and operational tasks.
Today SCM enters safe mode only at startup and automatically leaves it once its exit rules (datanode registration, container-replica reporting, healthy pipelines, etc.) are satisfied. After that, the only way back into safe mode is a full SCM restart — which is disruptive, re-triggers all startup rules, and cannot be scoped to one node. Operators need a lighter mechanism to temporarily freeze SCM write operations (container allocation, pipeline creation) while continuing to serve reads — for example during disaster recovery, state inspection/repair, or coordinated maintenance windows.
The doc describes introducing a second, independent reason for SCM to be in safe mode — manual — alongside the existing startup reason, where the effective state is simply startup OR manual. This is deliberately not a new client-visible mode: OM, the SDKs, and the S3 Gateway continue to observe a single "SCM is in safe mode" boolean with identical exceptions and retry behavior. The startup-vs-manual distinction is surfaced only through the admin CLI. Manual Safe Mode reuses the existing write-blocking and service-pause paths, never auto-exits, and requires an explicit ozone admin safemode exit.
Following the HDFS NameNode model (dfsadmin -safemode enter, and -fs host:port to target one node), Manual Safe Mode is per-SCM local state: ozone admin safemode enter applies to all SCMs (client fan-out), while --scm host:port targets a single SCM. The doc covers the state model, automatic-exit neutralization, CLI, wire-protocol changes (an additive EnterSafeMode RPC plus an optional admin-only reason field on the existing InSafeMode response — fully backward compatible), SCM HA behavior, a Startup-vs-Manual comparison table, and the alternatives considered (including a Ratis-replicated global flag, rejected because it cannot express single-SCM targeting).
What is the link to the Apache JIRA
HDDS-12706
How was this patch tested?
Doc generated by AI