docs: correct outdated Nightly contributor guide content - #2784
docs: correct outdated Nightly contributor guide content#2784killme2008 wants to merge 9 commits into
Conversation
Deploying greptime-docs with
|
| Latest commit: |
5300142
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://5fd3fc7d.greptime-docs.pages.dev |
| Branch Preview URL: | https://codex-audit-contributor-guid.greptime-docs.pages.dev |
killme2008
changed the title
|
|
||
| ## Batching mode | ||
|
|
||
| Batching mode reuses GreptimeDB's query engine instead of maintaining an operator graph for every incoming row. For a time-windowed Flow, its main loop is: |
There was a problem hiding this comment.
Please take a look at this document that describes the dataflow of Flow engine. @discord9
| - /health | ||
| - /leader | ||
| - /heartbeat | ||
| - /node-lease |
| - Load balancing for Datanode, determines who should handle new table creation requests, more precisely, it makes resource allocation decisions. | ||
| - Election & High Availability, GreptimeDB is designed in a Leader-Follower architecture, only Leader nodes can write while Follower nodes can read, the number of Follower nodes is usually >= 1, and Follower nodes need to be able to switch to Leader quickly when Leader is not available. | ||
| - Statistical data collection (reported via Heartbeats on each node), such as CPU, Load, number of Tables on the node, average/peak data read/write size, etc., can be used as the basis for distributed scheduling. | ||
| Metasrv is the metadata and coordination service in a distributed GreptimeDB cluster. It does not sit on the data path. Its main responsibilities are: |
| table_B | ||
| ... | ||
|
|
||
| A table route maps each Region to its current Datanode peer. It does not contain a separate list of read replicas: |
There was a problem hiding this comment.
Table routes can carry read followers. At 0cc83c4570, RegionRoute has both leader_peer and follower_peers, and TableRoute::try_from_raw populates the latter from follower_peer_indexes. The current statement and one-peer diagram are therefore inaccurate even when a deployment has no followers configured. Could we describe one write leader plus zero or more follower/read peers, and update the Chinese page as well?
There was a problem hiding this comment.
Fixed in 1be0a8f. The English and Chinese overviews now describe one leader plus zero or more followers, including the read-replica case, and the route diagram matches that model.
| - 存储 Catalog、Schema、Table、Region、路由和节点元数据； | ||
| - 为新 Region 选择 Datanode，并维护表路由； | ||
| - 选举一个 Metasrv leader 负责协调元数据变更； | ||
| - 通过可恢复的 Procedure 执行 DDL、Region 迁移、故障转移和 repartition； |
There was a problem hiding this comment.
The glossary uses Repartition (重分区), while this Chinese sentence leaves lowercase repartition in otherwise Chinese prose. Could we use 重分区 or the established Repartition spelling to keep the terminology consistent?
There was a problem hiding this comment.
Fixed in 1be0a8f. The Chinese overview now uses “重分区”.
ca9f14d to
1be0a8f
Compare
| description: How Mito organizes mutable Region data in memtables and moves it into SST files. | ||
| --- | ||
|
|
||
| # Memtable design |
There was a problem hiding this comment.
This a new document about memtable, please take a look @evenyag
| Batching mode reuses GreptimeDB's query engine instead of maintaining an operator graph for every incoming row. For a time-windowed Flow, its main loop is: | ||
|
|
||
| 1. A source-table write marks the affected time windows as dirty. | ||
| 2. A `BatchingTask` wakes on its schedule or after a notification and collects pending dirty windows. |
There was a problem hiding this comment.
BatchingTask is not awakened by dirty-window notifications in the current implementation. The dirty handlers only update dirty_time_windows; scheduled tasks wait until the next scheduled evaluation, while adaptive tasks check pending work after their polling sleep. Could we replace “wakes on its schedule or after a notification” with something like “runs on its evaluation schedule or adaptive polling cadence and consumes pending dirty windows on its next evaluation”? The same notification-wakeup wording in batching_mode.md should be adjusted as well.
WenyXu
left a comment
WenyXu
left a comment
There was a problem hiding this comment.
These comments clarify two Metasrv contracts that contributors must preserve: cache-invalidation fan-out and the separation between lease renewal and Region failure detection.
| - electing one Metasrv leader to coordinate metadata changes; | ||
| - running recoverable procedures for DDL, Region migration, failover, and repartitioning; | ||
| - tracking node leases and Region statistics through heartbeats; | ||
| - notifying Frontends and Datanodes when cached metadata or Region state changes. |
There was a problem hiding this comment.
MetasrvCacheInvalidator broadcasts each InvalidateCaches instruction to the Frontend, Datanode, and Flownode mailbox channels. This bullet and the diagram omit Flownode and couple cache invalidation to Region-state notifications. Could we state that cache invalidations go to all three channels, while Region lifecycle instructions target Datanodes, and update the Chinese page as well?
| Datanodes maintain heartbeat streams to the Metasrv leader. Heartbeat requests report node identity, lease information, Region statistics, and other state used for placement and supervision. Responses carry control messages such as Region lifecycle instructions and cache invalidations. | ||
|
|
||
| However, GreptimeDB still faces a more complex situation than DynamoDB, in particular, RCU doesn't fit to describe GreptimeDB's read workloads which require a lot of computation. We are working on that. | ||
| Metasrv treats a heartbeat as a lease renewal, not merely as a metrics sample. Lease expiration is therefore part of failure detection and can lead to a Region failover procedure. Changes to heartbeat timing must remain consistent with the lease and supervision intervals. |
There was a problem hiding this comment.
The failover path is not driven by lease expiration. DatanodeKeepLeaseHandler renews a Datanode lease, while RegionSupervisor applies per-Region Phi Accrual detectors to heartbeat-arrival intervals and submits failover migrations only when Region failover is enabled. Could we separate these contracts here and link to the Region Failover guide for its enablement and storage prerequisites? The Chinese text has the same causal claim.
3 participants
What changed
Correct and complete the Nightly contributor guide while keeping it focused on component contracts, main execution paths, and contributor decisions rather than source navigation.
The architecture and testing boundary follows established contributor documentation practice: explain abstractions, flows, invariants, and how to validate a change; leave implementation-level exploration to the source. No source directory or file-path navigation is added to the architecture pages.
Scope
Verification
DOC_LANG=en pnpm check:linksDOC_LANG=zh pnpm check:linkstyposon all modified Markdown filesgit diff --check0cc83c4570and the currentgreptime-protodefinitions.main; no released/0.x docs, user-guide pages, generated outputs, configuration, or lockfiles changed.Checklist