Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 30 additions & 1 deletion reference/replication/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -198,7 +198,36 @@ replication:

In this example, the local node only receives from `node-two` (one-way inbound) and only sends to `node-three` (one-way outbound).

> **Note**: When using controlled flow replication, avoid replicating the `system` database. The `system` database contains node configurations, so replicating it would cause all nodes to have identical (and incorrect) route configurations.
You can also scope flow per database, so different databases flow in different directions between the same two nodes. Use `sendsTo` / `receivesFrom` entries with a `database`:

```yaml
replication:
databases:
- cardata
- config
- system
routes:
- host: node-two

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In HarperDB replication route configuration, the property name is hostname rather than host. Using host will cause the configuration to be invalid.

Suggested change
- host: node-two
- hostname: node-two

replicates:
sendsTo:
- database: config # push central config downstream
receivesFrom:
- database: cardata # aggregate telemetry upstream
```

#### Replicating the `system` database with controlled flow

<VersionBadge type="changed" version="v5.2.0" />

Before v5.2, replicating the `system` database under controlled flow was discouraged: because `hdb_nodes` (the node registry) lives in `system` and each node advertised itself as a full-mesh participant, replicating `system` caused every node to discover and directly connect to every other node — collapsing a constrained topology into a full mesh.

As of v5.2 you can replicate `system` while keeping a constrained topology. When a node has directional routes, it advertises a **directional** registry record derived from those routes (which neighbors it sends to / receives from) instead of a blanket "connect to everyone." Discovered non-neighbor nodes therefore do not open direct connections. This lets central configuration — users, roles, and schemas — propagate transitively across the whole cluster while user-database connections stay on the routes you configured. For example, in a `roadside → middle → core` aggregation tree, a role created on a roadside node reaches the core through the middle tier, yet the core never opens a direct socket to a roadside node.

Notes and current limitations:

- This applies only when a node has **directional** routes (`replicates` with `sends`/`receives` or `sendsTo`/`receivesFrom`). A node with no directional routes keeps the legacy full-mesh advertisement.
- Central visibility of every node is not guaranteed: an aggregation node may not list every distant leaf in its `hdb_nodes` registry (the registry relay differs from data relay). This does not open a connection either way.
- Route changes to a node's own directionality take effect on restart.

### Explicit Subscriptions

Expand Down
20 changes: 20 additions & 0 deletions release-notes/v5-lincoln/5.2.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,23 @@ All patch release notes for 5.2.x are available on the [releases page](https://g
### Replicated `set_configuration`

The `set_configuration` operation now accepts `"replicated": true` to apply a configuration change to all cluster nodes in a single Operations API call, with per-node outcomes reported in the response's `replicated` array. Only cluster-appropriate parameters should be replicated — see [Configuration Operations](/reference/v5/configuration/operations#set-configuration).

## Replication

### Replicating the System Database with a Constrained Topology

Controlled-flow replication can now include the `system` database while keeping a constrained (non-mesh) topology.

Previously, replicating `system` was discouraged when using [controlled replication flow](/reference/v5/replication/overview#controlling-replication-flow): because the node registry (`hdb_nodes`) lives in `system` and each node advertised itself as a full-mesh participant, replicating `system` caused every node to discover and directly connect to every other node — defeating the point of a constrained topology.

As of 5.2, a node with directional routes advertises a **directional** registry record derived from those routes (the neighbors it sends to / receives from) instead of a blanket "connect to everyone." Discovered non-neighbor nodes therefore never open direct connections. This lets central configuration — users, roles, and schemas — propagate transitively across the entire cluster while user-database connections stay on the routes you configured.

For example, in a `roadside → middle → core` aggregation tree, replicating `system` now lets a role created on a roadside node reach the core (through the middle tier) without the core ever opening a direct socket to a roadside node.

Behavior notes:

- This applies only to nodes that have **directional** routes. A node with no directional routes keeps the legacy full-mesh advertisement, so existing full-mesh clusters are unaffected.
- `add_node` / `set_node` with database-scoped `sendsTo` / `receivesFrom` entries now restrict replication to those databases (consistent with the config-route behavior). To replicate all databases while excluding specific tables, use a wildcard entry (`excludeTables` with no `database`).
- Central visibility of every node is not guaranteed: an aggregation node may not list every distant leaf in its `hdb_nodes` registry. This does not open a connection either way.

See [Controlling Replication Flow](/reference/v5/replication/overview#controlling-replication-flow) for configuration details.