Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 0 additions & 7 deletions docs-data/property-overrides.json
Original file line number Diff line number Diff line change
Expand Up @@ -496,13 +496,6 @@
"cloud_topics_disable_reconciliation_loop": {
"config_scope": "cluster"
},
"cloud_topics_enabled": {
"description": "Enable Cloud Topics for the cluster. Cloud Topics are optimized for high-throughput, cost-sensitive workloads that can tolerate higher latencies compared to standard Kafka topics.",
"related_topics": [
"self-managed-only: xref:develop:manage-topics/cloud-topics.adoc[Cloud Topics]"
],
"config_scope": "cluster"
},
"cloud_topics_epoch_service_epoch_increment_interval": {
"description": "The interval, in milliseconds, at which the cluster epoch is incremented.\n\nThe cluster epoch is a frozen point in time of the committed offset of the controller log, used to coordinate partition creation and track changes in Tiered Storage. This property controls how frequently the epoch is refreshed. More frequent updates provide finer-grained coordination but may increase overhead.\n\nDecrease this interval if you need more frequent epoch updates for faster coordination in Tiered Storage operations, or increase it to reduce coordination overhead in stable clusters.",
"version": "v25.3.3"
Expand Down
18 changes: 7 additions & 11 deletions modules/develop/pages/manage-topics/cloud-topics.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
:page-topic-type: how-to
:personas: streaming_developer, platform_admin
:learning-objective-1: Describe the latency and cost trade-offs of Cloud Topics compared to standard Redpanda topics
:learning-objective-2: Create a Cloud Topic using rpk after enabling Cloud Topics on your cluster
:learning-objective-2: Create a Cloud Topic using rpk on a cluster that has cloud storage enabled

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Feediver1 just a note that these don't render on the page because it's missing "After reading this page, you will be able to: + * [ ] {learning-objective-1}". If you decide to fix, you'd need to make sure it renders properly in Cloud too, but not a blocker for this PR. Lots of pages still don't render these.

:learning-objective-3: Identify Cloud Topics limitations and configurations that reduce cross-AZ networking costs
// tag::single-source[]
ifdef::env-cloud[:producers-page: develop:topics/configure-producers-for-cloud-topics.adoc]
Expand Down Expand Up @@ -56,19 +56,15 @@ endif::[]

include::develop:partial$cloud-topics-limitations.adoc[tag=limitations]

== Enable Cloud Topics
== Create Cloud Topics

To enable Cloud Topics for a cluster:

[,bash]
----
rpk cluster config set cloud_topics_enabled=true
----

NOTE: This configuration update requires a restart to take effect.
Cloud Topics don't require a separate cluster property to enable them. When cloud storage is enabled for your cluster, you can create Cloud Topics directly.

ifndef::env-cloud[]
NOTE: In Redpanda versions earlier than v26.2, you must also set the `cloud_topics_enabled` cluster property to `true` and restart the cluster. This property is deprecated in v26.2 and later. For details, see xref:upgrade:deprecated/index.adoc[Deprecated features].
endif::[]

After enabling Cloud Topics, you can proceed to create new Cloud Topics:
To create a Cloud Topic, set the topic property `redpanda.storage.mode` to `cloud`:

[,bash]
----
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,6 @@ spec:
<object-storage-settings>
config:
cluster:
cloud_topics_enabled: true
cloud_storage_attempt_cluster_restore_on_bootstrap: true
----

Expand All @@ -99,7 +98,6 @@ storage:
<object-storage-settings>
config:
cluster:
cloud_topics_enabled: true
cloud_storage_attempt_cluster_restore_on_bootstrap: true
----
+
Expand All @@ -113,15 +111,13 @@ helm upgrade --install redpanda redpanda/redpanda --namespace <namespace> --crea
```bash
helm upgrade --install redpanda redpanda/redpanda --namespace <namespace> --create-namespace \
--set storage.tiered.<object-storage-settings> \
--set config.cluster.cloud_topics_enabled=true \
--set config.cluster.cloud_storage_attempt_cluster_restore_on_bootstrap=true
```
====
--
======

* `storage.tiered`: Configure the target cluster with the same object storage settings as the failed source cluster, including the same bucket or container.
* `config.cluster.cloud_topics_enabled`: Enable Cloud Topics on the target cluster so that restored topics can be served in cloud storage mode.
* `storage.tiered`: Configure the target cluster with the same object storage settings as the failed source cluster, including the same bucket or container. Enabling cloud storage also enables Cloud Topics on the target cluster, so restored topics can be served in cloud storage mode without additional configuration.
* `config.cluster.cloud_storage_attempt_cluster_restore_on_bootstrap`: Run the restore automatically when the new cluster bootstraps. No manual restore command is required.

The restore runs while the cluster starts. Cluster size does not need to match the source cluster, but the target cluster must be empty of user-managed data.
Expand Down
30 changes: 16 additions & 14 deletions modules/manage/pages/kubernetes/cloud-topics/k-cloud-topics.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
:page-topic-type: how-to
:personas: platform_admin, streaming_developer
:learning-objective-1: Configure object storage for Cloud Topics on Kubernetes
:learning-objective-2: Enable and create Cloud Topics using the Redpanda Operator or Helm
:learning-objective-2: Create Cloud Topics using the Redpanda Operator or Helm
:learning-objective-3: Verify topic storage mode configuration
:env-kubernetes: true
:page-aliases: manage:kubernetes/k-cloud-topics.adoc
Expand All @@ -23,7 +23,7 @@ For an overview of how Cloud Topics work, including storage modes, use cases, an
You must have the following:

- **kubectl**: Ensure you have the https://kubernetes.io/docs/tasks/tools/#kubectl[`kubectl`^] command-line tool installed and configured to communicate with your cluster.
- **Redpanda**: A xref:deploy:deployment-option/self-hosted/kubernetes/kubernetes-deploy.adoc[Redpanda Operator and a Redpanda resource deployed] in your Kubernetes cluster running Redpanda v26.1 or later.
- **Redpanda**: A xref:deploy:deployment-option/self-hosted/kubernetes/kubernetes-deploy.adoc[Redpanda Operator and a Redpanda resource deployed] in your Kubernetes cluster running Redpanda v26.2 or later.
- **Object storage**: A configured object storage backend (Amazon S3, Google Cloud Storage, Azure Blob Storage, or an S3-compatible store such as MinIO).
- **Enterprise license**: A valid Redpanda Enterprise license applied to the cluster.
+
Expand All @@ -41,7 +41,7 @@ rpk cluster license info

== Configure object storage

Cloud Topics use the same object storage configuration as Tiered Storage. If you have already configured object storage for Tiered Storage, you can skip this step and proceed to <<enable-cloud-topics>>.
Cloud Topics use the same object storage configuration as Tiered Storage. If you have already configured object storage for Tiered Storage, you can skip this step and proceed to <<set-the-default-storage-mode>>.

For detailed instructions including IAM role configuration, access key management, and encryption options, see xref:manage:kubernetes/tiered-storage/k-tiered-storage.adoc#configure-object-storage[Configure object storage] in the Tiered Storage documentation.

Expand Down Expand Up @@ -223,17 +223,21 @@ storage:

For details on managed identities and account access keys, see xref:manage:kubernetes/tiered-storage/k-tiered-storage.adoc#microsoft-absadls[Microsoft ABS/ADLS] in the Tiered Storage documentation.

== Enable Cloud Topics
== Set the default storage mode

To enable Cloud Topics, set the `cloud_topics_enabled` cluster property to `true` and set the default storage mode for all new topics to `cloud`.
Cloud Topics are enabled automatically when cloud storage is enabled for the cluster (xref:reference:properties/object-storage-properties.adoc#cloud_storage_enabled[`cloud_storage_enabled`] set to `true`), which is part of the object storage configuration in <<configure-object-storage>>. No separate cluster property is required.

NOTE: In Redpanda versions earlier than v26.2, you must also set the `cloud_topics_enabled` cluster property to `true`. This property is deprecated in v26.2 and later. For details, see xref:upgrade:deprecated/index.adoc[Deprecated features].

To make all new topics Cloud Topics by default, set the default storage mode for the cluster to `cloud`.

[tabs]
======
Helm + Operator::
+
--

. Add the following to your Redpanda custom resource to enable Cloud Topics and set the default storage mode:
. Add the following to your Redpanda custom resource to set the default storage mode:
+
.`redpanda-cluster.yaml`
[,yaml]
Expand All @@ -260,10 +264,9 @@ spec:
cloud_storage_bucket: <bucket-name>
config:
cluster:
cloud_topics_enabled: true
default_redpanda_storage_mode: cloud # <1>
----
<1> Optional. Set to `cloud` to make all new topics Cloud Topics by default. Omit this to create Cloud Topics individually.
<1> Set to `cloud` to make all new topics Cloud Topics by default. Omit this to create Cloud Topics individually.

. Apply the configuration:
+
Expand All @@ -279,12 +282,12 @@ kubectl apply -f redpanda-cluster.yaml -n <namespace>
kubectl rollout status statefulset redpanda -n <namespace> --watch
----

. Verify that Cloud Topics are enabled:
. Verify that cloud storage, which enables Cloud Topics, is turned on:
+
[,bash]
----
kubectl exec -n <namespace> redpanda-0 -c redpanda -- \
rpk cluster config get cloud_topics_enabled
rpk cluster config get cloud_storage_enabled
----
+
Expected output: `true`
Expand Down Expand Up @@ -315,10 +318,9 @@ storage:
cloud_storage_bucket: <bucket-name>
config:
cluster:
cloud_topics_enabled: true
default_redpanda_storage_mode: cloud # <1>
----
<1> Optional. Set to `cloud` to make all new topics Cloud Topics by default. Omit this to create Cloud Topics individually.
<1> Set to `cloud` to make all new topics Cloud Topics by default. Omit this to create Cloud Topics individually.

. Deploy or upgrade the Helm chart:
+
Expand All @@ -336,12 +338,12 @@ helm upgrade --install redpanda redpanda/redpanda \
kubectl rollout status statefulset redpanda -n <namespace> --watch
----

. Verify that Cloud Topics are enabled:
. Verify that cloud storage, which enables Cloud Topics, is turned on:
+
[,bash]
----
kubectl exec -n <namespace> redpanda-0 -c redpanda -- \
rpk cluster config get cloud_topics_enabled
rpk cluster config get cloud_storage_enabled
----
+
Expected output: `true`
Expand Down
4 changes: 4 additions & 0 deletions modules/upgrade/pages/deprecated/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ This index helps you to identify deprecated features in Redpanda releases and pl
|===
| Deprecated in | Feature | Details

| 26.2.1
| `cloud_topics_enabled`
| Redpanda now enables Cloud Topics by default when cloud storage is enabled for the cluster, controlled by xref:reference:properties/object-storage-properties.adoc#cloud_storage_enabled[`cloud_storage_enabled`]. Remove `cloud_topics_enabled` from your configuration; setting it has no effect.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| Redpanda now enables Cloud Topics by default when cloud storage is enabled for the cluster, controlled by xref:reference:properties/object-storage-properties.adoc#cloud_storage_enabled[`cloud_storage_enabled`]. Remove `cloud_topics_enabled` from your configuration; setting it has no effect.
| Redpanda now enables Cloud Topics by default when cloud storage is enabled for the cluster, controlled by xref:reference:properties/object-storage-properties.adoc#cloud_storage_enabled[`cloud_storage_enabled`].
Remove `cloud_topics_enabled` from your configuration; setting it has no effect.

suggesting the line break just for readability


| 25.3.1
| The following configuration properties:

Expand Down
Loading