diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index 089097e22c..8535d9fd1d 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -109,6 +109,7 @@ *** xref:manage:kubernetes/k-manage-topics.adoc[Manage Topics] *** xref:manage:kubernetes/k-cloud-topics.adoc[Cloud Topics] *** xref:manage:kubernetes/k-manage-connectors.adoc[Manage Kafka Connect] +*** xref:manage:kubernetes/k-connect-pipelines.adoc[Redpanda Connect Pipelines] *** xref:manage:kubernetes/storage/index.adoc[Storage] **** xref:manage:kubernetes/storage/k-volume-types.adoc[Volume Types] **** xref:manage:kubernetes/storage/k-configure-storage.adoc[Configure Storage] diff --git a/modules/manage/pages/kubernetes/k-connect-pipelines.adoc b/modules/manage/pages/kubernetes/k-connect-pipelines.adoc new file mode 100644 index 0000000000..94d5acd2d1 --- /dev/null +++ b/modules/manage/pages/kubernetes/k-connect-pipelines.adoc @@ -0,0 +1,30 @@ += Redpanda Connect Pipelines in Kubernetes +:description: Run Redpanda Connect pipelines declaratively with the Redpanda Operator using the Pipeline resource. +:env-kubernetes: true +:page-categories: Management, Integration +:page-topic-type: concept + +// DRAFT (DOC-2275): Hold until the Pipeline resource ships in a released operator +// and rp-connect-docs#457 is published. See that PR for the full guide this page routes to. + +Run xref:connect:home:index.adoc[Redpanda Connect] pipelines as Kubernetes resources managed by the Redpanda Operator. You define each pipeline in a `Pipeline` custom resource, and the operator deploys it, validates its configuration before startup, rolls it on every change, and reports its health in the resource status. Pipelines follow the same declarative workflow as your Topic and User resources, so you can build and operate streaming data pipelines with the same GitOps tooling you already use for cluster resources. This page explains when to choose the Pipeline resource over other options and where to find the deployment guide. + +This feature requires an xref:get-started:licensing/overview.adoc[Enterprise Edition license] and Redpanda Operator or later. +// TODO(DOC-2275): Set the real minimum operator version when the feature ships. + +== When to use the Pipeline resource + +Use the Pipeline resource when you want: + +* *Declarative, GitOps-friendly management*: Pipelines are plain Kubernetes manifests that you can version, review, and deploy with tools such as Argo CD or Flux. +* *Validated rollouts*: The operator lints every configuration before the pipeline starts, and rolls the workload automatically when the configuration changes. +* *Cluster integration*: A pipeline can reference a Redpanda resource in the same namespace, and the operator injects the connection details, TLS material, and credentials for you. +* *Built-in connectors*: Redpanda Connect ships hundreds of connectors and processors, including AI integrations, without the JVM or plugin management that Kafka Connect requires. + +For Kafka Connect, which is community-supported with Redpanda, see xref:manage:kubernetes/k-manage-connectors.adoc[]. To run Redpanda Connect on Kubernetes without the operator, see xref:connect:install:helm-chart.adoc[]. + +== Get started + +For installation steps, pipeline examples, secrets handling, and lifecycle management, see xref:connect:install:kubernetes-operator.adoc[Run Pipelines with the Redpanda Operator] in the Redpanda Connect documentation. + +For the Pipeline CRD source file, see the xref:reference:k-crd-index.adoc[CRD reference]. diff --git a/modules/manage/pages/kubernetes/k-manage-connectors.adoc b/modules/manage/pages/kubernetes/k-manage-connectors.adoc index 7937fe1d48..2424cd24fc 100644 --- a/modules/manage/pages/kubernetes/k-manage-connectors.adoc +++ b/modules/manage/pages/kubernetes/k-manage-connectors.adoc @@ -4,7 +4,7 @@ :page-categories: Management, Integration :env-kubernetes: true -When you have Kafka Connect deployed, you can create and manage connectors using Redpanda Console or the Kafka Connect REST API. +When you have Kafka Connect deployed, you can create and manage connectors using Redpanda Console or the Kafka Connect REST API. To build streaming data pipelines with enterprise support and without Kafka Connect infrastructure, consider running xref:manage:kubernetes/k-connect-pipelines.adoc[Redpanda Connect pipelines] with the Redpanda Operator instead. include::shared:partial$community-supported-connectors.adoc[] diff --git a/modules/reference/pages/k-crd-index.adoc b/modules/reference/pages/k-crd-index.adoc index cc3f6044cc..9cf9373571 100644 --- a/modules/reference/pages/k-crd-index.adoc +++ b/modules/reference/pages/k-crd-index.adoc @@ -27,6 +27,10 @@ The CRD source definitions are maintained in the https://github.com/redpanda-dat | https://github.com/redpanda-data/redpanda-operator/blob/main/operator/config/crd/bases/cluster.redpanda.com_consoles.yaml[`cluster.redpanda.com_consoles.yaml`^] | Defines Redpanda Console resources. +| Pipeline +| https://github.com/redpanda-data/redpanda-operator/blob/main/operator/config/crd/bases/cluster.redpanda.com_pipelines.yaml[`cluster.redpanda.com_pipelines.yaml`^] +| Defines Redpanda Connect pipeline resources. See xref:connect:install:kubernetes-operator.adoc[Run Pipelines with the Redpanda Operator]. + | Schema | https://github.com/redpanda-data/redpanda-operator/blob/main/operator/config/crd/bases/cluster.redpanda.com_schemas.yaml[`cluster.redpanda.com_schemas.yaml`^] | Defines Schema Registry resources.