[26.2] Schema context support for Iceberg translation

docs-data/property-overrides.json

@@ -1813,6 +1813,16 @@
       ],
       "config_scope": "topic"
     },
+    "redpanda.schema.registry.context": {
+      "description": "The xref:manage:schema-reg/schema-reg-contexts.adoc[Schema Registry context] Redpanda uses to resolve schema IDs for records in this topic, for example when translating the topic to an Iceberg table. Defaults to the Schema Registry default context (`.`).",
+      "related_topics": [
+        "xref:manage:iceberg/specify-iceberg-schema.adoc#resolve-schemas-within-a-context[Resolve schemas within a Schema Registry context]"
+      ],
+      "config_scope": "topic",
+      "category": "schema-registry",
+      "type": "string",
+      "default": "."
+    },
     "redpanda.storage.mode": {
       "description": "The storage mode for a topic. Determines how topic data is stored and whether it is eligible for upload to object storage.\n\nAccepted values:\n\n* `local`: Topic data is stored only on the broker's local disk. Object storage upload is disabled for the topic, regardless of cluster-level Tiered Storage settings.\nifndef::env-cloud[]\n* `tiered`: Topic data is stored on local disk and also uploaded to object storage. Enables xref:manage:tiered-storage.adoc[Tiered Storage] for the topic.\n* `cloud`: Topic data is stored in object storage using the xref:develop:manage-topics/cloud-topics.adoc[Cloud Topics] architecture. Local storage is used only as a write buffer.\nendif::[]\nifdef::env-cloud[]\n* `tiered`: Topic data is stored on local disk and also uploaded to object storage. Enables Tiered Storage for the topic.\n* `cloud`: Topic data is stored in object storage using the Cloud Topics architecture. Local storage is used only as a write buffer.\nendif::[]\n* `unset`: Specifies that the topic's storage mode is unset, regardless of the cluster default. The topic may still have Tiered Storage enabled through the legacy properties `redpanda.remote.read` and `redpanda.remote.write`.\n\nThis property overrides the cluster-wide config_ref:default_redpanda_storage_mode,true,properties/cluster-properties[] setting for individual topics.",
       "related_topics": [

modules/manage/pages/iceberg/iceberg-troubleshooting.adoc

@@ -27,6 +27,7 @@ Use this page to:
 If Redpanda encounters an error while writing a record to the Iceberg table, Redpanda by default writes the record to a separate DLQ Iceberg table named `<topic-name>~dlq`. The following can cause errors to occur when translating records in the `value_schema_id_prefix` and `value_schema_latest` modes to the Iceberg table format:
 
 - Redpanda cannot find the embedded schema ID in the Schema Registry.
+- Redpanda cannot resolve the record's schema ID within the topic's configured Schema Registry context. For example, the schema exists only in a different context. See xref:manage:iceberg/specify-iceberg-schema.adoc#resolve-schemas-within-a-context[Resolve schemas within a Schema Registry context].
 - Redpanda fails to translate one or more schema data types to an Iceberg type.
 - In `value_schema_id_prefix` mode, you do not use the Schema Registry wire format with the magic byte.
 

modules/manage/pages/iceberg/specify-iceberg-schema.adoc

@@ -109,6 +109,43 @@ value_schema_latest:subject=<subject-name>,protobuf_name=<protobuf-message-full-
 +
 NOTE: If you don't specify the fully qualified Protobuf message name, Redpanda pauses the data translation to the Iceberg table until you fix the topic misconfiguration.
 
+[[resolve-schemas-within-a-context]]
+=== Resolve schemas within a Schema Registry context
+
+If you use xref:manage:schema-reg/schema-reg-contexts.adoc[Schema Registry contexts] to isolate schemas (for example, by environment or tenant), set the `redpanda.schema.registry.context` topic property to bind the topic to that context. Redpanda then resolves schema IDs referenced by records in the topic against the configured context instead of the default context (`.`).
+
+This property applies to `value_schema_id_prefix` and `value_schema_latest` modes, both of which rely on a schema registered in the Schema Registry to determine the Iceberg table structure.
+
+Starting in Redpanda 26.2, Schema Registry contexts are enabled by default. See xref:manage:schema-reg/schema-reg-contexts.adoc[Schema Registry contexts] to learn about contexts and qualified subject naming before you configure this property.
+
+.Set the Schema Registry context on a new topic
+[,bash]
+----
+rpk topic create <topic-name> --topic-config redpanda.schema.registry.context=<context-name>
+----
+
+.Set the Schema Registry context on an existing topic
+[,bash]
+----
+rpk topic alter-config <topic-name> --set redpanda.schema.registry.context=<context-name>
+----
+
+The context name must start with a period (`.`), for example `.staging`. If you don't set this property, Redpanda resolves schemas in the default context (`.`).
+
+If Redpanda cannot resolve a record's schema ID within the configured context, it doesn't translate the record and instead writes it to a dead-letter queue (DLQ) table. See xref:manage:iceberg/iceberg-troubleshooting.adoc[Troubleshoot Iceberg Topics].
+
+[IMPORTANT]
+====
+Redpanda resolves schema IDs using the topic's current `redpanda.schema.registry.context` value, not the context that was active when a record was ingested. Changing the context on a topic that has pending, uncommitted Iceberg translation entries can cause resolution errors or unexpected DLQ routing for records still being translated.
+
+To change the Schema Registry context on a topic that is actively translating to Iceberg:
+
+. Disable Iceberg translation for the topic.
+. Wait for pending translation entries to commit.
+. Change `redpanda.schema.registry.context`.
+. Re-enable Iceberg translation.
+====
+
 == How Iceberg modes translate to table format
 
 ifndef::env-cloud[]

modules/manage/pages/schema-reg/schema-reg-contexts.adoc

@@ -79,7 +79,6 @@ The `schema_registry_enable_qualified_subjects` property defaults to `false`, so
 
 * *Non-Java SerDe clients*: Not supported in Schema Registry contexts.
 * *Server-side schema ID validation*: Schema ID validation using Kafka record headers does not support contexts. However, schema ID validation using magic byte and prefix are supported.
-* *Iceberg topics*: You cannot use schemas within a context for Iceberg Topics.
 * *`referencedby` endpoint*: `GET /subjects/\{subject\}/versions/\{version\}/referencedby` returns a list of bare schema IDs with no context information. When references span contexts, it is not possible to determine which context each returned ID belongs to.
 * *Cross-context isolation*: Contexts provide organizational and ID-space isolation, but do not prevent cross-context schema references. There is no mechanism to block schemas in one context from referencing schemas in another context.
 * *Default context cannot be deleted*: You cannot delete the default context (`.`).

modules/reference/pages/properties/topic-properties.adoc

@@ -155,4 +155,10 @@ include::reference:partial$properties/topic-properties.adoc[tags=category-remote
 
 Integrate Redpanda topics as Iceberg tables.
 
-include::reference:partial$properties/topic-properties.adoc[tags=category-iceberg-integration;!deprecated;!exclude-from-docs]
+include::reference:partial$properties/topic-properties.adoc[tags=category-iceberg-integration;!deprecated;!exclude-from-docs]
+
+== Schema Registry and Validation Properties
+
+The following properties control server-side schema ID validation and schema resolution for topics when using Schema Registry.
+
+include::reference:partial$properties/topic-properties.adoc[tags=category-schema-registry;!deprecated;!exclude-from-docs]

modules/reference/partials/properties/topic-properties.adoc

@@ -1190,6 +1190,44 @@ endif::[]
 
 // end::category-iceberg-integration[]
 
+// tag::category-schema-registry[]
+=== redpanda.schema.registry.context
+
+The xref:manage:schema-reg/schema-reg-contexts.adoc[Schema Registry context] Redpanda uses to resolve schema IDs for records in this topic, for example when translating the topic to an Iceberg table.
+
+[cols="1s,2a"]
+|===
+| Property | Value
+
+| Type
+| `string`
+
+
+
+| Default
+|
+ifdef::env-cloud[]
+Available in the Redpanda Cloud Console
+endif::[]
+ifndef::env-cloud[]
+`.`
+endif::[]
+
+| Nullable
+| No
+
+ifndef::env-cloud[]
+| Restored on xref:manage:whole-cluster-restore.adoc[Whole Cluster Restore]
+| Yes
+endif::[]
+
+| Related topics
+|xref:manage:iceberg/specify-iceberg-schema.adoc#resolve-schemas-within-a-context[Resolve schemas within a Schema Registry context]
+
+|===
+
+// end::category-schema-registry[]
+
 // tag::category-schema-registry[]
 === redpanda.key.schema.id.validation