Fixes 30929: Add Data Product as a top-level Explore filter - #31940
Conversation
Data Product could only be filtered through Advanced Search. Add it to the common and data-asset quick-filter dropdowns so it sits alongside Domain, Owner, Tag and Tier on Explore, and allow the "No Data Products" empty-value option. `dataProducts.displayName.keyword` is normalized to lowercase in the index, so the filter carries `sourceFields: 'dataProducts.displayName'` (same as Domain) to recover the properly cased label from the aggregation top hit. Closes #30929 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
✅ Playwright Results — workflow succeededValidated commit ✅ 569 passed · ❌ 0 failed · 🟡 1 flaky · ⏭️ 0 skipped · 🧰 0 lifecycle flaky PerformanceBlocking targets: ✅ met · Optimization targets: 🟡 in progress Shard-job maxima below are not the full workflow wall time; the linked run includes build, fixture, planning, and reporting. 🕒 Full workflow signal wall (to summary) 54m 58s ⏱️ Max setup 6m 40s · max shard execution 17m 27s · max shard-job elapsed before upload 23m 21s · reporting 5s 🌐 213.60 requests/attempt · 2.78 app boots/UI scenario · 22.73% common-shard skew Optimization targets still in progress:
🟡 1 flaky test(s) (passed on retry)
How to debug locally# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip # view trace |
|
| Count | Rule |
|---|---|
| 18 | sonarjs/no-duplicate-string |
All findings
| Location | Rule | Message | |
|---|---|---|---|
| 🟡 | src/constants/AdvancedSearch.constants.ts:22:12 |
sonarjs/no-duplicate-string |
Define a constant instead of duplicating this literal 7 times. |
| 🟡 | src/constants/AdvancedSearch.constants.ts:24:19 |
sonarjs/no-duplicate-string |
Define a constant instead of duplicating this literal 7 times. |
| 🟡 | src/constants/AdvancedSearch.constants.ts:32:12 |
sonarjs/no-duplicate-string |
Define a constant instead of duplicating this literal 8 times. |
| 🟡 | src/constants/AdvancedSearch.constants.ts:39:19 |
sonarjs/no-duplicate-string |
Define a constant instead of duplicating this literal 7 times. |
| 🟡 | src/constants/AdvancedSearch.constants.ts:42:12 |
sonarjs/no-duplicate-string |
Define a constant instead of duplicating this literal 5 times. |
| 🟡 | src/constants/AdvancedSearch.constants.ts:44:19 |
sonarjs/no-duplicate-string |
Define a constant instead of duplicating this literal 5 times. |
| 🟡 | src/constants/AdvancedSearch.constants.ts:47:12 |
sonarjs/no-duplicate-string |
Define a constant instead of duplicating this literal 5 times. |
| 🟡 | src/constants/AdvancedSearch.constants.ts:49:19 |
sonarjs/no-duplicate-string |
Define a constant instead of duplicating this literal 5 times. |
| 🟡 | src/constants/AdvancedSearch.constants.ts:52:12 |
sonarjs/no-duplicate-string |
Define a constant instead of duplicating this literal 5 times. |
| 🟡 | src/constants/AdvancedSearch.constants.ts:107:12 |
sonarjs/no-duplicate-string |
Define a constant instead of duplicating this literal 3 times. |
| 🟡 | src/constants/AdvancedSearch.constants.ts:112:12 |
sonarjs/no-duplicate-string |
Define a constant instead of duplicating this literal 3 times. |
| 🟡 | src/constants/AdvancedSearch.constants.ts:117:12 |
sonarjs/no-duplicate-string |
Define a constant instead of duplicating this literal 4 times. |
| 🟡 | src/constants/AdvancedSearch.constants.ts:282:12 |
sonarjs/no-duplicate-string |
Define a constant instead of duplicating this literal 3 times. |
| 🟡 | src/constants/AdvancedSearch.constants.ts:284:15 |
sonarjs/no-duplicate-string |
Define a constant instead of duplicating this literal 3 times. |
| 🟡 | src/constants/explore.constants.ts:20:40 |
sonarjs/no-duplicate-string |
Define a constant instead of duplicating this literal 5 times. |
| 🟡 | src/constants/explore.constants.ts:46:11 |
sonarjs/no-duplicate-string |
Define a constant instead of duplicating this literal 4 times. |
| 🟡 | src/constants/explore.constants.ts:53:11 |
sonarjs/no-duplicate-string |
Define a constant instead of duplicating this literal 4 times. |
| 🟡 | src/constants/explore.constants.ts:55:11 |
sonarjs/no-duplicate-string |
Define a constant instead of duplicating this literal 4 times. |
Fix locally (fast - only checks files changed in this branch):
make ui-checkstyle-changedAdding an asset to a data product updates the asset's search document asynchronously, and the Data Products dropdown builds its options from that aggregation — so opening the dropdown right after the bulk add could miss the option. Poll the table's search doc until it carries the data product. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Code Review ✅ ApprovedAdds Data Product as a top-level Explore quick filter alongside Domain and Tier, complete with proper aggregation casing and empty-filter support. No issues found. OptionsDisplay: compact → Showing less information. Comment with these commands to change the behavior for this request:
Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source |
|
* feat(ui): add Data Product as a top-level Explore filter Data Product could only be filtered through Advanced Search. Add it to the common and data-asset quick-filter dropdowns so it sits alongside Domain, Owner, Tag and Tier on Explore, and allow the "No Data Products" empty-value option. `dataProducts.displayName.keyword` is normalized to lowercase in the index, so the filter carries `sourceFields: 'dataProducts.displayName'` (same as Domain) to recover the properly cased label from the aggregation top hit. Closes #30929 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(ui): wait for the data product link to be indexed before filtering Adding an asset to a data product updates the asset's search document asynchronously, and the Data Products dropdown builds its options from that aggregation — so opening the dropdown right after the bulk add could miss the option. Poll the table's search doc until it carries the data product. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> (cherry picked from commit a40588c)
* feat(ui): add Data Product as a top-level Explore filter Data Product could only be filtered through Advanced Search. Add it to the common and data-asset quick-filter dropdowns so it sits alongside Domain, Owner, Tag and Tier on Explore, and allow the "No Data Products" empty-value option. `dataProducts.displayName.keyword` is normalized to lowercase in the index, so the filter carries `sourceFields: 'dataProducts.displayName'` (same as Domain) to recover the properly cased label from the aggregation top hit. Closes #30929 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(ui): wait for the data product link to be indexed before filtering Adding an asset to a data product updates the asset's search document asynchronously, and the Data Products dropdown builds its options from that aggregation — so opening the dropdown right after the bulk add could miss the option. Poll the table's search doc until it carries the data product. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> (cherry picked from commit a40588c)
…32015) * Fixes 30929: Add Data Product as a top-level Explore filter (#31940) * feat(ui): add Data Product as a top-level Explore filter Data Product could only be filtered through Advanced Search. Add it to the common and data-asset quick-filter dropdowns so it sits alongside Domain, Owner, Tag and Tier on Explore, and allow the "No Data Products" empty-value option. `dataProducts.displayName.keyword` is normalized to lowercase in the index, so the filter carries `sourceFields: 'dataProducts.displayName'` (same as Domain) to recover the properly cased label from the aggregation top hit. Closes #30929 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(ui): wait for the data product link to be indexed before filtering Adding an asset to a data product updates the asset's search document asynchronously, and the Data Products dropdown builds its options from that aggregation — so opening the dropdown right after the bulk add could miss the option. Poll the table's search doc until it carries the data product. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> (cherry picked from commit a40588c) * fix spec * fix(ui): Fix lineage canvas scrolling when the filters panel is expanded [1.13] (#32077) * Fixes #29940: render the tab that is on screen when the URL has no tab segment (#31448) (#32029) * Fixes #29940 - Render the tab that is on screen when the URL has no tab `GenericProvider` read the active tab only from the `:tab` route param and, when it was absent, fell back to the first tab of the persona-customized page. Entity landing URLs carry no tab segment (`/domain/:fqn`), and the domain tree view keeps the tab in local state and never writes it to the URL at all, so the provider resolved a different tab than the one the page was rendering. With a persona customization that reorders the tabs, the Documentation tab rendered the first customized tab's layout instead of its own. Every non-documentation domain tab persists an empty layout, so the pane came up blank. Clicking another tab and back recovered it (the URL then carried a tab); in the tree view it never recovered. Give `GenericProvider` an explicit `activeTab` prop, defaulting to the route param, and pass the rendered tab from the pages that default the tab bar to a fixed tab rather than letting it land on the customized first tab. The `filteredKeys` reset now keys off the same value, so it also fires on an inline tab switch in the tree view. * fix persona level issue * fix the tree level persona issue * chore: drop activeTab pass-through from entity pages that don't need it These 13 entity-detail pages (APIEndpoint, Chart, Dashboard, DriveService/*, Metric, Topic, DatabaseSchema, SearchIndex, StoredProcedure, TagPage) always carry a :tab URL segment, so GenericProvider's existing routeTab fallback is sufficient. Passing activeTab={activeTab} here is a no-op and adds needless churn to the fix for #29940. The actual fix lives in GenericProvider, DomainDetails, DomainTreeView, DataProductsDetailsPage, getRenderedActiveTab, and DomainClassBase — the six files that address the real root cause. * fix lint checks * addressed comments --------- (cherry picked from commit 489eb6e) Co-authored-by: Sid <30566406+siddhant1@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> * Fixes #31980: handle null persona customization pages (#31981) (#32037) Backport of #31981 to the 1.13 branch. The 1.13 branch predates the react-query docStore refactor, so the fix is adapted to 1.13's data-fetching path rather than cherry-picked verbatim: - Add PersonaPage.utils.ts (getPersonaPage / normalizePersonaDocument / updatePersonaDocumentPage), which safely filters null/invalid page entries before they reach the UI. Ported unchanged from the source PR since it only depends on generated types. - useCustomPages, CustomizeStore, CustomizablePage, MyDataPage, and DataMarketplacePage now use these helpers instead of raw data.pages.find(...) / manual page mutation. - MyDataPage guards against a non-array customized layout and falls back to the default layout. Files that do not exist on 1.13 were omitted: - rest/queries/docStoreQuery.ts (react-query layer not present) - DataMarketplacePage.component.test.tsx (page has no test on 1.13) The react-query-specific test additions to useCustomPages.test.ts were also omitted; the equivalent MyDataPage tests (null legacy page + invalid layout) were ported and adapted to 1.13's getDocumentByFQN-based test harness. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(mlflow): use single-quoted filter for UC model version search (#31387) #30856 built the version-search filter with double quotes, which the Databricks endpoint rejects: INVALID_PARAMETER_VALUE: Unsupported filter query : `name="..."`. Please specify your filter parameter in the format `name = 'model_name'`. So UC ingestion cleared the original TypeError and then failed one step later, recording every model as "Version not found". Unity Catalog does not parse this filter locally -- UcModelRegistryStore forwards the raw string to the Databricks REST endpoint. The double-quoted form was validated against MLflow's client-side SearchModelUtils parser, which is more permissive and is never invoked on the UC path, so that check proved nothing about the code path that runs. Revert to name='<model_name>', the form the reporter verified against their own registry. Replace the round-trip test, whose premise this falsifies, with one pinning the required syntax against the real failing name. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * fix(ingestion): detect missing Lake Formation grants in Athena test connection (#29515) Backported to 1.13, which still exposes the Athena handler as module-level functions rather than main's BaseConnection subclass. The GetTables/GetViews behaviour is identical; only the plumbing differs: - `_get_targeted_schemas` becomes a module-level `get_targeted_schemas` taking the service connection explicitly instead of reading `self`. - The executors close over the `engine` and `service_connection` parameters instead of `self.client` / `self.service_connection`. - Ruff's `# noqa: UP006/UP035` markers are dropped (1.13 lints with black/isort/pylint, not ruff) and the code is black-formatted at 88 cols. Tests are ported minus the cases that only assert main's class refactor (BaseConnection subclassing, engine disposal, classmethod URL builder). The URL tests are omitted as duplicates - 1.13 already covers them in tests/unit/test_source_connection.py. `get_connection_url`, `get_connection` and `get_lake_formation_client` are left byte-identical to 1.13. (cherry picked from commit 10aefa4) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(bigquery): ingest table last-modified timestamp into lifeCycle.updated (#30316) Backported to 1.13. The execution behaviour matches main: the mixin parses an optional UPDATED_AT column and populates lifeCycle.updated when present, and BigQuery uses the region-scoped INFORMATION_SCHEMA query (joining TABLE_STORAGE for storage_last_modified_time) whenever the dataset region resolves, falling back to the dataset-scoped created-only query otherwise. Adapted for 1.13, which has diverged from main in two ways: - BigquerySource caches dataset objects in a per-schema LRUCache (_dataset_obj_cache), where main still uses a single-slot _current_dataset_obj. The ported get_life_cycle_query tests reset the keyed cache instead, and 1.13's TestBigqueryPerSchemaCaching / _EvictedOnReadCache coverage is preserved untouched. - 1.13 lints with black/isort/pylint rather than ruff, so the `# noqa: UP006/UP045` markers are dropped and the code is black-formatted at 88 cols. The `# pyright: ignore` comments are kept (1.13 does gate on basedpyright) and repositioned onto the lines black's rewrap moved them off, so the new code adds no type errors. The two life-cycle query-shape tests from main are ported as-is; main's stored-procedure query imports are omitted because 1.13's stored-procedure tests assert on query text rather than the constants. Redshift and Snowflake are unaffected: neither life-cycle query selects an updated_at column, so lifeCycle.updated stays unset for them, and get_life_cycle_query() receives exactly the arguments the inlined format did. (cherry picked from commit 4381871) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(db2): adapt sqlalchemy-ibmi dialect to SQLAlchemy 2.0 (#30731) The ibmi (DB2 for i) scheme failed its connection test at the first step with "Not an executable object: 'VALUES CURRENT_SCHEMA'". sqlalchemy-ibmi 0.9.3 pins sqlalchemy<2 but is installed with --no-deps, so its SA-1.x call sites survive into our SA 2.0 runtime and fail on first use. The dialect's initialize() passes raw strings to Connection.execute(), which SA 2.0 rejects, and its reflection queries use the removed select([cols], whereclause) form. Patch the dialect at runtime before building the ibmi engine: wrap both raw-string statements in text(), and reassign the module-level select to a compat shim. The shim covers all 11 legacy reflection call sites at once because the dialect resolves select as a global on each call, which avoids reimplementing ~250 lines of vendor SQL that would drift from upstream. The shim carries order_by over, which get_foreign_keys and get_indexes rely on to group composite constraints in column order, and patch_ibmi_dialect resolves the dialect via getattr so a partially initialised module falls back instead of raising AttributeError. (cherry picked from commit 1ab7634) Adapted for 1.13: the patch call is placed in the ibmi branch of get_connection rather than Db2IbmiStrategy.build(), since 1.13 predates the connection Strategy refactor. Only the tests this commit introduced are ported; the pre-existing url/args/dispatch tests belong to that refactor and are left out. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(ingestion): prevent Airflow metadata loss on API failures (#31371) Cherry-picked from cdff317 on main. * fix(ingestion): preserve Airflow metadata on API failures * fix(ingestion): satisfy Airflow source type check * fix(ingestion): harden Airflow pagination failures * test(ingestion): complete Airflow task instance page (cherry picked from commit cdff317) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Fixes 31401: Handle unnamed and quoted ROW fields in Trino reflection (#31402) * fix(trino): handle unnamed and quoted ROW fields during reflection Two defects in the Trino ROW/struct type parser, both verified against a live Trino 483 instance. 1. Tables with unnamed ROW fields lost every column. Trino allows the field name to be omitted -- `CREATE TABLE t AS SELECT ROW(1,'x') AS s` reports `row(integer, varchar(1))` -- but the parser unpacked each field as `name type` and raised ValueError. sql_column_handler swallows that and falls back to an empty column list, so the table was published with zero columns behind a WARNING while the pipeline still reported success. The driver's own `datatype.parse_sqltype` has the same defect and runs first, so it is guarded too; the OpenMetadata data type comes from `system_data_type` for complex columns, so NULLTYPE there is harmless. Unnamed fields are named positionally to match Trino's 1-based field access, so `s[1]` reads as `field1`. 2. Struct child columns were named with literal quotes. Trino renders named fields quoted (`row("a" bigint)`, escaping an embedded quote as `""`), so children were `"a"` instead of `a`, breaking nested column FQNs and column-level lineage matching. The type string is also no longer lower-cased wholesale, which preserves the case of quoted field names in dataTypeDisplay. A leading quote is what distinguishes a named field from an unnamed one whose type contains spaces (`timestamp(3) with time zone`, `interval day to second`), both of which are now covered. Tests use verbatim `SHOW COLUMNS` output captured from Trino 483; the existing fixtures used an unquoted form that Trino never emits, which is why this went unnoticed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(trino): keep positional ROW field names from shadowing explicit ones A ROW may mix an unnamed field with one named literally `fieldN`: `CAST(ROW(1,'x') AS row(bigint, "field1" varchar))` is accepted by Trino 483 and reported as `row(bigint, "field1" varchar)`. Naming the unnamed field positionally then produced two struct children called `field1`, which makes the nested column FQN ambiguous and breaks column-level lineage that assumes unique child names. Verified end-to-end: both children arrived as `field1` with no validation error anywhere in the ingestion path. Field naming moves out of split_row_field, which now reports an unnamed field as None (mirroring how the driver's own ROW type models it), into resolve_field_names, which sees the whole ROW and suffixes a positional name until it is unique. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> (cherry picked from commit 0048e2f) * Fixes #30597: Parse array-root REST API schemas (#31698) * fix(ingestion): parse array-root REST schemas * fix(ingestion): avoid duplicate schema warnings (cherry picked from commit afcf248) * fix(ui): Fix lineage canvas scrolling when the filters panel is expanded [1.13] Replaces explicit height/scroll calculations on .ant-card-body with a flex-column layout so the canvas occupies available space when the filters panel opens, preventing bottom control buttons from scrolling out of view. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(ui): adjust fullscreen lineage height after linter update [1.13] Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Anujkumar Yadav <anujf0510@gmail.com> Co-authored-by: Sid <30566406+siddhant1@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com> Co-authored-by: Harsh Soni <64592571+harshsoni2024@users.noreply.github.com> Co-authored-by: harshsoni2024 <harshsoni2024@gmail.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Rohit Jain <60229265+Rohit0301@users.noreply.github.com> Co-authored-by: Anujkumar Yadav <anujf0510@gmail.com> Co-authored-by: Sid <30566406+siddhant1@users.noreply.github.com> Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com> Co-authored-by: Harsh Soni <64592571+harshsoni2024@users.noreply.github.com> Co-authored-by: harshsoni2024 <harshsoni2024@gmail.com>



Describe your changes:
Fixes #30929
Data Product could only be filtered from Advanced Search. I added it as a top-level Explore quick filter so it sits next to Domain, Owner, Tag and Tier.
COMMON_DROPDOWN_ITEMSandDATA_ASSET_DROPDOWN_ITEMSnow carry alabel.data-product-pluralentry keyed onEntityFields.DATA_PRODUCT(dataProducts.displayName.keyword).SUPPORTED_EMPTY_FILTER_FIELDSgainsDATA_PRODUCT, so the dropdown also offers the "No Data Products" (must_not exists) option like Domain/Tier/Tag do.keywordwith alowercase_normalizerin the index mappings, so the item declaressourceFields: 'dataProducts.displayName'— exactly what the Domain filter does — andgetOptionsFromAggregationBucketrecovers the properly cased display name from the aggregation top hit instead of showing the lowercased bucket key.No backend change is needed:
dataProducts.displayName.keywordis not in the global aggregation list, andExploreQuickFiltersalready falls back to/api/v1/search/aggregate?field=<key>for any facet the page aggregations don't carry (which is the path everysourceFieldsfilter takes anyway, Domain included). Filter application is generic —getExploreQueryFilterMustbuilds theterm/must_not existsclauses from the field key.Screen.Recording.2026-08-24.at.1.41.33.PM.mov
Screen.Recording.2026-08-24.at.8.51.26.PM.mov
Type of change:
High-level design:
N/A — small change.
Tests:
Use cases covered
the current result scope, with proper casing and counts.
Unit tests
openmetadata-ui/src/main/resources/ui/src/utils/SearchClassBase.test.ts(asserts the Data Product filter is exposed for the
tableanddataAssetindices)Backend integration tests
Ingestion integration tests
Playwright (UI) tests
openmetadata-ui/src/main/resources/ui/playwright/e2e/Features/ExploreQuickFilters.spec.tsshould filter assets by data producttest (creates a data product in the fixture domain,adds the fixture table to it, filters Explore by it and asserts the asset card)
should search for empty or null filtersnow also coversData ProductsManual testing performed
Not yet run against a local stack — reviewer-facing verification below is what I ran:
npx jest src/utils/SearchClassBase.test.ts src/components/Explore src/components/ExploreV1→ 290 passed, 4 skippednpx jest src/utils/AdvancedSearchUtils src/components/Glossary/GlossaryTerms/tabs src/components/SearchDropdown→ 141 passedtsc --noEmitreports no errors in the changed filesUI screen recording / screenshots:
Pending — will attach a recording of the Data Products dropdown on Explore.
Checklist:
Fixes #30929above.🤖 Generated with Claude Code
Greptile Summary
Adds Data Product as a top-level Explore quick filter, including support for filtering assets without an assigned data product.
Confidence Score: 5/5
The PR appears safe to merge.
No blocking failure remains.
Important Files Changed
Reviews (2): Last reviewed commit: "test(ui): wait for the data product link..." | Re-trigger Greptile