Skip to content

[feat](fe) Show inverted index storage format for partitions#65776

Open
hoshinojyunn wants to merge 1 commit into
apache:masterfrom
hoshinojyunn:feat/partition_index_format_observability
Open

[feat](fe) Show inverted index storage format for partitions#65776
hoshinojyunn wants to merge 1 commit into
apache:masterfrom
hoshinojyunn:feat/partition_index_format_observability

Conversation

@hoshinojyunn

@hoshinojyunn hoshinojyunn commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary:

Original inverted_index_storage_format table property was not visible in SHOW PARTITIONS, this will increase testing costs for subsequent modifications to the partitioned inverted index rolling upgrade. For now, we'll focus on adding observability functionality to the partitioned inverted index format..SHOW TABLET STORAGE FORMAT can also serve as an observable interface.

This PR adds InvertedIndexStorageFormat to SHOW PARTITIONS and supports filtering by that column. It also extends SHOW TABLET STORAGE FORMAT to report per-tablet inverted-index formats in VERBOSE mode and aggregate InvertedIndexV2Count and InvertedIndexV3Count values. The local and cloud both return the required schema format information now.

Example:

CREATE TABLE example_inverted_index_format (
    k INT,
    v VARCHAR(20),
    INDEX idx_v (v) USING INVERTED
)
DUPLICATE KEY(k)
PARTITION BY RANGE(k) (
    PARTITION p1 VALUES [("0"), ("10"))
)
DISTRIBUTED BY HASH(k) BUCKETS 1
PROPERTIES (
    "replication_num" = "1",
    "inverted_index_storage_format" = "V3"
);

SHOW PARTITIONS FROM example_inverted_index_format
WHERE InvertedIndexStorageFormat = 'V3';

SHOW TABLET STORAGE FORMAT VERBOSE;

Example results (relevant columns):

PartitionName InvertedIndexStorageFormat
p1 V3
BackendId TabletId StorageFormat InvertedIndexStorageFormat
10001 123456789 V2 V3

Release note

SHOW PARTITIONS now displays and filters by InvertedIndexStorageFormat. SHOW TABLET STORAGE FORMAT now reports inverted-index storage formats for local and Cloud tablets.

Check List (For Author)

  • Test

    • Regression test: regression-test/suites/cloud_p0/test_partition_cloud_inverted_index_format.groovy
    • Unit Test: ShowPartitionsCommandTest.java
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes. SHOW PARTITIONS and SHOW TABLET STORAGE FORMAT expose inverted-index storage formats, including Cloud mode.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: SHOW PARTITIONS did not expose the configured inverted-index storage format or allow users to filter partitions by it. SHOW TABLET STORAGE FORMAT also omitted the inverted-index format and could not inspect Cloud tablets. Add the format column to partition output and filtering, return the tablet schema format from local and Cloud BEs, and include per-tablet and aggregate inverted-index format results. The Cloud regression now writes to every dynamic partition before validating the tablet output, because Cloud tablets are materialized on first write.

### Release note

SHOW PARTITIONS now reports and filters by InvertedIndexStorageFormat. SHOW TABLET STORAGE FORMAT reports inverted-index storage formats for local and Cloud tablets.

### Check List (For Author)

- Test: Regression test
    - source ~/.venv/bin/activate && export LOCAL_DORIS_PATH=/mnt/disk1/ganderun/partition_index_format_observability/tmp && ./run-regression-test.sh --run -d cloud_p0 -s test_cloud_show_inverted_index_storage_format -runMode=cloud
- Behavior changed: Yes (SHOW PARTITIONS and SHOW TABLET STORAGE FORMAT expose inverted-index storage formats, including Cloud mode)
- Does this need documentation: No
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@hoshinojyunn hoshinojyunn changed the title [feature](fe) Show inverted index storage format for partitions [feat](fe) Show inverted index storage format for partitions Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants