Skip to content

[fix](ann-index) Fix ANN range search state leakage and incorrect slot index tracking.#63965

Merged
yiguolei merged 1 commit into
apache:branch-4.1from
kaka11chen:cherry-pick-63666_4.1
Jun 2, 2026
Merged

[fix](ann-index) Fix ANN range search state leakage and incorrect slot index tracking.#63965
yiguolei merged 1 commit into
apache:branch-4.1from
kaka11chen:cherry-pick-63666_4.1

Conversation

@kaka11chen
Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

Cherrypick #63666

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

Copilot AI review requested due to automatic review settings June 1, 2026 12:13
@kaka11chen kaka11chen requested a review from yiguolei as a code owner June 1, 2026 12:13
@hello-stephen
Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@kaka11chen
Copy link
Copy Markdown
Contributor Author

run buildall

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes correctness issues in BE ANN range search pushdown by eliminating execution-state leakage across VExprContext clones and correcting how common-expression index-evaluation status is tracked when scan-block column order differs from storage column ids.

Changes:

  • Replace ANN “executed/fulfilled” state stored on shared VExpr roots with a per-evaluation result object propagated via evaluate_ann_range_search(...).
  • Track ANN range-search results in the current segment’s IndexExecContext and update common-expression index status using source column index (scan-block) rather than storage ColumnId.
  • Add BE unit tests and regression tests covering mixed indexed/non-indexed segments and remapped schema/index-status cases.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
regression-test/suites/ann_index_p0/ann_range_search_source_index_status_regression.groovy Adds regression coverage to ensure common-expr index status uses source column index so embedding reads can be skipped correctly.
regression-test/suites/ann_index_p0/ann_range_search_pushdown_regression.groovy Adds regression coverage for mixed indexed/non-indexed segments ensuring ANN execution state doesn’t leak and profile counter is correct.
be/test/storage/index/ann/ann_range_search_test.cpp Extends unit tests to validate no cross-clone state leakage and correct slot-map/index-status updates.
be/src/storage/segment/segment_iterator.cpp Counts ANN range-search executions per evaluation call and removes reliance on shared-root state.
be/src/exprs/virtual_slot_ref.h Updates ANN range-search virtual API signature to return per-call evaluation result.
be/src/exprs/virtual_slot_ref.cpp Propagates new ANN evaluation result through virtual slot ref.
be/src/exprs/vexpr.h Introduces AnnRangeSearchEvaluationResult and updates virtual method signature; removes shared-root execution state fields.
be/src/exprs/vexpr.cpp Removes legacy state accessors and adapts base method signature.
be/src/exprs/vexpr_context.h Extends context API to return per-call “executed” via out-param and support result caching flag.
be/src/exprs/vexpr_context.cpp Uses per-call evaluation result, stores bitmap in IndexExecContext, and fixes source-index-based slot-map lookup / index-status update.
be/src/exprs/vectorized_fn_call.h Updates ANN range-search override signature to return per-call evaluation result.
be/src/exprs/vectorized_fn_call.cpp Sets per-call executed/fulfilled result instead of mutating shared-root state.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread be/src/exprs/vexpr_context.cpp Outdated
Comment on lines 34 to 35
#include "exec/common/util.hpp"
#include "exprs/function_context.h"
@kaka11chen kaka11chen force-pushed the cherry-pick-63666_4.1 branch 2 times, most recently from e48e0b8 to 551cc28 Compare June 1, 2026 13:48
@kaka11chen
Copy link
Copy Markdown
Contributor Author

run buildall

…t index tracking. (apache#63666)

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

ANN range search execution state was stored on shared VExpr roots.
VExprContext clones share the root expression, so a segment that
executed ANN range search could leak that state into another segment
without an ANN index and incorrectly remove the common expression. ANN
range search also mixed schema column indexes with storage column ids
when updating common expression index status, so remapped schemas failed
to mark the source slot expression as evaluated. This patch returns ANN
execution state through the current evaluation call, stores ANN root
bitmap in the current segment IndexContext, and updates slot index
status by source column index.

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@kaka11chen kaka11chen force-pushed the cherry-pick-63666_4.1 branch from 551cc28 to 0eecfe6 Compare June 1, 2026 14:53
@kaka11chen
Copy link
Copy Markdown
Contributor Author

run buildall

@yiguolei
Copy link
Copy Markdown
Contributor

yiguolei commented Jun 2, 2026

skip buildall

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Jun 2, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 2, 2026

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 2, 2026

PR approved by anyone and no changes requested.

@yiguolei yiguolei merged commit f1cc4a3 into apache:branch-4.1 Jun 2, 2026
31 of 33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants