refactor: remove scoring autoprojection deprecation and commit to new behavior#7059
Draft
westonpace wants to merge 6 commits into
Draft
refactor: remove scoring autoprojection deprecation and commit to new behavior#7059westonpace wants to merge 6 commits into
westonpace wants to merge 6 commits into
Conversation
…vior Scoring autoprojection (automatically appending `_distance`/`_score` to the output when an explicit projection is specified) has been removed. The new permanent behavior matches what `disable_scoring_autoprojection` previously opted into: - No explicit projection (SELECT *): `_distance`/`_score` are still appended automatically. - Explicit projection without `_distance`/`_score`: they are NOT added. - Explicit projection that includes `_distance`/`_score`: they appear as requested. The `disable_scoring_autoprojection` flag is kept to avoid breaking callers: - `True` / `None`: silently accepted (new default is already active). - `False`: raises an error directing users to include the column explicitly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
westonpace
commented
Jun 2, 2026
Comment on lines
3802
to
3804
| tbl = dataset.scanner( | ||
| columns=["_rowid", "vec"], nearest=nearest, disable_scoring_autoprojection=False | ||
| columns=["_rowid", "vec", "_distance"], nearest=nearest | ||
| ).to_table() |
Member
Author
There was a problem hiding this comment.
We can remove cases that explicitly set disable_scoring_autoprojection=False. These were intended to test the flag. We don't support the legacy behavior anymore so the test case isn't needed.
| since = "0.0.0", | ||
| note = "Scoring autoprojection has been removed. This method is a no-op." | ||
| )] | ||
| pub fn disable_scoring_autoprojection(&mut self) -> &mut Self { |
Member
Author
There was a problem hiding this comment.
Go ahead and remove this method. Rust has compile-time checks so we don't have to keep old flags / methods around.
- Remove the `disable_scoring_autoprojection` Rust method entirely (Rust has compile-time checks, no need to keep a no-op around) - Drop test cases that exercised the now-removed `False` legacy behavior Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…tion After removing _distance/_score autoprojection for explicit projections, update tests and internal code to request these columns explicitly: - mem_wal vector_search.rs: add _distance to BaseTable/FlushedMemTable scanner projections (build_scanner_projection strips it; now re-added) - mem_wal fts_search.rs: add _score to BaseTable/FlushedMemTable scanner projections similarly - udtf.rs: add _score to FtsTableProvider schema so DataFusion exposes it as a selectable column; handle score_idx in projection mapping - scanner.rs tests: add _distance explicitly in tests that use explicit projections but expect distance in output; remove incorrect +1 in run_query helper for use_projection=true - knn.rs: use project(&["_distance"]) instead of empty Vec projection - Python tests: add _distance/_score to explicit projection lists where tests access those columns; update test_dynamic_projection assertion Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
The upstream lance-format/lance main still calls `scanner.disable_scoring_autoprojection()` from the Java JNI blocking_scanner.rs. Keep the method as a deprecated no-op so the merged CI build compiles. Also fix test_indexed_vector_scan assertion: explicit columns=["price"] no longer auto-appends _distance, so num_columns is 1 not 2. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The #[deprecated] attribute caused clippy to treat the call in the upstream blocking_scanner.rs as an error (-D warnings). Remove the deprecation so the method is a silent no-op until the upstream callers are updated. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scoring autoprojection (automatically appending
_distance/_scoreto the output when an explicit projection is specified) has been removed. The new permanent behavior matches whatdisable_scoring_autoprojectionpreviously opted into:_distance/_scoreare still appended automatically._distance/_score: they are NOT added._distance/_score: they appear as requested.The
disable_scoring_autoprojectionflag is kept to avoid breaking callers that had opted in. However, the legacy behavior is being removed (we have warned about this for nearly a year):True/None: silently accepted (new default is already active).False: raises an error directing users to include the column explicitly.