Skip to content

perf: stream analyze statistics with KLL sketch#354

Merged
KKould merged 1 commit into
mainfrom
optimize-analyze-kll-sketch
Jun 14, 2026
Merged

perf: stream analyze statistics with KLL sketch#354
KKould merged 1 commit into
mainfrom
optimize-analyze-kll-sketch

Conversation

@KKould

@KKould KKould commented Jun 14, 2026

Copy link
Copy Markdown
Member

Summary

  • add a KLL sketch builder/view for approximate quantile buckets
  • stream ANALYZE statistics into KLL and Count-Min Sketch instead of fully sorting all values
  • use a shared relative error setting for histogram and CM sketch statistics
  • refine index scan costing for unique equality and ordered index hints

Analyze Benchmark

Release builds comparing origin/main (b5965dc) against this PR after replacing memcomparable sorting with direct DataValue comparison.

where_by_index_explain.slt, five runs, measuring only analyze table t1:

version avg median min max
main 25.09ms 24.92ms 24.77ms 25.70ms
current 13.28ms 13.28ms 13.10ms 13.47ms

TPCC with heaptrack, --backend kitesql-lmdb --measure-time 1 --num-ware 1:

version wall max RSS analyze sum
main 31.02s 667,136 KB 1.46s
current 31.10s 254,552 KB 1.13s

Heaptrack peak memory consumers:

version largest peak
main 402.65M
current 9.82M

Tests

  • make test-all
  • cargo fmt --check
  • cargo check -p kite_sql

@KKould KKould force-pushed the optimize-analyze-kll-sketch branch from ec892e7 to e91dd80 Compare June 14, 2026 20:13
@KKould KKould self-assigned this Jun 14, 2026
@KKould KKould force-pushed the optimize-analyze-kll-sketch branch from e91dd80 to 025ff14 Compare June 14, 2026 20:47
@KKould KKould added the enhancement New feature or request label Jun 14, 2026
@KKould KKould force-pushed the optimize-analyze-kll-sketch branch 2 times, most recently from b4d2a25 to eec3ee4 Compare June 14, 2026 21:41
@KKould KKould force-pushed the optimize-analyze-kll-sketch branch from eec3ee4 to d8cc5e7 Compare June 14, 2026 21:58
@KKould KKould merged commit 420b7fa into main Jun 14, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant