feat(sei-db): expose state-store pebbledb block cache size as a config option

[kylee9612]

Problem

Historical EVM queries (debug_traceBlockByNumber,
debug_traceTransaction) are slow under heavy trace load.

Root cause

These reads bypass the in-memory memIAVL (SC) layer and are
served directly from the state-store (SS) pebbledb backend, whose block
cache is hardcoded to 32 MiB in sei-db/db_engine/pebbledb/mvcc/db.go
(pebble.NewCache(1024 * 1024 * 32)). For trace-heavy workloads this is too
small, so reads become disk-bound and there is no way to tune it without
recompiling.

Fix

Expose the cache size as ss-cache-size-bytes in the
[state-store] config. Default stays at 32 MiB (the previous hardcoded
value); values <= 0 fall back to the default, so existing configs are
unaffected. Type is int64 to match pebble.NewCache(int64).

[cursor]

PR Summary

Low Risk
Operational tuning knob with unchanged default; only affects Pebble cache memory sizing at SS open, not consensus or data correctness.

Overview
Adds ss-cache-size-bytes under [state-store] so operators can tune the Pebble block cache for the state-store (SS) backend instead of relying on a fixed 32 MiB in OpenDB.

CacheSizeBytes is plumbed through StateStoreConfig, app.toml / GetConfig, and CLI flags (state-store.ss-cache-size-bytes). Values ≤ 0 use DefaultSSCacheSizeBytes (32 MiB), matching prior behavior when the key is omitted.

The MVCC Pebble SS engine now calls pebble.NewCache(cacheSizeBytes) from that config, aimed at reducing disk-bound reads on historical EVM workloads (e.g. debug_trace*) that hit SS rather than memIAVL.

^{Reviewed by Cursor Bugbot for commit 55ac355. Bugbot is set up for automated code reviews on this repo. Configure here.}

[yzang2019]

Thanks for the contribution, do you have any evidence or any data to show that increasing block cache size would help making debug_trace query faster? If so, how much faster? Would appreciate to link some test result here. (We've internally tried that and not seeing much improvement)

We have other ways of improving debug_trace, for example you can switch to RocksDB backend, which gives you 2x perf improvement

[kylee9612]

Thanks for the contribution, do you have any evidence or any data to show that increasing block cache size would help making debug_trace query faster? If so, how much faster? Would appreciate to link some test result here. (We've internally tried that and not seeing much improvement)

We have other ways of improving debug_trace, for example you can switch to RocksDB backend, which gives you 2x perf improvement

Thanks for the feedback. I re-ran a controlled comparison and do see a consistent improvement, so sharing the numbers.

Setup: two nodes, identical hardware(AWS i7i.4xlarge) , same chain height, tracing the same block (debug_traceBlockByNumber with callTracer). Only difference is the SS pebbledb block cache size. Each value is the average of 11 calls.

SS block cache	avg latency
32 MiB (current default)	~1769 ms
4 GiB (ss-cache-size-bytes)	~403 ms

Roughly a 4x speedup, with cache size as the only variable. The gap shows up on dense / trace-heavy blocks where the working set exceeds 32 MiB.

We're not considering the RocksDB backend at the moment, and in our testing simply increasing the cache size was the most effective option. The default stays at 32 MiB (<= 0 falls back), so existing deployments are unaffected.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit 651c4e7. Configure here.}

[kylee9612]

Hi @Kbhat1 @cody-littley , could anyone of you take a look when you have a chance? Happy to address any additional feedback. Thanks.

[codecov]

Codecov Report

❌ Patch coverage is 55.55556% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.09%. Comparing base (96fa55d) to head (55ac355).

Files with missing lines	Patch %	Lines
app/seidb.go	0.00%	1 Missing and 1 partial ⚠️
sei-db/db_engine/pebbledb/mvcc/db.go	60.00%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3623      +/-   ##
==========================================
- Coverage   60.20%   59.09%   -1.12%     
==========================================
  Files        2328     2219     -109     
  Lines      194502   182184   -12318     
==========================================
- Hits       117104   107654    -9450     
+ Misses      66849    65041    -1808     
+ Partials    10549     9489    -1060     

Flag	Coverage Δ
sei-chain-pr	55.84% <55.55%> (?)
sei-db	70.41% <ø> (ø)
sei-db-state-db	?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
sei-cosmos/server/config/config.go	96.10% <100.00%> (+0.01%)	⬆️
sei-db/config/ss_config.go	100.00% <100.00%> (ø)
app/seidb.go	78.40% <0.00%> (-1.83%)	⬇️
sei-db/db_engine/pebbledb/mvcc/db.go	67.48% <60.00%> (-0.17%)	⬇️

... and 192 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.