Skip to content

[To dev/1.3][Pipe] Fix pipe tablet memory self-lock during batching (#18266)#18282

Merged
jt2594838 merged 1 commit into
apache:dev/1.3from
Caideyipi:cp-01d1c15-dev-1.3
Jul 23, 2026
Merged

[To dev/1.3][Pipe] Fix pipe tablet memory self-lock during batching (#18266)#18282
jt2594838 merged 1 commit into
apache:dev/1.3from
Caideyipi:cp-01d1c15-dev-1.3

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

Backport of #18266 to dev/1.3.

A realtime Pipe can retain raw tablet events whose zero-sized PipeTabletMemoryBlock instances are accounted through forceResize. Before this fix, resize checked only remaining non-floating memory and could exhaust the pool, preventing the sink batch from allocating the memory needed to transfer and release those tablets.

This change applies the existing tablet/TsFile hard admission thresholds when a typed data-structure block grows through resize. Normal blocks, including sink batch buffers, keep the existing total-memory check.

The cherry-pick conflict was adapted to the 1.3 memory-manager implementation, which tracks usage through usedMemorySizeInBytes. The regression test uses the branch's configured non-floating memory size.

Tests

mvn test -pl iotdb-core/datanode -Dtest=PipeMemoryManagerResizeTest -DforkCount=0 -Dsurefire.forkNumber=0 -Ddevelocity.off=true

Result: 1 test run, 0 failures, 0 errors, 0 skipped. Checkstyle, Spotless, main compilation, and test compilation also passed.


This PR has:

  • been self-reviewed.
  • added comments explaining the intent of the code.
  • added a unit test for the resized-tablet admission path.

@jt2594838
jt2594838 merged commit 945ad79 into apache:dev/1.3 Jul 23, 2026
16 checks passed
@jt2594838
jt2594838 deleted the cp-01d1c15-dev-1.3 branch July 23, 2026 01:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants