Skip to content

Fix bulk data download stalling over WAN / high-latency links (#31) - #61

Open
hmshb wants to merge 1 commit into
caobo171:masterfrom
hmshb:fix/wan-large-download-stall
Open

Fix bulk data download stalling over WAN / high-latency links (#31)#61
hmshb wants to merge 1 commit into
caobo171:masterfrom
hmshb:fix/wan-large-download-stall

Conversation

@hmshb

@hmshb hmshb commented Jun 9, 2026

Copy link
Copy Markdown

Problem

readWithBuffer() (and therefore getAttendances()) works fine on a LAN but truncates the download to roughly the first ~2 chunks when the device is reached across the internet or any slow / high-latency link. It usually shows up as Fetched 0 logs or only stale records, with recent entries (at the tail of the history) never arriving. This is the issue reported in #31 ("get attendance timeout with large amounts of data").

I traced it on a live ZKTeco K50 behind PPPoE, polled from a cloud server: connect, voice test and record-count all worked; only the bulk download stalled, consistently at the same byte boundary regardless of timeout.

Root causes

  1. All chunk requests were fired up front (for (i=0..numberChunks) sendChunkRequest(...)). On a LAN the device keeps up. Over a high-latency link its send buffer fills faster than the link drains, the remaining chunks never arrive, and the stream wedges — truncating the download.

  2. The inter-packet timeout (TIMEOUTS.CHUNK_TCP) ignored the constructor timeout, so slow links couldn't be given more room.

Changes (backward compatible)

  • Request chunks sequentially — send one request, wait for the chunk to fully arrive, then request the next. Only one chunk is ever in flight, so a slow link keeps up and the download completes. (Same approach as Python pyzk.)
  • Use the constructor timeout for the inter-packet wait, falling back to TIMEOUTS.CHUNK_TCP when unset.
  • New optional maxChunk argument on ZKLib/ZKLibTCP (default unchanged). Lowering it (e.g. 8184) makes each chunk individually more robust on poor links.
// unchanged default behaviour:
const zk = new ZKLib(ip, 4370, 10000, 4000)
// smaller chunks + longer timeout for a slow/WAN link:
const zk = new ZKLib(ip, 4370, 60000, 4000, 0, 'tcp', 8184)

Result

On the real K50 over the WAN link: stalled at ~3,200 records (months-old data) with defaults → pulled the full ~9,900 records including same-day punches in ~18s with maxChunk=8184.

No API breakage — omit maxChunk and behaviour is identical. Verified against a live device; node -c clean. Happy to adjust naming/placement to fit your conventions.

…inks

readWithBuffer() returns the full reply on a LAN but truncates to the
first ~2 chunks (often surfacing as "0 logs" or only old records) when
the device is reached across the internet / a slow link. Long-standing
pain point — see caobo171#31.

- Request chunks sequentially (send one, await its full arrival, then
  request the next) instead of firing every sendChunkRequest up front.
  Firing all at once works on a LAN but over a high-latency link the
  device's send buffer fills faster than the link drains and the later
  chunks never arrive, truncating the download. One chunk in flight lets
  the slow link keep up (same approach pyzk uses).
- Respect the constructor `timeout` for the inter-packet wait instead of
  the fixed TIMEOUTS.CHUNK_TCP (falls back to it when unset).
- New optional `maxChunk` arg on ZKLib/ZKLibTCP (default unchanged).
  Lowering it (e.g. 8184) makes each chunk individually more robust on
  poor links. Fully backward compatible.

Verified against a live ZKTeco K50 behind PPPoE: stalled at ~3,200
records with defaults, pulled the full ~9,900 in ~18s with maxChunk=8184.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant