fix: avoid stack overflow in hasBinary() for objects with a toJSON()

[junsj119]

Problem

hasBinary() in lib/util.ts recurses into all own enumerable keys before checking for a custom toJSON() method:

for (const key in obj) {
  if (Object.prototype.hasOwnProperty.call(obj, key) && hasBinary(obj[key])) {
    return true;
  }
}

if (obj.toJSON && typeof obj.toJSON === "function" && !toJSON) {
  return hasBinary(obj.toJSON(), true);
}

Objects that define toJSON() but also hold internal circular references overflow the stack before the toJSON() short-circuit is ever reached. The clearest real-world case is a Mongoose document: its internal $__ cache (InternalCache) points back to the document and into the schema/model graph, so the for...in traversal never terminates.

In sharded mode the resulting RangeError: Maximum call stack size exceeded propagates through encode() → doPublish() → broadcast(), and the event is silently dropped — no error is surfaced unless DEBUG=socket.io-redis is enabled.

Fixes #572.

Fix

Check toJSON() before the key traversal. This mirrors how the payload is actually serialized (an object with toJSON() is emitted through its toJSON() output), so the function inspects the representation that will really be sent instead of walking the object's internal fields.

if (obj.toJSON && typeof obj.toJSON === "function" && !toJSON) {
  return hasBinary(obj.toJSON(), true);
}

for (const key in obj) { ... }

Tests

Added test/hasBinary.ts (no Redis required) covering:

• binary detection (ArrayBuffer / TypedArray, nested, in arrays)
• non-binary payloads
• regression: an object with a circular reference and a toJSON() method no longer overflows the stack
• binary detection through the toJSON() representation

Without the fix the last two cases throw RangeError; with it all four pass. Wired into test-runner.ts alongside the existing suites.