Skip to content

Intraday candle auto-chunking duplicates every chunk-boundary day (API to= is inclusive, not exclusive as the code assumes) #53

Description

@MarketDataDev03

Summary

stocks.candles auto-splits intraday requests spanning more than 365 days into contiguous year-sized chunks where chunk N's to date equals chunk N+1's from date. The code assumes the API treats a date-only to= as exclusive, so no deduplication is done after merging. Live-API verification shows to= is inclusive for intraday candles: every internal boundary day is fetched twice and its candles appear duplicated in the merged response.

Affected code (current main, 10fd6d7)

  • StocksResource.candleChunks (src/main/java/com/marketdata/sdk/StocksResource.java ~L312-344): produces shared-boundary ranges (current = nextCut). Its javadoc states "to is exclusive so the boundary candle isn't duplicated" — this assumption is what the live API contradicts.
  • The merge (~L106-133) concatenates the decoded chunks with no dedup.
  • StockCandlesRequest.Builder.to javadoc ("The rightmost candle (exclusive)", stocks/StockCandlesRequest.java L129) documents the same incorrect semantics.
  • StocksResourceTest.candlesIntradayLongRangeSplitsIntoYearChunksAndMerges (L671-693) pins the overlapping wire shape: chunk 1 sends to=2020-12-31, chunk 2 sends from=2020-12-31.

Live-API evidence (2026-07-31)

GET /v1/stocks/candles/1H/AAPL/?from=2026-07-20&to=2026-07-27  → returns the 7 hourly candles of 07-27
GET /v1/stocks/candles/1H/AAPL/?from=2026-07-27&to=2026-07-28  → returns those same 7 candles again, plus 07-28

A date-only to= includes that day's intraday candles. Two contiguous chunks sharing a boundary date therefore both return the full boundary day.

Impact

Silent data corruption for any intraday request spanning >365 days: one full trading day duplicated per internal boundary (~390 rows/boundary at 1-minute resolution; a 3-year range has 2 boundaries). Duplicates are chronologically interleaved, so volume sums, VWAP, and backtests double-count without any visible signal.

Suggested fix

Either start chunk N+1 the day after chunk N's to (making the wire ranges truly disjoint), or dedupe by timestamp during the merge — plus correcting the two javadoc claims. The PHP SDK's chunking already does the former (addYear()->subDay() / next chunk addDay()) and can serve as the reference.

Cross-SDK note

This bug was found during development of v2 of the Go SDK, whose auto-chunking shared the same design. A cross-SDK audit followed: the Python and JS SDKs have the same bug (issues are being filed in each repo); the PHP SDK is not affected — its chunker already produces disjoint ranges.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions