Feature request: TV-parity continuous-contract data source for futures

[Zombajo]

Summary

For futures strategies, PyneCore users currently have to choose between:

1. Raw per-contract Databento data, which jumps at roll boundaries and is not what TV charts show.
2. Databento's pre-stitched .c.0 continuous feed, which uses a documented ratio/difference adjustment that does not match TradingView's proprietary continuous-contract algorithm (offset ~700 points on NQ vs CME_MINI:NQ1!).

Either choice produces a price level mismatch vs TV. For ratio-based indicators that combine close with a moving average of close — BBW = 2*stdev(close,9)/sma(close,9), PMAR = close/vwma(close,20) — the ~700pt offset between TV's continuous and Databento's specific contract cascades into ~3pct BBW divergence and 7-10 absolute points of BBWP / PMARP percentile drift. That in turn produces ~19-24pct trade-level filter disagreement when running the same Pine source on TV vs PyneCore.

We solved this on our side with the reconstruction algorithm below (Apache-2.0, ~300 lines of Python over raw Databento stype_in: parent data). It would be natural for PyneCore to offer this as a first-class data source so users don't have to maintain a parallel reconstruction pipeline.

The reconstruction algorithm (Apache-2.0, freely usable)

Repo path: research/data/reconstruction/generate_tradingview_parity_contract.py in https://github.com/Zombajo/sanesignals-algotrading.
Doc path: METHODOLOGY/15_CONTINUOUS_CONTRACT_RECONSTRUCTION.md in the same repo.

Three steps:

1. Volume-dominance rollover detection. For each consecutive contract pair (current, next): on each trading day where both have volume, if daily_volume(next) > daily_volume(current), ROLL on that day. TV's exact trigger appears to be volume dominance — confirmed empirically by matching roll dates to TV's continuous front-month over a 16-year sample.

2. Chicago Central Time session boundaries for the daily-volume aggregation. Globex trading day starts 17:00 CT (previous calendar day). session_date = (timestamp_CT - 17 hours).normalize(). UTC aggregation produces 1-2 day roll-date offsets that compound at high-volume rollovers.

3. Additive backward adjustment. At each roll, gap = close(new_contract, roll_day) - close(old_contract, roll_day); all bars before the roll get +gap added to OHLC; gaps compound backward in time.

Validation results (NQ, 16 years)

Metric	Value
Open price offset vs TV	~2 points (constant)
Close price offset vs TV	Variable, sub-bar
BBWP impact of residual	< 0.01 percentile points
PMARP impact of residual	< 0.001 percentile points
Verdict	PARITY_APPROXIMATE — close enough for indicator reconciliation

After applying this reconstruction:

• NQ Edge+ Phase 2 (BBWP filter alone): TV PF and PyneCore PF reconcile to $0.00 median delta on matched trade samples.
• NQ Edge+ Phase 3 (PMARP filter alone): same outcome.

Why a built-in would help

Right now, every futures-strategy user who wants TV parity has to maintain their own reconstruction pipeline. That includes acquiring raw per-contract Databento data, running the volume-dominance roll detector, and applying additive backward-adjustment. It's a one-time problem to solve, and the result is reusable — having it as a native PyneCore data source would let strategy authors just declare the symbol with PyneCore handling the reconstruction transparently, the way TV does.

Suggested implementation surface

• New provider option, e.g. databento.continuous(symbol="NQ", method="volume_dominance_backadjusted"), that takes raw per-contract data and returns a reconstructed series.
• Or: integrate as a post-processing step in the existing pyne data convert-from CLI when an explicit --continuous-method tv_parity flag is set.

We are happy to:

• Open a draft PR porting the script into PyneCore's providers/ layout.
• Share the 16-year NQ validation dataset for the maintainers' own roll-date diff sanity-check.
• Document the algorithm in the PyneCore docs site.

Environment

• PyneCore version: 6.4.8
• Instrument tested: NQ (CME_MINI:NQ1! on TV)
• Other instruments at parity-pending status: ES, CL, GC, SI, HG, ZB, YM, RTY

[wallneradam]

Hi, and thanks for the detailed write-up — and especially for offering the
algorithm, the validation data, and a PR. That's a lot of careful work.

We think this is a real problem, and the right home for it is PyneCore's
provider-plugin layer rather than the core. We're currently building out that
plugin system, and a Databento provider plugin is exactly the kind of thing it
is meant to host: the continuous-contract reconstruction (roll detection +
back-adjustment) can live entirely inside the plugin, which keeps the core data
model untouched and lets the necessarily-approximate, TV-tracking logic evolve
on its own release cadence.

The plugin system is expected to become public around August/September. Once it
is out, we'd be very glad if you built — or helped us build — a Databento
provider plugin around your reconstruction. What you found clearly helps other
futures users too, so we'd love to see it become a proper installable plugin.

We'll keep this issue open as the tracking thread for that work.

[Zombajo]

Thanks — really glad the continuous-contract reconstruction has a home in the provider-plugin layer; that's exactly the right place for it. Count us in to build the Databento provider plugin around it. A few notes from our side: The roll-detection + back-adjustment logic and its TradingView-parity validation data are ready; happy to shape them into whatever the plugin interface expects. Whenever the plugin system is far enough along to build against (you mentioned ~Aug/Sep), please loop us in — even a draft interface spec would let us start packaging early. One thing we'd like to understand up front so there are no surprises later: how you see authorship/ownership and crediting working for community-built provider plugins (we'd contribute as Slingshot Studios / SaneSignals), and the intended license. Happy to keep this thread as the tracking place. Thanks again for the thoughtful response, and for building PyneCore in the open. From: Adam Wallner ***@***.***> To: ***@***.***> Cc: ***@***.***>, ***@***.***> Date: Thu, 04 Jun 2026 15:53:14 +0300 Subject: Re: [PyneSys/pynecore] Feature request: TV-parity continuous-contract data source for futures (Issue #60) wallneradam left a comment #60 (comment) Hi, and thanks for the detailed write-up — and especially for offering the algorithm, the validation data, and a PR. That's a lot of careful work. We think this is a real problem, and the right home for it is PyneCore's provider-plugin layer rather than the core. We're currently building out that plugin system, and a Databento provider plugin is exactly the kind of thing it is meant to host: the continuous-contract reconstruction (roll detection + back-adjustment) can live entirely inside the plugin, which keeps the core data model untouched and lets the necessarily-approximate, TV-tracking logic evolve on its own release cadence. The plugin system is expected to become public around August/September. Once it is out, we'd be very glad if you built — or helped us build — a Databento provider plugin around your reconstruction. What you found clearly helps other futures users too, so we'd love to see it become a proper installable plugin. We'll keep this issue open as the tracking thread for that work. — Reply to this email directly, #60?email_source=notifications&email_token=BQ22AXHJYFWNTOXRVR5R5KT46FWLVA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTINRSGIZTCNBXHA4KM4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2KYZTPN52GK4S7MNWGSY3L#issuecomment-4622314788 , or https://github.com/notifications/unsubscribe-auth/BQ22AXAORDXXFOOPHQVRIPD46FWLVAVCNFSM6AAAAACZVI7XKCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DMMRSGMYTINZYHA . You are receiving this because you authored the thread.

[wallneradam]

Great — glad to hear it, and thanks for the clear "count us in." Having your roll-detection / back-adjustment logic and the TV-parity validation data already in hand is exactly the head start this needs.

On licensing and authorship. The plugin is your work and your copyright — it's yours to license as you see fit, and contributing as Slingshot Studios / SaneSignals is completely fine. For what it's worth, our suggestion would be Apache-2.0, to match the rest of the ecosystem (PyneCore and the official plugins are all Apache-2.0, and your original write-up was too). If you'd like downstream attribution, add a NOTICE file: under Apache-2.0 anyone who distributes a derivative work must carry its attribution text forward, so your credit travels with the code. (That binds redistribution, not private use — but for "anyone who builds on and ships it must credit us," it's the standard, friction-free mechanism.) Either way, the choice is yours.

On timing. August/September stands as the public release target — before then I want to do a pass to review and unify the surface, and the broker and hook layers are still moving.

That said, you don't have to wait for that to start. A continuous-contract data source is a plain offline ProviderPlugin — the bottom, most stable layer of the plugin hierarchy (ProviderPlugin -> LiveProviderPlugin -> BrokerPlugin), so the parts still in flux (broker / hooks) sit above it and don't affect you. The interface and the pyne data CLI are already documented on the public plugin-system branch:

• Plugin system / ProviderPlugin guide: https://github.com/PyneSys/pynecore/blob/plugin-system/docs/development/plugin-system.md
• pyne data CLI: https://github.com/PyneSys/pynecore/blob/plugin-system/docs/cli/data.md

Caveat: the surface may still be tidied before the public release, so treat it as a draft spec rather than a frozen API. But your hard part — the reconstruction algorithm producing OHLCV — is independent of the exact adapter shape, so you can build against it now and the thin adapter layer is cheap to adjust later. If you start prototyping and hit anything unclear or missing, that feedback is genuinely useful and will feed straight into the pre-release cleanup.

I'll keep this thread as the tracking place, and ping here when the interface is frozen.