ENH: lib: Add deflated_sharpe_ratio() to judge optimize() results

[ipezygj]

What this adds

lib.deflated_sharpe_ratio(stats, trial_sharpe_ratios) — the probability (0–1) that the best run returned by Backtest.optimize() has a Sharpe ratio genuinely greater than zero, after correcting for the multiple testing the optimization itself performs.

Why

optimize() returns the maximum over all tried parameter combinations. The expected best Sharpe of N skill-less trials is already well above zero and grows with N, so the winning run's Sharpe is inflated by selection — the more combinations tried, the more the "best" result reflects luck rather than edge. The library currently reports the winner's Sharpe with no way to ask whether it clears that bar. This is the standard correction: Bailey & López de Prado (2014), The Deflated Sharpe Ratio (doi:10.3905/jpm.2014.40.5.094).

The trial count and the trial Sharpe dispersion are taken from the search itself (the heatmap), not from an assumed default — optimize() already produces exactly the data the estimator needs:

stats, heatmap = bt.optimize(fast=range(5, 30, 5), slow=range(10, 70, 10),
                             maximize='Sharpe Ratio', return_heatmap=True)
deflated_sharpe_ratio(stats, heatmap)

Concrete example with the test SmaCross on GOOG (28 combinations): the winner's Sharpe looks significant as a single test (PSR ≈ 0.98) but does not clear the hurdle its own 28-trial search sets by chance (DSR ≈ 0.18). That flip is the information the number adds.

Implementation notes

• No new dependencies — normal CDF/PPF come from stdlib statistics.NormalDist; the rest is numpy/pandas already in use.
• The periodic-returns resampling was extracted from compute_stats() into _stats.periodic_returns() and reused, so the winner's returns are computed by the exact same code path as the reported Sharpe — no duplicated freq logic.
• Warns if the passed heatmap's name isn't 'Sharpe Ratio' (i.e. optimization maximized something else, so the values wouldn't be trial Sharpes).
• Verified against an independent implementation of the same estimator (agreement to 3+ decimals on the example above and on the single-trial degenerate case, where DSR reduces to the probabilistic Sharpe ratio).
• Unit test included; flake8 clean; full python -m backtesting.test suite passes.

[ipezygj]

Pushed a correctness fix to this branch, found by probing my own implementation rather than by review.

A constant return series has no Sharpe ratio, but it did not reach the deflation arithmetic as a nan: the standard deviation of a constant series is floating-point residue rather than an exact zero, so a flat 0.1% series divided out to a Sharpe of ~1e16. That is finite, so it passed every guard, and the deflated Sharpe came out 1.0 — certainty of a real edge, from the one input that carries no information about one. The statistic exists to catch results that are too good to be true, so answering with maximum confidence there is the worst possible failure direction.

The guard now compares the dispersion against the resolution of a float at the scale of the data, scaled by the number of terms summed (n * eps * scale), rather than against zero. Measured over constant series spanning values 1e-7..1e3 and lengths 3..10000, the residue reaches at most 1.96 eps × scale, so this keeps a margin of at least 3.9x at every length, while a real series with sigma=1e-12 sits more than ten orders of magnitude above the threshold — nothing legitimate is caught, and the test asserts that a quiet series still gets a number.

Two notes on the test, since they are the reason this took a second pass:

• It sweeps values × lengths instead of asserting one series. My first attempt used a single flat series, passed, and still leaked at other lengths — the residue depends on both the value and the length, so a guard calibrated on one point tests only that point.
• It was run against the unfixed function and fails there, so it tests the guard rather than merely accompanying it.

Happy to split this into a separate commit or reword anything if you would rather review it apart from the feature.

[kernc]

The expected best Sharpe of N skill-less trials is already well above zero and grows with N, so the winning run's Sharpe is inflated by selection — the more combinations tried, the more the "best" result reflects luck rather than edge.

That's a reasonable idea.

Can you offer any comparison of Bailey-LdP scoring method against a simpler:

n_trials, sr_mean, sr_std, sr_max = len(heatmap), heatmap.mean(), heatmap.std(), heatmap.max()
penalty = (1 + lamda * np.log1p(n_trials))
score = norm.cdf((sr_max - sr_mean) / sr_std / penalty)