From 9d164a19461fafff32a09d620a8af658f91bcae1 Mon Sep 17 00:00:00 2001 From: igerber Date: Sat, 11 Jul 2026 13:13:19 -0400 Subject: [PATCH 1/3] docs: add maintainer paper reviews for LW 2025/2026 (PR #588 validation rubric) Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01FZK3FD9jxWGxBrPDw5APSg --- .../papers/lee-wooldridge-2025-review.md | 198 +++++++++++ .../papers/lee-wooldridge-2026-review.md | 329 ++++++++++++++++++ 2 files changed, 527 insertions(+) create mode 100644 docs/methodology/papers/lee-wooldridge-2025-review.md create mode 100644 docs/methodology/papers/lee-wooldridge-2026-review.md diff --git a/docs/methodology/papers/lee-wooldridge-2025-review.md b/docs/methodology/papers/lee-wooldridge-2025-review.md new file mode 100644 index 00000000..8af69965 --- /dev/null +++ b/docs/methodology/papers/lee-wooldridge-2025-review.md @@ -0,0 +1,198 @@ +# Paper Review: A Simple Transformation Approach to Difference-in-Differences Estimation for Panel Data + +**Authors:** Soo Jeong Lee (Southern Illinois University Carbondale), Jeffrey M. Wooldridge (Michigan State University) +**Citation:** Lee, S.J., & Wooldridge, J.M. (2025). A Simple Transformation Approach to Difference-in-Differences Estimation for Panel Data. SSRN Working Paper No. 4516518. First posted 27 Jul 2023; version reviewed dated April 26, 2026, last revised June 8, 2026. https://ssrn.com/abstract=4516518 +**PDF reviewed:** /private/tmp/claude-501/-Users-igerber-diff-diff-LWDiD/fa768cf3-1977-474a-9ddf-b98c87817229/scratchpad/ssrn-4516518.pdf +**Review date:** 2026-07-11 + +--- + +## Methodology Registry Entry + +*Formatted to match docs/methodology/REGISTRY.md structure. Heading levels and labels align with existing entries - copy the `## LWDiD` section into the appropriate category in the registry.* + +## LWDiD + +**Primary source:** Lee, S.J., & Wooldridge, J.M. (2025). A Simple Transformation Approach to Difference-in-Differences Estimation for Panel Data. SSRN Working Paper No. 4516518 (revision of June 8, 2026). https://ssrn.com/abstract=4516518 + +**Key implementation requirements:** + +*Assumption checks / warnings:* +- **Common timing** (Section 2): all treated units start treatment at date S with `1 < S <= T` (at least one pre-treatment period); treatment stays in place through T. + - **Assumption NAC** (No Anticipation, Common Timing; Equation (2.7)): `E[Y_t(1) - Y_t(0) | D = 1] = 0` for `t = 1, ..., S-1`. Implied by the stronger `Y_t(1) = Y_t(0)` for pre-periods (implicit in Heckman, Ichimura and Todd 1997; explicit in Abadie 2005). If announcement effects are suspected, drop a period or two just prior to the intervention (Section 4.4 robustness check). + - **Assumption CPTC** (Conditional Parallel Trends, Common Timing; Equation (2.10)): `E[Y_t(0) - Y_1(0) | D, X] = E[Y_t(0) - Y_1(0) | X]`, `t = 2, ..., T`. PT holds within sub-partitions defined by X; PT need not hold across the entire population. + - **Assumption OVLC** (Overlap, Common Timing; Equations (2.14)-(2.15)): propensity score `p(x) = P(D = 1 | X = x) < 1` for all x in Supp(X). + - Theorem 2.1: NAC + CPTC + OVLC identify `tau_r`, r = S, ..., T. +- **Staggered interventions** (Section 4): cohorts defined by first treatment period `g in {S, ..., T, infinity}`; treatment is absorbing (no reversibility); cohort indicators `D_S, ..., D_T, D_infinity` mutually exclusive and exhaustive. + - **Assumption CNAS** (Conditional No Anticipation, Staggered; Equation (4.4)): `E[Y_t(g) | D_g = 1, X] = E[Y_t(infinity) | D_g = 1, X]` for `g in {S,...,T}`, `t in {1,...,g-1}`. If only the never-treated (NT) group is used as control for each (g, r), the conditioning on X can be dropped (p. 17). + - **Assumption CPTS** (Conditional PT, Staggered; Equation (4.6)): `E[Y_t(infinity) - Y_1(infinity) | D, X] = E[Y_t(infinity) - Y_1(infinity) | X]`, `t = 2, ..., T`, for `D = (D_S, ..., D_T)`. + - **Assumption OVLS** (Overlap, Staggered; Equation (4.10)): `P(D_g = 1 | D_g + A_{r+1} = 1, X = x) < 1` for all x in Supp(X), where `A_{r+1} = D_{r+1} + ... + D_T + D_infinity` is the legitimate control-pool indicator at (g, r). + - Theorem 4.1: under CNAS, Equation (4.5) holds; adding CPTS makes cohort assignments D unconfounded with respect to `Y_dot_rg(infinity)` (conditional mean sense) given X. +- **Heterogeneous linear trends** (Section 5): **Assumption CHT** (Conditional Heterogeneous Trends; Equation (5.3)) allows a separate linear trend in the never-treated state per treatment cohort: `E[Y_t(infinity) | D, X] = eta_S*(D_S*t) + ... + eta_T*(D_T*t) + q_infinity(X) + sum_g D_g*q_g(X) + m_t(X)`. Under CHT, PT fails even conditional on X (Equation (5.4)) and the Section 3/4 demeaning estimators are inconsistent; detrending (Procedure 5.1) restores consistency under CNAS + CHT + OVLS. +- **Minimum pre-treatment periods:** demeaning requires >= 1 pre-treatment period per cohort; detrending requires >= 2 (`g >= 3` in Appendix B). Cohorts/cells failing the minimum are not estimable. +- IPWRA consistency requires correct specification of either the propensity score model or the outcome model (doubly robust; p. 14). + +*Estimator equation (Equations (3.2), (4.11) in paper, as implemented):* + +The rolling transformation converts each (cohort, period) analysis into a standard cross-sectional treatment-effects problem. Common timing (Equation (3.2)): + + Y_dot_ir = Y_ir - (1/(S-1)) * sum_{q=1}^{S-1} Y_iq = Y_ir - Ybar_{i,pre} + +Staggered (Equation (4.11)): + + Y_dot_irg = Y_ir - (1/(g-1)) * sum_{s=1}^{g-1} Y_is = Y_ir - Ybar_{i,pre(g)} + +where: +- `Y_ir` = outcome for unit i in calendar period r +- `S` = common intervention date; `g` = first treatment period of unit i's cohort +- `Ybar_{i,pre}` / `Ybar_{i,pre(g)}` = unit-level average of pre-treatment outcomes +- `D_i` (common timing) / `D_ig` (staggered) = treatment / cohort indicator +- `X_i` = time-constant pre-treatment covariates +- Target parameters: `tau_r = E[Y_r(1) - Y_r(0) | D = 1]` (Equation (2.1)); `tau_gr = E[Y_r(g) - Y_r(infinity) | D_g = 1]` (Equation (4.2)) + +Linear RA, single regression per period (Equation (3.3)); `tau_hat_r` = coefficient on `D_i`: + + Y_dot_ir on 1, D_i, X_i, D_i*(X_i - Xbar_1), i = 1, ..., N + +with `Xbar_1 = N_1^{-1} sum_i D_i X_i` the covariate average over treated units (centering at `Xbar_1` ensures the coefficient on `D_i` recovers the ATT). No-covariate special case (Equation (3.4)) is plain DiD: + + tau_hat_r = (Ybar_1r - Ybar_0r) - (Ybar_{1,pre} - Ybar_{0,pre}) + +Detrending variant (Section 5 / Appendix B): replace demeaning with unit-specific linear detrending. Run unit-specific regressions `Y_it on 1, t` over `t = 1, ..., g-1` (Equation (5.6)); for `r >= g` compute out-of-sample predictions `Yhat_irg` and residuals `Yddot_irg = Y_ir - Yhat_irg`. In the simplest case (T=3, S=3, no covariates) the estimator of `tau_3` is the difference-in-difference-in-differences (Equation (5.7)): + + [(Ybar_13 - Ybar_12) - (Ybar_03 - Ybar_02)] - [(Ybar_12 - Ybar_11) - (Ybar_02 - Ybar_01)] + +Event-study / placebo transformations over ALL periods (Appendix D): demeaning (Equation (D.1)) `Y_dot_itg = Y_it - (1/(g-1)) sum_{q=1}^{g-1} Y_iq` for `t = 1,...,T`, `t != g-1`; detrending (Equation (D.2)) `Y_ddot_itg = Y_it - Yhat_itg` for `t = 1,...,T`, `t not in {g-2, g-1}`. Anchor periods (event time r = -1 under demeaning; r = -2, -1 under detrending) are excluded. + +*With covariates / doubly robust (Equations (E.1), IPW weights p. 55):* + +After the transformation, ANY standard cross-sectional TE estimator applies: linear RA, IPW, IPWRA (doubly robust), propensity-score/covariate matching, causal ML for high-dimensional X (Belloni et al. 2014; Chernozhukov et al. 2018). Per-(g,t) cell RA regression (Equation (E.1)): + + Y_dot_{i,g,t} = alpha_{g,t} + theta_{g,t} D_{i,g} + X_i' gamma_{g,t} + D_{i,g} (X_i - Xbar_g)' delta_{g,t} + eps_{i,g,t} + +IPWRA (workhorse; two-step per Wooldridge 2025a, Section 19.4): (1) logit propensity score `p_{g,t}(X_i; gamma) = Lambda(X_i' gamma_{g,t})` estimated by ML on the cell sample `A_{g,t}`; (2) weighted least squares of (E.1) with weights + + w_{i,g,t}(gamma) = D_{i,g} + (1 - D_{i,g}) * p_{g,t}(X_i; gamma) / (1 - p_{g,t}(X_i; gamma)) + +ATT(g,t) = coefficient on `D_{i,g}` (`theta_hat_{g,t} = e_2' eta_hat_{g,t}`). IPW is the special case omitting the outcome-regression component (estimating equation in Appendix E.4). The abstract states the doubly robust IPWRA "works particularly well in terms of bias and efficiency." + +*Standard errors (Appendix E):* +- Default: influence-function-based **multiplier bootstrap** for SEs and **simultaneous (sup-t) confidence bands** over the event-study path `{WATT(r) : r in R}` (Algorithm 1). Influence functions come from the stacked estimating equations; no re-estimation per bootstrap replication. +- Influence functions: RA (Appendix E.2) `IF_hat^{RA}_{i,g,t} = e_2' (Z'Z)^{-1} Z_i u_hat_{i,g,t}` - exact in finite samples; IPWRA (Appendix E.3) `IF_{eta,i,g,t} = Q_{w,g,t}^{-1} (w_{i,g,t} Z_{i,g} u_{i,g,t} + H_{g,t} IF_{gamma,i,g,t})` with block upper-triangular stacked Jacobian; IPW (Appendix E.4) `IF^{IPW}_{i,g,t} = psi_{i,g,t} - Gamma_{g,t}' IF_{gamma,i,g,t}`. IPW/IPWRA IFs include first-stage logit-score corrections `IF_{gamma,i,g,t} = A_{g,t}^{-1} X_i (D_{i,g} - p_{g,t}(X_i; gamma_{g,t}))` where `A_{g,t}` here is the logit information matrix. +- Bootstrap: **Rademacher multipliers** `xi_i in {-1, +1}` drawn **per unit i** (one multiplier per unit across all its cells/event times - unit-level clustering by construction, preserving within-unit dependence). Iterations: Algorithm 1 leaves B generic; B = 999 used in the Walmart application (p. 58). Bootstrap applied only to reported event-time coefficients, excluding anchor periods "set to zero by construction" (p. 50). IFs are centered across observations before multiplier draws for finite-sample stability. +- Clustering: unit (panel) level via the per-unit multiplier; no other clustering options, degrees-of-freedom, or t-vs-normal small-sample conventions discussed in the paper (small-sample inference is the subject of the companion paper, SSRN 5325686 - reviewed separately). +- Single-cell inference: for one cohort-time effect, standard cross-sectional TE inference applies directly (e.g., heteroskedasticity-robust SEs from (3.3); Stata `teffects`); IPWRA is a two-step M-estimator with standard asymptotics (Wooldridge 2007; Newey and McFadden 1994). Matching-based variants: standard matching software SEs reflect matching uncertainty (p. 20). + +*Aggregation (Section 6.2 / Appendix D.3; unnumbered equation):* + + WATT(r) = sum_{g in G_r} w(g,r) * ATT(g, g+r), w(g,r) = N_g / N_{G_r}, N_{G_r} = sum_{g in G_r} N_g + +cohort-size-weighted average over cohorts `G_r` for which `ATT(g, g+r)` is identified at event time `r = t - g` (Appendix E.1 phrases the weight as the number of treated units in cohort g contributing at event time r over the total treated units contributing at that event time). Estimable event times: `r = -1` excluded under demeaning; `r = -2, -1` excluded under detrending. Aggregated IF: `IF_{i,r} = sum_g omega_{g,r} IF_{i,g,g+r}`. + +*Edge cases:* +- Cohort with zero pre-treatment periods (demeaning) or fewer than two (detrending): transformation undefined -> cell not estimable; detrending rank condition `(J'_{g-1} J_{g-1})` invertible iff >= 2 distinct pre-treatment time points (Appendix B, Equations (B.1)-(B.3)). +- Anchor periods: event-study plots omit `r = -1` (demeaning) / `r = -2, -1` (detrending); bootstrap excludes them (p. 50). +- All units eventually treated (Section 4.3): no NT group -> state CPT relative to `Y_t(T)`; effects defined as `E[Y_r(g) - Y_r(T) | D_g = 1]` for `g in {S,...,T-1}`; no effect estimable for the final cohort; compute `Y_dot_irg` only for `g in {S,...,T-1}` and drop `D_infinity` from Steps 2-3 of Procedure 4.1; when `r = T`, cohort `D_T = 1` is the only control. By no anticipation, for `r < T` the ATTs coincide with the NT-referenced ones. +- Unbalanced panels (Section 4.4): apply demeaning/detrending to the observed data; a (g, r) cell is usable only if `Y_ir` is observed and there are enough pre-g observations (one for demeaning, two for detrending). Selection may depend on unobserved time-constant heterogeneity (like FE); detrending additionally allows trend heterogeneity to correlate with selection; selection must not be systematically related to shocks to `Y_it(infinity)`. +- Suspected anticipation: drop one or more periods just prior to the intervention from the pre-period average/trend and redo the analysis (Section 4.4); procedures apply unchanged with skipped periods. No anticipation is load-bearing for detrending: the second term of (B.8) vanishes only under no anticipation. +- Control-group choice: default control pool at (g, r) is `A_{i,r+1} = 1` (never-treated plus not-yet-treated as of r); an NT-only subset is allowed and lets the X-conditioning in CNAS be dropped. Appendix D.3 generalizes to pre- and post-treatment cells: comparison sample `A_{g,t} = {G_i = g} ∪ {G_i = 0} ∪ {G_i > max(g,t)}` (G_i = 0 denoting never treated), so pre-treatment placebo cells use cohorts treated after g and post-treatment cells cohorts still untreated at t. +- Incidental parameters: the unit-specific trend regressions (5.6) do NOT create an incidental parameters problem with small T; only removal of the CHT heterogeneity is needed, not "good" unit-level trend estimates (p. 28; logic of Wooldridge 2010, Section 11.7.2). +- Normalization differences vs CS: CS (2021) event studies are normalized to `r = -1`, while demeaned rolling estimates are deviations from each unit's own pre-treatment average - event-study points are not directly numerically comparable across the two (p. 30). +- Efficiency: when `r = g` the rolling RA with all controls reproduces the POLS/ETWFE estimator of Wooldridge (2025b); when `r > g` it does not, and under standard assumptions the rolling approach is inefficient for dynamic effects - the trade-off is applicability of doubly robust and matching estimators (p. 19). In the common-timing case, Theorem 3.1 gives full numerical equivalence between the per-period regressions (3.3) and pooled OLS (3.6). + +*Algorithm (Procedures 3.1, 4.1, 5.1; Algorithm 1 in paper):* + +Procedure 3.1 (Rolling Methods, Common Timing): +1. For a given time period `r in {S, ..., T}` and each unit i, compute `Y_dot_ir` as in (3.2). +2. Using all of the units, apply standard TE methods - such as linear RA, IPW, IPWRA, matching - to the cross section `{(Y_dot_ir, D_i, X_i) : i = 1, ..., N}`. + +Procedure 4.1 (Rolling Methods, Staggered Interventions): +1. For a given `g in {S,...,T}` and time period `r in {g,...,T}`, compute `Y_dot_irg = Y_ir - Ybar_{i,pre(g)}` as in (4.11). +2. Choose as the control group the units with `A_{i,r+1} = 1` (or, if desired, a subset, such as the NT group). +3. Using the subset of data with `D_ig + A_{i,r+1} = 1`, apply standard TE methods - such as linear RA, IPW, IPWRA, matching - to the cross section `{(Y_dot_irg, D_ig, X_i) : i = 1,...,N}`, with `D_ig` acting as the treatment indicator. + +Procedure 5.1 (Staggered Entry, detrending): +1. For a specified cohort `g in {S,...,T}`, run the unit-specific regressions `Y_it on 1, t`, `t = 1,...,g-1` (Equation (5.6)). For `r in {g,...,T}`, compute out-of-sample predictions `Yhat_irg` and residuals `Yddot_irg = Y_ir - Yhat_irg`. (Not needed for units treated prior to period g.) +2. Same as Procedure 4.1. +3. Identical to Procedure 4.1 with `Yddot_irg` replacing `Y_dot_irg`. + +Algorithm 1 (Multiplier Bootstrap for Simultaneous Inference on WATT_hat(r); Appendix E, p. 52): +1. For each group-time cell (g,t), compute `ATT_hat(g,t)` and observation-level influence contributions `IF_hat_{i,g,t}`. +2. Aggregate: `WATT_hat(r) = sum_g omega_{g,r} ATT_hat(g, g+r)`. +3. Construct event-time influence contributions `IF_hat_{i,r} = sum_g omega_{g,r} IF_hat_{i,g,g+r}`; center: `IF_tilde_{i,r} = IF_hat_{i,r} - (1/N) sum_j IF_hat_{j,r}`. +4. For each `b = 1,...,B`, draw independent unit-level Rademacher multipliers `xi_i^(b) in {-1,+1}` and compute `WATT_hat^{*(b)}(r) = sum_i xi_i^(b) IF_tilde_{i,r}` (IFs already contain sample-size scaling; no additional normalization). +5. Compute bootstrap SEs `se_hat_boot(r)` from the empirical variance of `{WATT_hat^{*(b)}(r)}`. +6. For each b, compute the supremum statistic `T*_b = sup_{r in R} |WATT_hat^{*(b)}(r) / se_hat_boot(r)|`. +7. With `c_hat^{sup}_{1-alpha}` the empirical (1-alpha)-quantile of `{T*_b}`, the simultaneous band is `WATT_hat(r) ± c_hat^{sup}_{1-alpha} * se_hat_boot(r)`, `r in R`. + +**Reference implementation(s):** +- Stata: user-written command `lwdid` - "implements the full procedure and multiplier-bootstrap inference"; see Hur, Lee and Wooldridge (2026) for details (pp. 30, 32). Walmart results replicable with `lwdid`. +- R: none mentioned in the paper. + +**Requirements checklist:** +- [ ] Rolling demeaning transformation (3.2)/(4.11) computed per (cohort g, period r) using ALL pre-g periods +- [ ] Rolling detrending transformation (5.6)/(D.2) via unit-specific OLS on (1, t) over pre-g periods, out-of-sample prediction for r >= g +- [ ] Minimum pre-period enforcement: >= 1 (demeaning), >= 2 (detrending); cells failing are dropped/not estimable +- [ ] Control pool per (g, r): NT + not-yet-treated (`A_{i,r+1} = 1`) by default; NT-only option +- [ ] Pre-treatment placebo cells use the Appendix D.3 comparison rule (`G_i > max(g,t)`) +- [ ] RA estimator (E.1) with treated-cohort-centered covariate interactions `D_{i,g}(X_i - Xbar_g)` +- [ ] IPWRA: logit PS by ML per cell + WLS of (E.1) with weights `w_{i,g,t}`; IPW special case +- [ ] Influence functions per E.2 (RA, exact), E.3 (IPWRA), E.4 (IPW) including first-stage logit-score corrections +- [ ] WATT(r) cohort-size-weighted aggregation over identified (g, g+r) cells +- [ ] Multiplier bootstrap (Algorithm 1): unit-level Rademacher draws, centered IFs, sup-t simultaneous bands; anchor periods excluded +- [ ] Anchor-period exclusion: r = -1 (demeaning); r = -2, -1 (detrending) +- [ ] Section 4.3 support: all-eventually-treated panels (drop D_infinity; last cohort as control at r = T; no effect for last cohort) +- [ ] Section 4.4 support: unbalanced panels (transform on observed data; per-cell observability checks) and anticipation-robustness period dropping +- [ ] Common-timing rolling RA reproduces plain DiD (3.4) with no covariates and matches pooled OLS (3.6) per Theorem 3.1 + +--- + +## Implementation Notes + +### Data Structure Requirements +- Panel data: random sample of N units observed for t = 1, ..., T periods; balanced panel is the baseline, unbalanced supported per Section 4.4 (transformation applied to observed data with per-cell observability requirements). +- Required variables: outcome `Y_it`; first-treatment period / cohort variable (`g in {S,...,T}`, never-treated coded separately - Appendix D.3 uses `G_i = 0` for never treated); time-constant pre-treatment covariates `X_i` (optional). +- Treatment must be absorbing (no reversibility) with common or staggered entry; at least one pre-treatment period overall (`1 < S <= T`). +- Identification treats each (g, r) analysis as a cross-sectional treatment-effects problem after the transformation; no unit fixed effects are estimated (unit heterogeneity removed by the transformation). + +### Computational Considerations +- Per-(g,r)-cell estimation: one transformation pass plus one cross-sectional fit (OLS, or logit-ML + WLS for IPWRA) per cell; cells are independent given the transformed data, so cell-level parallelization is natural. +- Detrending: N unit-specific 2-parameter OLS regressions per cohort (constant + trend on pre-g periods) - vectorizable; no incidental parameters problem with small T (p. 28). +- Multiplier bootstrap requires NO re-estimation per replication: perturb centered influence contributions only ("computationally efficient"; Appendix E). Storage: N x |R| matrix of aggregated IF contributions; B draws of N Rademacher signs. +- Monte Carlo evidence (Appendix C, common timing, T=6, S=4): works down to N = 100 with modest bias (average propensity score ~0.16, i.e. ~16 treated units); rolling IPWRA gives up at most ~2% SD relative to POLS/RA when everything is correctly specified (Table A2) and is uniformly the most efficient consistent estimator under conditional-mean misspecification (Table A3), with the smallest RMSE in all but one design. + +### Tuning Parameters + +| Parameter | Type | Default | Selection Method | +|-----------|------|---------|-----------------| +| Transformation (demean vs detrend) | choice | demeaning (Procedure 4.1) | Detrend (Procedure 5.1) when pre-treatment event-study estimates show unit-specific linear trends (upward/downward pre-trend pattern); costs event time r = -2 and requires >= 2 pre-periods per cohort | +| Control group | choice | all `A_{i,r+1} = 1` (NT + not-yet-treated) | NT-only subset optional; NT-only lets CNAS drop conditioning on X | +| TE estimator per cell | choice | none prescribed; IPWRA recommended | RA / IPW / IPWRA / matching / causal ML; IPWRA "works particularly well in terms of bias and efficiency" (abstract; Appendix C) | +| Propensity score model | model | logit (estimated by ML per cell) | Fixed in the paper's derivations (Appendix E.3/E.4) | +| Bootstrap repetitions B | int | none stated (generic in Algorithm 1) | B = 999 used in the application (p. 58) | +| Confidence level 1 - alpha | float | 95% bands shown in application | User choice; sup-t simultaneous bands are the default inference mode | +| Anticipation-robustness window | int | 0 (use all pre-g periods) | Drop 1-2 periods just before g as robustness check when no-anticipation is doubtful (Section 4.4); "recommendation is context specific" | + +### Relation to Existing diff-diff Estimators +- **CallawaySantAnna**: same (g, t) cell structure, same control pools (NT or NT + not-yet-treated), same RA/IPW/doubly-robust menu, and same multiplier-bootstrap-with-sup-t-bands inference style - but a different outcome transformation: CS uses the long difference `Y_ring_irg = Y_ir - Y_{i,g-1}` (Equation (4.12)), ignoring pre-(g-1) periods, while LWDiD's rolling demeaning (4.11) weights all pre-treatment periods by `1/(g-1)`. Both are consistent under no anticipation + conditional PT; the paper positions them as complementary ("it makes sense to try both", p. 4) with different biases under PT failure and neither uniformly more efficient (CS can win under strong positive serial correlation, being similar to first-differencing). Table 1 (p. 22) contrasts pre-treatment information use. CS is also normalized differently in event-study plots (to r = -1). diff-diff's `CallawaySantAnna` machinery (per-cell estimation loops, IPW/OR/DR components, multiplier bootstrap, event-study aggregation) is the closest structural template for an LWDiD implementation. +- **WooldridgeDiD (ETWFE)**: in the common-timing case, rolling RA is numerically equivalent to Wooldridge (2025b) POLS/ETWFE (Theorem 3.1, proof in Appendix A); in the staggered case equivalence holds only at r = g (instantaneous effects). Under standard error-component assumptions the (3.6) estimators are BLUE (Wooldridge 2025b, Theorem 6.2), so the transformation discards no useful information; the rolling approach trades some efficiency at r > g for doubly-robust/matching applicability. Theorem 3.1-type equivalence is a natural cross-estimator test target against diff-diff's existing `WooldridgeDiD`. +- **DifferenceInDifferences**: the no-covariate common-timing special case (3.4) is plain 2x2 DiD on (pre-average, period-r) means; with T=2, `Y_dot_2 = Y_2 - Y_1` and LWDiD coincides with the canonical estimator - another equivalence test target. +- Detrending (Procedure 5.1) has no current diff-diff analogue; (5.7) shows it is a difference-in-difference-in-differences in the simplest case. +- Reusable diff-diff infrastructure: `linalg.solve_ols()` for all per-cell OLS/WLS fits, `safe_inference()` for joint inference fields, existing logit-IPW propensity code and multiplier-bootstrap/sup-t band code from the CallawaySantAnna family. + +--- + +## Gaps and Uncertainties + +1. **Citation-year consistency (RESOLVED 2026-07-11 - use "Lee & Wooldridge (2025)"):** the SSRN working paper was first posted 27 Jul 2023 and the revision reviewed here is dated April 26, 2026 (last revised June 8, 2026), but the companion inference paper (SSRN 5325686, Section 9, p31) cites this paper as "Lee and Wooldridge (2025)" - the authors' own year handle. All PR surfaces (REGISTRY.md, docstrings, llms.txt, docs/references.rst) should cite "Lee & Wooldridge (2025)" with the SSRN 4516518 link, optionally noting the revision date. +2. **Small-sample inference deferred to companion paper:** no degrees-of-freedom corrections, t-vs-normal conventions, or clustering options beyond the unit-level multiplier are discussed anywhere in this paper (flagged by the estimation extractor for pp. 16-32 and the appendix extractor for Appendix E). This is the subject of the separate small-sample-inference companion paper (SSRN 5325686), whose review is being written separately - cross-reference that review for small-sample conventions. +3. **No overall-ATT or per-cohort aggregation:** only the event-study aggregation WATT(r) is defined (Section 6.2 / Appendix D.3; unnumbered equation). No overall ATT, per-cohort, or calendar-time aggregation appears in any extraction. An implementation offering such aggregations goes beyond the paper. +4. **Bootstrap defaults:** Algorithm 1 leaves B generic; the only concrete value is B = 999 in the application (p. 58). No recommended default is stated. Only Rademacher multipliers are specified - no alternative weight distributions are discussed. +5. **Anchor-period rationale (as printed, two phrasings):** p. 29 motivates excluded event times via degrees of freedom ("fitting a unit-specific mean and a unit-specific linear trend requires at least one and two pre-treatment periods, respectively"), while p. 50 refers to "anchor periods that are set to zero by construction" excluded from the bootstrap. Both extractors report their page's phrasing; whether an implementation should display r = -1 (and r = -2 under detrending) as hard zeros or omit them entirely should be settled against the `lwdid` Stata behavior. +6. **Notation collision on `A_{g,t}` (as printed):** the symbol is used for (i) the comparison sample of cell (g,t) (Appendix D.3/E.2), (ii) the stacked-moment Jacobian in E.3, and (iii) the logit information matrix in E.4 (flagged by the appendix extractor: "same symbol ... but a different object"). Also `A_{r+1}` in the main text is the control-pool indicator. Implementations should not conflate these. +7. **E.2 comparison-sample nuance:** Appendix E.2 defines `A_{g,t}` with an additional intersection `{i : T_i = t}`, indicating estimation on the cross-section of observations at calendar time t - notation not fully unpacked in the extraction; relevant for unbalanced-panel handling of the IF sample. +8. **Simulation coverage gaps:** results tables for Scenarios 2C and 4C (defined in Table A1: PS misspecified with correct mean; both misspecified) were not in any extractor's page range - only Table A2 (Scenario 1C) and Table A3 (Scenario 3C) are extracted. There are **no staggered-adoption Monte Carlos** in the paper (Appendix C is common-timing only), and no simulations exercising the detrending estimator were extracted. +9. **WATT weight phrasing:** Section 6.2 / Appendix D.3 give `w(g,r) = N_g / N_{G_r}` (cohort sizes), while Appendix E.1 phrases `omega_{g,r}` as the number of treated units in cohort g *contributing at event time r* over the total contributing at that event time. These coincide in a balanced panel; under unbalanced panels the E.1 phrasing (contributing units) is the operative one. Not a contradiction, but the balanced-panel shorthand should not be hard-coded. +10. **Minor page-boundary fragment:** a sentence on selection into the sample begins on p. 27 and ends on p. 28 (flagged by the edge-cases extractor); its content appears consistent with the Section 4.4 selection discussion (selection may depend on time-constant heterogeneity, and additionally on trend heterogeneity under detrending) but the exact sentence was split across extraction ranges. +11. **Reference implementation details:** `lwdid` internals are documented in Hur, Lee and Wooldridge (2026), which was not reviewed here; parity targets (e.g., exact placebo construction beyond Appendix D, default B, display conventions) require consulting that reference or the command itself. +12. **No contradictions between extraction files were found.** All overlapping content (Procedure 4.1 transcriptions, WATT(r) definitions, anchor-period exclusions, Walmart headline numbers WATT(1) = 0.032 (SE 0.005) vs Table A4 row r = 1, minimum pre-period requirements, CS comparison) is mutually consistent; apparent scope differences (e.g., the edge-cases extractor noting unbalanced panels "not addressed" in its pages while the estimation extractor covers Section 4.4) are page-range artifacts resolved in this synthesis. diff --git a/docs/methodology/papers/lee-wooldridge-2026-review.md b/docs/methodology/papers/lee-wooldridge-2026-review.md new file mode 100644 index 00000000..4a54cbdf --- /dev/null +++ b/docs/methodology/papers/lee-wooldridge-2026-review.md @@ -0,0 +1,329 @@ +# Paper Review: Simple Approaches to Inference with Difference-in-Differences Estimators with Small Cross-Sectional Sample Sizes + +**Authors:** Soo Jeong Lee (Southern Illinois University Carbondale), Jeffrey M. Wooldridge (Michigan State University) +**Citation:** Lee, S.J., & Wooldridge, J.M. (2026). Simple Approaches to Inference with Difference-in-Differences Estimators with Small Cross-Sectional Sample Sizes. SSRN Working Paper No. 5325686, dated February 3, 2026. https://ssrn.com/abstract=5325686 +**PDF reviewed:** /private/tmp/claude-501/-Users-igerber-diff-diff-LWDiD/fa768cf3-1977-474a-9ddf-b98c87817229/scratchpad/ssrn-5325686.pdf +**Review date:** 2026-07-11 + +--- + +## Methodology Registry Entry + +*Formatted to match docs/methodology/REGISTRY.md structure. This paper supplies the small-sample inference layer of the LWDiD estimator (whose estimation core is documented in lee-wooldridge-2025-review.md) - the section below is written as ADDITIONS to the LWDiD registry entry, not a separate estimator.* + +## LWDiD - small-sample inference layer + +**Primary source:** Lee, S.J., & Wooldridge, J.M. (2026). Simple Approaches to Inference with Difference-in-Differences Estimators with Small Cross-Sectional Sample Sizes. SSRN Working Paper No. 5325686. https://ssrn.com/abstract=5325686 + +Companion estimation core: Lee & Wooldridge (2025) [LW (2025)] - panel DiD estimators are obtainable via cross-sectional regressions after unit-level time-series ("rolling") transformations; this paper adds exact small-sample inference by collapsing the panel to `{(ΔȲ_i, D_i) : i = 1, ..., N}` in the spirit of Donald and Lang (2007). + +**Key implementation requirements:** + +*Assumption checks / warnings:* +- Sampling: data draws are i.i.d. across `i`; general dependence and changing distributions allowed across `t` (stated after eq. 2.2). Because inference is based on a cross-sectional regression, NO adjustment for serial correlation is needed, even under strong time-series dependence (unit-root-like processes need not push `ΔȲ_i` far from normality); large `T` requires no modification. +- No anticipation (NA), weakest version (eq. 2.14): `E[Y_it(1) − Y_it(0) | D_i = 1] = 0, t = 1, ..., S−1`. Sufficient: `Y_it(1) = Y_it(0), t = 1, ..., S−1`. +- Parallel trends (PT) (eq. 2.15): `E[Y_it(0) − Y_i1(0) | D_i] = E[Y_it(0) − Y_i1(0)] ≡ δ_t` for all `t = 2, ..., T`; `δ_t` unrestricted over time. Allows `D_i` correlated with the level `Y_i1(0)`; rules out assignment based on differential trends. PT implies (eq. 2.16, from LW 2025): `E[ΔȲ_i(0) | D_i] = α`; with NA this identifies `τ` and makes `τ̂_DD` conditionally unbiased. With small `N1` or `N0` the paper relies on unbiasedness, not consistency. +- Conditional PT with time-constant controls `X_i` (1×K) (eq. 2.17): `E[Y_it(0) − Y_i1(0) | D_i, X_i] = α_t + X_i β_t` (linearity imposed "because we have little choice in a small-N setting"; unconfoundedness in differences, not levels). +- Classical linear model (CLM) assumptions for EXACT inference (eqs. 2.7-2.9): `ΔȲ_i = α + τ D_i + U_i`; `E(U_i | D_i) = 0`; `U_i | D_i ~ Normal(0, σ_U²)`. Eq. (2.9) bundles conditional normality AND homoskedasticity - the paper flags homoskedasticity as a (technical) drawback; HC3 is the relaxation when N is not too small. With controls, the CLM version is (2.18)-(2.19). +- Normality justification (p32): because the method averages across time, the CLT across the time dimension often justifies the CLM normality assumption; works best with large `T0` and `T1`, but under exact joint normality and homoskedasticity applies with few periods and few units (even `N = 3`: two controls, one treated). A log outcome transform can make normality of the transformed outcome a better approximation (used in the Prop 99 application, p19). +- Sample-size checks: `N0 ≥ 1`, `N1 ≥ 1`, `N = N0 + N1 ≥ 3` (no controls); `N > K + 2` with K controls. Interacted-controls regression requires `N0 > K + 1` AND `N1 > K + 1` - NOT feasible with small `N1`. +- Staggered case: `N_∞ ≥ 2` needed when only never-treated units serve as controls; with `N_∞ ≥ 2`, treated cohorts may have as few as one unit (p26). Not-yet-treated (NYT) units are also valid controls under suitable NA + PT assumptions from LW (2025), per (7.7). +- No incidental-parameters problem: `ΔȲ_i` and `Ȳ̈_i` are linear functions of `{Y_it}` computed unit-by-unit; independence across i delivers independent cross-sectional observations. +- Caveat (p19, p32): the detrending advantage relies on unit-specific *linear* trends; with pre-trend patterns too complex for linear detrending the method "will not always work better than existing alternatives." + +*Inference procedures (with equation/procedure numbers):* + + Exact t (core result, eq. 2.10): under (2.7) + (2.9), conditional on D_i, + tau-hat_DD is exactly normal and + + (tau-hat_DD − tau) / se(tau-hat_DD) ~ T_{N−2} (2.10) + + - Regression (2.5): DeltaYbar_i on 1, D_i (i = 1, ..., N); usual OLS t + statistic and standard error; exact tests of any null; CIs from T_{N−2} + percentiles have exact coverage. Valid for any N0 >= 1, N1 >= 1, N >= 3. + - With K controls (2.18)-(2.19): OLS on DeltaYbar_i on 1, D_i, X_i requires + N > K + 2; t statistic is exactly T_{N−K−2}. + - Preferred full regression when N0 > K+1 and N1 > K+1 (unnumbered, p9): + DeltaYbar_i on 1, D_i, X_i, D_i·(X_i − Xbar_1), identical to separate + regressions for D_i = 0 and D_i = 1. + - N1 = 1 (single treated unit): the t statistic tau-hat_DD / se(tau-hat_DD) + is the "studentized residual" from outlier analysis (Wooldridge 2025a, + Section 9.5) - the test asks whether the single treated unit is an + "outlier" relative to controls. No particular number of time periods + required provided (2.9) holds (contrast Hagemann 2025). + - Detrending (Section 3): identical CLM logic for tau-hat_DT from (3.4), + valid even with N1 = 1: Ybar-ddot_i = alpha + tau_DT·D_i + U_i, + U_i | D_i ~ Normal(0, sigma^2) (unnumbered display, p11). + - Per-period effects (2.20): Y-dot_it on 1, D_i gives tau-hat_{t,DD} for + t = S, ..., T; identical to a standard panel DiD on periods + {1, ..., S−1, t}. Exact CI per tau_t under CLM. CAUTION (p10): SEs for + linear combinations of the tau-hat_{t,DD} are not easily obtained + (serial-correlation-induced dependence across per-period estimates). + - Alternative baselines: CS-style long difference (2.21) + Y-ring_it = Y_it − Y_{i,S−1}; anticipation guard (2.22) uses + Ybar_{i,S0} = (1/S0)·sum_{r=1}^{S0} Y_ir with S0 < S−1. + - Second differencing (unnumbered, p12): + Y-dddot_it = (Y_it − Y_{i,S−1}) − (Y_{i,S−1} − Y_iR), 1 <= R < S−1; + a triple-difference form, no pre/post averaging, so likely more + sensitive to normality violations. + + Staggered rollout (Section 7.1): cohorts g in {S, ..., T} with indicators + D_ig; never-treated D_{i,infinity}; tau_gt (7.1), cohort averages tau_g (7.2). + Transformations use only data through g−1: demeaning regressions (7.3) then + Y-dot_itg = Y_it − Ybar_{i,pre(g)} (7.4); detrending regressions (7.5) then + Y-ddot_itg = Y_it − A-hat_ig − B-hat_ig·t (7.6). Valid controls at (g,t) per + (7.7): D_{i,t+1} + ... + D_iT + D_{i,infinity} = 1 (any subset valid; all + controls preferred for efficiency). + + tau-hat_gt: Y-dot_itg on 1, D_ig using D_ig + C_{i,t+1} = 1 (7.8) + tau-hat_g: Ybar-dot_ig on 1, D_ig using D_ig + D_{i,inf} = 1 (7.10) + + with Ybar-dot_ig (and Ybar-ddot_ig) the post-treatment time averages (7.9). + Under homoskedastic normality, exact inference applies in (7.8)/(7.10) even + if N_g = 1 and the number of control units is small ("We can appeal to + exact theory if N_g is small, including N_g = 1", p28). + + Aggregated effect tau-hat_omega (7.11) with cohort-share weights + omega-hat_g = N_g / (N_S + ... + N_T) (7.12). Via the difference-in-means + representation (7.13) and algebra (7.14)-(7.17), define the composite + outcome (7.18): + + Ybar-dot-bar_i = D_iS·Ybar-dot_iS + ... + D_iT·Ybar-dot_iT + + D_{i,infinity}·( sum_{g=S}^{T} omega-hat_g·Ybar-dot_ig ) (7.18) + + (eventually-treated unit: its own cohort's post-average residual; + never-treated unit: the omega-hat_g-weighted average of its cohort-specific + post-average residuals across all treated cohorts). With the ever-treated + indicator D_i = D_iS + ... + D_iT (7.15), run the single cross-sectional + regression + + Ybar-dot-bar_i on 1, D_i, i = 1, ..., N (7.19) + + tau-hat_omega is the coefficient on D_i. Replace Ybar-dot_ig with + Ybar-ddot_ig everywhere for the detrended version. Advantages: (i) + automatically accounts for correlations among the tau-hat_g; (ii) usable + whether N_treat / N_control are small or large; (iii) if both are even + moderately large, HC3 from (7.19) is justified asymptotically. + +where: +- `N`, `N0`, `N1` = total, control, treated cross-sectional unit counts; `T0`, `T1` = pre/post period counts; `S` = first treated period +- `D_i` = time-constant treatment-group indicator (2.1); `W_it = D_i · post_t` (2.2) +- `ΔȲ_i ≡ Ȳ_{i,post} − Ȳ_{i,pre}` = the rolling transformation (2.4); `Ẏ_it` = demeaned out-of-sample residual (2.12); `Ÿ_it` = detrended outcome (3.2); `Y̊_it` = long difference vs period S−1 (2.21) +- `U_i` = cross-sectional regression error with variance `σ_U²`; `T_{N−2}`, `T_{N−K−2}` = t distributions with the indicated df +- `τ̂_DD` / `τ̂_DM` = demeaning estimator (Procedure 2.1); `τ̂_DT` = detrending estimator (Procedure 3.1); `τ̂_{t,DD}`, `τ̂_{t,DT}` = per-period versions +- Staggered: `g` = cohort (time of first treatment); `D_ig` = cohort indicator; `D_{i,∞}` = never-treated indicator; `N_g`, `N_∞` = cohort sizes; `N_treat = N_S + ... + N_T`; `ω̂_g` = cohort share (7.12); `Ȳ̇_ig` / `Ȳ̈_ig` = cohort-specific post-average transformed outcomes (7.9); composite outcome per (7.18) +- `Â_i, B̂_i` = unit-specific pre-period intercept and linear-trend slope (Procedure 3.1); `R` = reference period for second differencing; `S0` = anticipation-guard pre-period count (2.22) + +*Standard errors:* +- Default: usual OLS standard error from the collapsed cross-sectional regression, with exact `T_{N−2}` (no controls) or `T_{N−K−2}` (K controls) reference distribution under CLM assumptions (2.7)-(2.9) / (2.18)-(2.19). Exact coverage for CIs; valid down to `N = 3` and `N1 = 1` (or `N_g = 1` in the staggered case). +- Alternative: HC3 heteroskedasticity-robust SEs (the MacKinnon and White (1985) estimator, per Simonsohn (2021) commenting on Young (2019)) when N is "not too small" - the paper's operational phrasing is "provided there are at least a handful of treated units" (p32). Exact homoskedastic-normal inference is the fallback when treated units are too few for HC3. +- Randomization inference: exact p-values for the sharp null that all treatment effects are zero; two-sided RI p-value = `c/N` where `c` = number of permutation test statistics as or more extreme than observed and `N` = total number of permutations (p7 - note this `N` is the permutation count, an overload of the sample-size symbol). Does not rely on normality. Stata: `ritest`, or the Stata `lwdid` command's `ri` option. In the Prop 99 application, Procedure 3.1's exact p-value (0.021) and RI p-value with 1,000 replications (0.020) are nearly identical (Table 3, Note 2, p23). +- Clustering (Section 8.2, pp30-31): because the final regressions are cross-sectional, SEs can be clustered at a level higher than i (e.g., i = county, policy at state level, cluster at state, given sufficiently many treated and control states) - cites Abadie, Athey, Imbens and Wooldridge (2023). Clustering can also be done separately by time period. With spatially correlated treatment assignment, use heteroskedasticity-and-spatial-correlation robust ("SHAC") standard errors (Conley 1999). It does not matter how large T0 and T1 are. + +*Edge cases:* +- `N1 = 1` (single treated unit): detection trivial from treatment counts -> exact t inference still valid; the t statistic is the studentized residual / outlier statistic (p6-7). Same for the detrending estimator (p11). +- `N_g = 1` (single treated unit per cohort, staggered): -> exact inference in (7.8)/(7.10) under homoskedastic normality (pp27-28). +- Never-treated group too small: NT-only control strategy requires `N_∞ ≥ 2` (pp26-27) -> otherwise use not-yet-treated controls per (7.7). +- Periods with no new treated units (staggered): -> no treatment effects estimated in those periods (p26). +- Control-group choice: never-treated only, or any NYT unit under NA + PT per (7.7); all possible controls preferred for efficiency but any subset (including NT-only) valid (p27). +- Heteroskedasticity: "always an issue in cross-sectional regressions" -> use HC3 provided at least a handful of treated units (p32); with too few treated units, fall back to exact homoskedastic-normal inference. +- Too few observations for controls: `N ≤ K + 2` -> the with-controls exact result unavailable; interacted regression additionally needs `N0 > K + 1` and `N1 > K + 1` (p9). +- Complex nonlinear pre-trends: linear detrending inadequate -> method may be more biased than SC/SDiD; not universal (p19, p32). With large `S`, higher powers (`t²`, `t³`) can be added in step 1, but removing too much variation hurts detection power (p12). +- Anticipation: -> replace `Ȳ_{i,pre}` with `Ȳ_{i,S0}` averaging only the first `S0 < S − 1` pre-periods (2.22), or use a long difference `Y_it − Y_{i,S0}`. +- Seasonality (quarterly/monthly/weekly data): -> include seasonal dummies in step 1 of Procedure 2.1 or 3.1 to deseasonalize/detrend; afterwards the procedure is exactly the same (p12). +- Linear combinations of per-period effects: -> SEs not easily obtained because of serial-correlation-induced dependence across per-period estimates (p10); the aggregated regression (7.19) is the paper's device for a valid SE on the weighted average. +- Choosing the number of pre-treatment periods (Section 8.1, p30): robustness via varying the data's starting point (varying `T0`), in the spirit of Rambachan and Roth (2023); the approach does not rely on large `T0` or `T1` (but does rely on normality). Fewer pre-treatment periods may suffice when policy is based on past outcomes. +- Repeated cross-sections (pp13-14): aggregate micro outcomes to the assignment level, `Ȳ_st = Σ_{i∈(s,t)} w_ist Y_ist` with weights summing to 1 (equal weights `1/n_st` allowed; survey/design weights usable); random sampling within `(s,t)` cells plus large cell sizes justify treating the cell means as the population means (Donald and Lang 2007) with no adjustment for sampling error; then Sections 2-3 procedures apply unmodified. Sub-annual frequency uses `Ȳ_stq` with quarter dummies `q ∈ {q1, ..., q4}` in the transformation step. + +*Procedures (Procedure 2.1 / 3.1 in paper):* + +Procedure 2.1 (Unit-Specific Demeaning), p8: +1. Obtain `Ȳ_{i,pre}` for each i from the pre-intervention regression `Y_it on 1, t = 1, ..., S−1` (2.11) (the lone coefficient is `Ȳ_{i,pre}`). +2. In each post-intervention period, form out-of-sample residuals (prediction errors): `Ẏ_it = Y_it − Ȳ_{i,pre}, t = S, ..., T` (2.12). +3. Average the out-of-sample residuals to obtain the same regressand as in (2.5): `Ȳ̇_i ≡ (1/(T−S+1)) Σ_{t=S}^{T} Ẏ_it = Ȳ_{i,post} − Ȳ_{i,pre} = ΔȲ_i`. +4. Obtain the average effect `τ̂_DM`, its standard error and confidence interval, from `Ȳ̇_i on 1, D_i, i = 1, ..., N` (2.13). + +Procedure 3.1 (Unit-Specific Detrending), p11 (easily modified for higher-order trends): +1. For each i, obtain `Â_i, B̂_i` from the pre-treatment periods by regressing `Y_it on 1, t, t = 1, ..., S−1` (3.1). +2. For the post-treatment periods, remove the pre-treatment trends: `Ÿ_it = Y_it − Ŷ_it ≡ Y_it − Â_i − B̂_i·t, t = S, ..., T` (3.2), where `Ŷ_it ≡ Â_i + B̂_i·t` is the projected value. +3. For each unit, average the adjusted outcomes: `Ȳ̈_i ≡ (1/(T−S+1)) Σ_{t=S}^{T} Ÿ_it` (3.3). +4. Obtain the average effect `τ̂_DT`, its standard error and confidence interval, from `Ȳ̈_i on 1, D_i, i = 1, ..., N` (3.4). + +Staggered analogue (Section 7.1): per-cohort transformations (7.3)-(7.6) using only data through g−1 (compute for every unit, sort out valid controls afterwards); per-(g,t) regressions (7.8); cohort-average regressions (7.10); aggregate via composite-outcome regression (7.19). + +Synthetic-control diagnostic (Section 4, eqs. 4.1-4.2): after transformation, the cross-sectional average of control residuals `{N0^{-1} Σ_{i=1}^{N0} Ẏ_it : t = 1, ..., T}` (4.1) acts as the synthetic control for the average of treated-unit residuals `{N1^{-1} Σ_{i=N0}^{N0+N1} Ẏ_it : t = 1, ..., T}` (4.2) (as printed, the lower summation index in (4.2) is `i = N0`; treated units are the `N1` units indexed after the `N0` controls). With detrending, `Ÿ_it` replaces `Ẏ_it`. Success = close agreement of the two average residual series over `t = 1, ..., S−1`; estimated treatment effects are the differences for `t ≥ S` and are exactly the coefficients in (2.5) or (3.4). + +**Reference implementation(s):** +- Stata: `lwdid` (publicly available user-written command; implements Procedure 2.1 and Procedure 3.1; randomization-inference p-values via its `ri` option). Stata `ritest` cited for RI generally. Stata 18 `sdid` package used for the SDiD comparisons in the castle-laws application. + +**Requirements checklist:** +- [ ] Exact t inference on the collapsed cross-sectional regression: `T_{N−2}` df without controls, `T_{N−K−2}` with K controls (eq. 2.10; p9) +- [ ] Validity down to `N0 ≥ 1, N1 ≥ 1, N ≥ 3`; guard `N > K + 2` with controls; interacted controls require `N0 > K + 1` and `N1 > K + 1` +- [ ] `N1 = 1` supported (studentized-residual interpretation); `N_g = 1` supported in the staggered case +- [ ] Per-period effects via (2.20) with exact CIs; per-period equivalence to panel DiD on `{1, ..., S−1, t}` +- [ ] Alternative baselines: CS-style long difference (2.21); anticipation guard `Ȳ_{i,S0}` (2.22) +- [ ] Detrending Procedure 3.1 with the same exact-inference layer (3.4); optional higher-order trends and seasonal dummies +- [ ] Staggered aggregation via composite outcome (7.18) and single regression (7.19) with cohort-share weights (7.12) +- [ ] Staggered control-group options: never-treated (requires `N_∞ ≥ 2`) or not-yet-treated per (7.7) +- [ ] HC3 SEs offered when there are at least a handful of treated units +- [ ] Randomization inference for the sharp null (two-sided p = c / #permutations) +- [ ] Higher-level clustering and SHAC (Conley 1999) SEs available for larger cross sections (Section 8.2) +- [ ] Repeated cross-sections handled by cell-level aggregation with weights summing to 1 (Donald-Lang) +- [ ] Synthetic-control-style diagnostic plot of (4.1) vs (4.2) residual averages + +--- + +## Implementation Notes + +### Data Structure Requirements + +- Balanced panel of `N` cross-sectional units over `T` periods, intervention at `S ∈ {2, ..., T}` remaining in place through `T` (common timing, Sections 2-6). Pre-periods `1, ..., S−1`; post periods `S, ..., T`. +- Staggered rollout (Section 7): cohorts `g ∈ {S, ..., T}` = time of first treatment; cohort indicators `D_ig`; never-treated indicator `D_{i,∞}`. Transformations for cohort g use only data through `g−1`. Easiest to compute the transformation for every unit, then sort out valid control units afterwards (p26-27). +- Time-constant controls `X_i` (1×K) enter the collapsed cross-sectional regression when N is large enough. +- Repeated cross-sections: aggregate micro-level outcomes `Y_ist` to the assignment level (e.g., state-year) with weights `w_ist` summing to 1 within each `(s,t)` cell (equal weights `1/n_st` allowed; survey/design weights usable); quarterly-or-higher frequency uses state-year-quarter means `Ȳ_stq` with quarter dummies (pp13-14). + +### Computational Considerations + +- All estimation reduces to unit-by-unit OLS on pre-periods (a constant, or constant + trend, optionally + seasonal dummies) followed by ONE cross-sectional OLS regression; no incidental-parameters problem (transformed outcomes are linear functions of `{Y_it}` computed per unit; independence across i gives independent observations). +- No serial-correlation correction of any kind: strong dependence (e.g., time-constant unobserved effects) is eliminated by differencing post- and pre-averages; large `T` requires no modification. +- The demeaning estimator reproduces the standard DiD/TWFE estimator: pooled regression (2.3) `Y_it on 1, D_i, post_t, W_it` (equivalently full unit and time FE, per Wooldridge 2025b, 2010) equals the collapsed cross-sectional regression (2.5) via (2.6) `τ̂_DD = ΔȲ_treat − ΔȲ_control`. +- Per-period coefficients from (2.20) are identical to running a standard panel DiD on time periods `{1, ..., S−1, t}`. +- Staggered aggregation needs no covariance matrix across `τ̂_g`: the composite-outcome regression (7.19) delivers `τ̂_ω` and its SE in one pass (the difference-in-means algebra (7.13)-(7.17) uses `D_i · D_ig = D_ig` and `D_i + D_{i,∞} = 1`). +- Randomization inference used 1,000 replications in the Prop 99 application (Table 3, Note 2). + +### Tuning Parameters + +| Parameter | Type | Default | Selection Method | +|-----------|------|---------|-----------------| +| Transformation | categorical: demean (Procedure 2.1) / detrend (Procedure 3.1) | none stated | Detrend when unit-specific trends plausible/assignment depends on trends; demean is badly biased under heterogeneous trends (Table 2); detrend caveat for complex nonlinear pre-trends (p19) | +| Trend order | integer (1 = linear; `t²`, `t³` possible with large `S`) | linear | Higher powers only with large `S`; removing too much variation hurts detection power (p12) | +| Seasonal dummies | boolean/set | off | Include in step 1 for quarterly/monthly/weekly data (p12) | +| Pre-treatment baseline | full pre-average (2.4) / period S−1 long difference (2.21) / first `S0` periods (2.22) | full pre-average | `S0 < S−1` guards against anticipation; any (even weighted) average of pre-periods valid | +| `R` (second differencing) | integer `1 ≤ R < S−1` | not used | Only for the triple-difference variant (p12); more sensitive to normality violations | +| Control group (staggered) | never-treated only / any not-yet-treated subset per (7.7) | not stated | All possible controls preferred for efficiency; NT-only requires `N_∞ ≥ 2` | +| SE type | exact-normal OLS t / HC3 / RI / clustered / SHAC | exact-normal OLS t | HC3 with "at least a handful of treated units" (p32); RI needs no normality; clustering/SHAC for larger N (Section 8.2) | +| RI permutations | integer | not stated (1,000 used in application) | Two-sided p = c / #permutations | +| `T0` (pre-period span) | data window choice | full window | Sensitivity analysis by varying the starting point, Rambachan-Roth (2023) spirit (Section 8.1) | +| Donor pool | subset of controls | all `N0` controls | Paper deliberately uses ALL donors via transformation (contrast SC's sparse weights); subset robustness in Sections 6.2 / Appendix A | +| Aggregation weights `ω̂_g` | cohort shares (7.12) | `N_g / N_treat` | Fixed by the target parameter `τ_ω` (7.11) | +| RC cell weights `w_ist` | weights summing to 1 per cell | equal `1/n_st` | Survey/design weights allowed (pp13-14) | + +### Relation to Existing diff-diff Estimators + +- This paper is the inference companion to LW (2025), the estimation core of the LWDiD estimator under evaluation in third-party PR #588; the registry additions above extend that entry rather than defining a new estimator. +- Equivalences stated in the paper: the demeaning estimator equals the standard 2x2/pooled DiD (2.3) and the TWFE estimator (Wooldridge 2025b, 2010) - i.e., the point estimates coincide with `DifferenceInDifferences` / `TwoWayFixedEffects` on collapsed data; only the inference layer differs (exact t on the collapsed cross-section vs. cluster-robust panel inference, which "performs poorly" with small N or N1). +- The long-difference baseline (2.21) `Y̊_it = Y_it − Y_{i,S−1}` is explicitly identified as the transformation used by Callaway and Sant'Anna (2021) (`CallawaySantAnna` in diff-diff). +- Section 4 positions the method against synthetic control (ADH 2010) and synthetic DiD (AAHIW 2021) (`SyntheticDiD` in diff-diff): SDiD requires weakly dependent (I(0)) series and large `N0, T0, T1`; the exact approach has minimal restrictions on `N0, N1, T0, T1`, gives per-period effects with CIs ("a feature not allowed by SDiD methods", p32), and accommodates heterogeneous trends - but can be more biased than SDiD when pre-trend differences are complicated, and can be considerably less efficient. For staggered designs, AAHIW (2021)'s appendix suggestion (split by adoption date, SDiD per cohort with never-treated donors) is "exactly what we propose with our unit-specific demeaning or detrending" (p29), making the two directly comparable. +- The second-differencing variant (p12) yields a difference-in-difference-in-differences estimator (cf. `TripleDifference`). +- Randomization inference for the sharp null and Conley SHAC standard errors connect to diff-diff's existing placebo/RI and spatial-SE machinery. + +### Replication Targets + +All California numbers use log per-capita cigarette sales (ADH used levels), California as the single treated state (`N1 = 1`), 19 pre-treatment years (1970-1988), 12 treatment years (1989-2000). + +**Table 3 (p23): Estimated ATTs using 38 states as the donor pool** + +| | Procedure 2.1 (DiD) | Procedure 3.1 (Unit-Specific Detrending) | SC | Synthetic DiD | +|---|---|---|---|---| +| Average Effect | -0.422*** (0.121) | -0.227** (0.094) | -0.304*** (0.112) | -0.286*** (0.097) | +| tau_1989 | -0.168* (0.096) | -0.043 (0.059) | | | +| tau_1995 | -0.484*** (0.137) | -0.282** (0.112) | | | +| tau_2000 | -0.667*** (0.164) | -0.403** (0.152) | | | + +Note 1: Standard errors in parentheses. \*\*\* p < 0.01, \*\* p < 0.05, \* p < 0.1. +Note 2: For Procedure 3.1, the exact-inference p-value (under normality) is **0.021**, and the randomization-inference p-value (1,000 replications) is **0.020**. The two are nearly identical. + +The table reports per-period effects only for 1989, 1995, 2000; the framework estimates all 12 post-period effects. + +**PR #588 cross-check:** PR #588 claims "demeaning ATT = -0.422, detrending ATT = -0.227" for Table 3. This MATCHES the extracted Table 3 Average Effect row exactly (Procedure 2.1 = -0.422, Procedure 3.1 = -0.227). + +**Table 4 (p25): Estimated ATTs using AL, AR, LA, MS as the donor pool** (four Southern states a priori NOT similar to California; Section 6.2) + +| | Procedure 2.1 (DiD) | Procedure 3.1 (Unit-Specific Detrending) | SC | Synthetic DiD | +|---|---|---|---|---| +| Average | -0.556*** (0.080) | -0.215** (0.039) | -0.571*** (0.034) | -0.392*** (0.030) | +| 1989 | -0.247 (0.107) | -0.027 (0.052) | | | +| 1995 | -0.611*** (0.077) | -0.259** (0.055) | | | +| 2000 | -0.839*** (0.032) | -0.377** (0.115) | | | + +Note: standard errors in parentheses: \*\*\* p<0.01, \*\* p<0.05, \* p<0.1. Key finding (p25): Procedure 3.1 with the 4-state pool (-0.215) closely mirrors the all-38-state result (-0.227); SC and SDiD yield larger-magnitude estimates, reflecting poorer pre-treatment fits (Figure 6). + +**Table A1 (p35): Estimated ATTs using IL, IA, MN and OH as the donor pool** (Appendix A, Midwestern pool) + +| | Procedure 2.1 (DiD) | Procedure 3.1 (Unit-Specific Detrending) | SC | Synthetic DiD | +|---|---|---|---|---| +| Average | -0.413** (0.118) | -0.198* (0.079) | -0.437** (0.184) | -0.275* (0.154) | +| 1989 | -0.178* (0.071) | -0.040 (0.045) | | | +| 1995 | -0.462** (0.133) | -0.239* (0.088) | | | +| 2000 | -0.655** (0.183) | -0.363* (0.136) | | | + +Note: standard errors in parentheses: \*\*\* p<0.01, \*\* p<0.05, \* p<0.1. + +**Castle-laws application (Section 7.2, pp29-30), staggered:** data from Cunningham (2021); all 50 U.S. states, 2000-2010; adoptions: 1 state in 2005, 13 in 2006, 4 in 2007, 2 in 2008, 1 in 2009; `N_treat = 21` eventually treated, `N_control = 29` never-treated; outcome = log of annual homicides per 100,000 state residents. Using regression (7.19) with the composite outcome (7.18): +- Demeaning: **τ̂_ω ≈ 0.092** (about 9.2% more homicides); usual OLS SE = 0.057, t ≈ 1.61 (not quite significant at 10%, two-sided); HC3 t statistic = 1.50. +- Detrending (Ȳ̈ composite): estimate decreases to **0.067**; HC3 SE 0.055, t ≈ 1.21. +- SDiD (Stata 18 `sdid`): estimate **0.099**, placebo-method SE 0.069 (t = 1.41) - close agreement with the demeaning method. + +**Monte Carlo Table 2 (p18), headline cells:** common timing, N = 20, T = 20, treatment at t = 11, 1,000 replications, true average post-treatment effect = 2 by construction. Full table as extracted: + +Scenario 1, P(D_i = 1) = 0.32 (Sample ATT 1.991): + +| Estimator | Average Effect | Bias | SD | RMSE | Coverage | Avg SE | +|---|---|---|---|---|---|---| +| Demeaning | 3.905 | 1.914 | 5.10 | 5.445 | 0.94 | 4.96 | +| Detrending | 2.000 | 0.009 | 1.73 | 1.734 | 0.96 | 1.74 | +| Detrending (HC3) | 2.000 | 0.009 | 1.73 | 1.734 | 0.96 | 1.87 | +| SC | 1.616 | -0.375 | 2.14 | 2.177 | 0.97 | 3.27 | +| SDiD | 2.383 | 0.392 | 1.77 | 1.808 | 0.96 | 2.56 | + +Scenario 2, P(D_i = 1) = 0.24 (Sample ATT 2.011): + +| Estimator | Average Effect | Bias | SD | RMSE | Coverage | Avg SE | +|---|---|---|---|---|---|---| +| Demeaning | 4.264 | 2.253 | 5.67 | 6.101 | 0.93 | 5.48 | +| Detrending | 1.969 | -0.042 | 1.89 | 1.892 | 0.95 | 1.91 | +| Detrending (HC3) | 1.969 | -0.042 | 1.89 | 1.892 | 0.93 | 2.04 | +| SC | 1.622 | -0.389 | 2.35 | 2.384 | 0.94 | 2.73 | +| SDiD | 2.395 | 0.384 | 1.89 | 1.925 | 0.95 | 2.25 | + +Scenario 3, P(D_i = 1) = 0.17 (Sample ATT 1.996): + +| Estimator | Average Effect | Bias | SD | RMSE | Coverage | Avg SE | +|---|---|---|---|---|---|---| +| Demeaning | 4.566 | 2.570 | 6.78 | 7.254 | 0.94 | 6.43 | +| Detrending | 2.161 | 0.165 | 2.37 | 2.380 | 0.95 | 2.26 | +| Detrending (HC3) | 2.161 | 0.165 | 2.37 | 2.380 | 0.91 | 2.60 | +| SC | 2.962 | 0.966 | 2.92 | 3.078 | 0.92 | 2.85 | +| SDiD | 2.615 | 0.619 | 2.35 | 2.435 | 0.94 | 2.33 | + +DGP for regenerating (Section 5.1, Table 1): `C_i ~ N(0, σ_C²)`, `G_i ~ N(1, σ_G²)`, σ_C = 2, σ_G = 1; AR(1) errors `U_i1 ~ N(0, sqrt(2/(1−ρ²)))`, `U_it = ρ·U_{i,t−1} + A_it`, `A_it ~ N(0,2)`, ρ = 0.75; `Y_it(0) = λ_t·fs_t − C_i + G_i·t + U_it`, `Y_it(1) = Y_it(0) + δ_t·fs_t + V_it`, `V_it ~ N(0, √2)`; logistic assignment `Pr(D_i = 1) = 1[α_0 − α_1·C_i + α_2·G_i + A_i > 0]`, `A_i ~ Logistic(0,1)`; treatment-rule parameters (α_0, α_1, α_2): Scenario 1 = (−1, −1/3, 1/4), Scenario 2 = (−1.5, 1/3, 1/4), Scenario 3 = (−2, 1/3, 1/4); time FE (λ_1..λ_20) = (0, 0, 0, 0, 0.2, 0.6, 0.7, 0.8, 0.6, 0.9, 0.9, 1, 1.1, 1.3, 1.2, 1.5, 0.6, 1.4, 1.8, 1.9); treatment effects (δ_11..δ_20) = (1, 2, 3, 3, 3, 2, 2, 2, 1, 1); δ_t = 0 for t < 11. Demeaning is badly biased here because the DGP has unit-specific trends `G_i·t` correlated with assignment. + +**SC unit weights (Figure 4, p22, 38-state pool):** positive weights on only six donor states - Nevada (0.31), Montana (0.30), Utah (0.28), Connecticut (0.06), Colorado (0.03), New Hampshire (0.02). Useful for validating an SC comparison harness. + +--- + +## Gaps and Uncertainties + +**As-printed internal inconsistencies in the paper (flagged by the second extractor; carried verbatim):** + +1. (p22 vs Table 3, p23): the p22 text says the estimated effect grows over time, "starting off small but and reaching -0.403 (t = -1.52) by the year 2000" - but Table 3's tau_2000 for Procedure 3.1 is -0.403 with SE 0.152, which gives t ≈ -2.65, not -1.52. Extractor note as printed: "the -0.403 with SE 0.152 in Table 3 gives t ≈ -2.65; the t = -1.52 in the text appears to refer to a different specification or is possibly an inconsistency in the working paper - transcribed as printed." +2. (p17 vs Table 2, p18): the p17 text says "the SD for SC is 1.73, while the average standard error is 2.66," which does not match Table 2's SC row for Scenario 1 (SD 2.14, Avg SE 3.27); transcribed as printed, likely a working-paper inconsistency - the 1.73/1.74 pair belongs to the Detrending row. + +**Contradictions between extraction files (both resolved against the PDF, 2026-07-11):** + +3. Working-paper date: RESOLVED - the PDF cover page (p1) reads "February 3, 2026". The citation above uses that date. (An earlier metadata source said January 3; that was wrong.) +4. Section numbering for the closing material: RESOLVED - the PDF contains Section 8.1 (choosing the pre-treatment window, p30), Section 8.2 "Clustering and Spatial Correlation with Larger Cross Sections" (pp30-31), and Section 9 "Concluding Remarks" (pp31-32). The first extraction's "Section 8 (conclusion)" description was an inference from the intro, not a read of those pages. Note: Section 9's opening sentence cites the companion transformation paper as "Lee and Wooldridge (2025)" - the authors' own year handle for SSRN 4516518. + +**PR #588 cross-check result (no discrepancy):** Table 3 as extracted shows Procedure 2.1 (demeaning) Average Effect = -0.422 and Procedure 3.1 (detrending) Average Effect = -0.227, matching PR #588's claimed reproduction exactly. No flag needed. + +**Other gaps / uncertainties:** + +- The paper contains NO numbered theorems, propositions, or lemmas on pages 1-19; formal results are inline distributional claims (eq. 2.10 and variants) plus Procedures 2.1 and 3.1. The formal CLM underpinnings invoked for the staggered "exact theory" claims (p28) are the Section 2 assumptions; no separate staggered-case theorem is stated. +- Randomization-inference mechanics are described only briefly (p7: two-sided p = c/N with N = number of permutations; note the symbol overload with sample size N). No further RI mechanics are given in the later page range; the permutation scheme (which units are permuted, whether the full assignment vector is permuted) is not detailed in the extractions. +- Table 3, Note 2 attributes the 0.021 / 0.020 p-values to "Procedure 3.1" without stating explicitly that they refer to the Average Effect row (the natural reading, consistent with its ** star). +- Tables 3, 4, and A1 report per-period effects only for 1989, 1995, 2000; the remaining 9 post-period estimates (and all SC/SDiD per-period cells) are not available from the extractions. +- Pages 33-34 (references) were skipped per extraction instructions; the exact reference-list entries are not transcribed. +- Figures 1-6 and A1 are described qualitatively only; no underlying series values are available for pinning plot-level goldens. +- The default behavior of the Stata `lwdid` command (e.g., default SE type, default transformation) is not described beyond it implementing Procedures 2.1/3.1 and offering the `ri` option; parity testing against `lwdid` will need its documentation or source. +- The Hagemann (2025) comparison (single treated cluster, unequal variances) is mentioned only in passing (p7); how its variance-heterogeneity robustness relates to the homoskedasticity requirement (2.9) is not elaborated. +- Guidance thresholds are qualitative: HC3 requires "at least a handful of treated units" / N "not too small" - no numeric cutoff is given. From 4692e72274b40c6aa364fcbcb1f338940b30aaf7 Mon Sep 17 00:00:00 2001 From: igerber Date: Mon, 13 Jul 2026 11:53:31 -0400 Subject: [PATCH 2/3] feat(datasets): add load_prop99 + load_walmart; docs: maintainer LWDiD REGISTRY entry + references (precursor to #588) Loaders download the MIT-licensed ancillary datasets of the authors' Stata lwdid package (SSC): pinned SHA-256 verification of every byte-load (HTTP-only host), stale-cache re-download, structural validation against source invariants, loud UserWarning + df.attrs['source'] marker on synthetic fallback, seeded local-RNG fallback constructors. REGISTRY.md gains the maintainer-authored LWDiD section (E.1 contributing-unit WATT weights; provenance pinned via reviewed-PDF SHA-256 + live-verified SSRN metadata). Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01FZK3FD9jxWGxBrPDw5APSg --- .gitignore | 3 + CHANGELOG.md | 9 + TODO.md | 1 + diff_diff/__init__.py | 4 + diff_diff/datasets.py | 419 +++++++++++++++++- diff_diff/guides/llms-full.txt | 3 + diff_diff/guides/llms.txt | 2 +- .../_autosummary/diff_diff.load_prop99.rst | 7 + .../_autosummary/diff_diff.load_walmart.rst | 7 + docs/api/datasets.rst | 55 +++ docs/api/index.rst | 2 + docs/methodology/REGISTRY.md | 89 ++++ .../papers/lee-wooldridge-2025-review.md | 7 +- .../papers/lee-wooldridge-2026-review.md | 3 +- docs/references.rst | 11 + tests/test_datasets.py | 348 ++++++++++++++- 16 files changed, 959 insertions(+), 11 deletions(-) create mode 100644 docs/api/_autosummary/diff_diff.load_prop99.rst create mode 100644 docs/api/_autosummary/diff_diff.load_walmart.rst diff --git a/.gitignore b/.gitignore index e4c627d7..6a62748f 100644 --- a/.gitignore +++ b/.gitignore @@ -74,6 +74,9 @@ target/ .claude/paper-review/ .claude/scheduled_tasks.lock +# Playwright MCP browser-session artifacts +.playwright-mcp/ + # MCP configuration (may contain tokens) .mcp.json diff --git a/CHANGELOG.md b/CHANGELOG.md index 3422dd64..f5137ca9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -48,6 +48,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ignores) with `warn_unused_ignores = true` to keep them from going stale — 23 already-stale ignores were removed. Tightening tracked in TODO.md. No public API or numerical behavior change. +- **`load_prop99` + `load_walmart` dataset loaders.** California Proposition 99 + smoking panel (39 states, single treated unit, Lee-Wooldridge cohort format) and + the Walmart entry county panel (1,277 counties, staggered 1986-1999 openings, + Brown & Butts CBP construction), downloaded with local caching from the + MIT-licensed ancillary data of the authors' Stata `lwdid` package (SSC). + Every byte-load is verified against a pinned SHA-256 and the parsed panels are + validated against their source invariants; if the real data cannot be obtained, + a seeded same-schema synthetic fallback is returned with an explicit + `UserWarning` and `df.attrs["source"] = "synthetic_fallback"`. - **Internal: rdrobust sharp-RD bandwidth-selection port (`diff_diff/_rdrobust_port.py`).** Step-1 machinery for the `RegressionDiscontinuity` estimator above: a faithful pure-Python NumPy/SciPy port of R `rdrobust` 4.0.0's `rdbwselect` sharp-RD path (all 10 data-driven diff --git a/TODO.md b/TODO.md index 2db92bae..f914a0c9 100644 --- a/TODO.md +++ b/TODO.md @@ -50,6 +50,7 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m | ChangesInChanges/QDiD tutorial notebook (2x2 distributional walkthrough: QTE grid, interior range, uniform bands, CiC-vs-QDiD comparison) - deferred from the implementation PR as a documented decision. | `docs/tutorials/` | #682 | Mid | Low | | Tighten the mypy suppressions that back the enforced-zero posture: burn down `prep_dgp`'s per-module `[index]` override (needs a None-vs-array restructure that preserves the seeded RNG stream), and evaluate re-enabling the globally disabled codes (`arg-type`, `return-value`, `var-annotated`, `assignment`) one at a time — `assignment` alone hid several real annotation drifts found during the 2026-07 triage. | `pyproject.toml` `[tool.mypy]`, `diff_diff/prep_dgp.py` | lint-CI | Mid | Low | | `practitioner_next_steps()` dedicated handler for `ChangesInChangesResults` (currently falls back to `_handle_generic`, which is safe; a dedicated handler is the established full-integration step, cf. HAD Phase 5). | `diff_diff/practitioner.py` | #682 | Quick | Low | +| Align the four legacy dataset loaders (`load_card_krueger`, `load_castle_doctrine`, `load_divorce_laws`, `load_mpdta`) with the loud-fallback pattern of `load_prop99`/`load_walmart`: `UserWarning` + `df.attrs["source"]` marker on synthetic fallback (currently silent), plus optional checksum pinning for the CSV downloads. | `diff_diff/datasets.py` | LWDiD precursor | Quick | Low | --- diff --git a/diff_diff/__init__.py b/diff_diff/__init__.py index 8b9ac0f1..f2319ef4 100644 --- a/diff_diff/__init__.py +++ b/diff_diff/__init__.py @@ -72,6 +72,8 @@ load_dataset, load_divorce_laws, load_mpdta, + load_prop99, + load_walmart, ) from diff_diff.diagnostic_report import ( DIAGNOSTIC_REPORT_SCHEMA_VERSION, @@ -537,6 +539,8 @@ "load_castle_doctrine", "load_divorce_laws", "load_mpdta", + "load_prop99", + "load_walmart", "load_dataset", "list_datasets", "clear_cache", diff --git a/diff_diff/datasets.py b/diff_diff/datasets.py index 98a14546..58031217 100644 --- a/diff_diff/datasets.py +++ b/diff_diff/datasets.py @@ -8,9 +8,11 @@ for subsequent use. """ -from io import StringIO +import hashlib +import warnings +from io import BytesIO, StringIO from pathlib import Path -from typing import Dict +from typing import Dict, cast from urllib.error import HTTPError, URLError from urllib.request import urlopen @@ -53,11 +55,65 @@ def _download_with_cache( ) from e +def _get_cache_path_binary(name: str) -> Path: + """Get the cache path for a binary (Stata .dta) dataset.""" + _CACHE_DIR.mkdir(parents=True, exist_ok=True) + return _CACHE_DIR / f"{name}.dta" + + +def _download_with_cache_binary( + url: str, + name: str, + sha256: str, + force_download: bool = False, +) -> bytes: + """Download a binary file (e.g. Stata .dta), verify its checksum, and cache it. + + The source host serves plain HTTP, so every byte-load (cache or fresh + download) is verified against a pinned SHA-256. A stale/corrupt cache + triggers one re-download; a checksum mismatch on freshly downloaded + bytes raises. + """ + cache_path = _get_cache_path_binary(name) + + if cache_path.exists() and not force_download: + content = cache_path.read_bytes() + if hashlib.sha256(content).hexdigest() == sha256: + return content + # Cached copy is stale or corrupt: fall through to re-download + + try: + with urlopen(url, timeout=30) as response: + content = response.read() + except (HTTPError, URLError) as e: + if cache_path.exists(): + content = cache_path.read_bytes() + if hashlib.sha256(content).hexdigest() == sha256: + # Use cached version if download fails + return content + raise RuntimeError( + f"Failed to download dataset '{name}' from {url}: {e}\n" + "Check your internet connection or try again later." + ) from e + + if hashlib.sha256(content).hexdigest() != sha256: + raise RuntimeError( + f"Checksum mismatch for dataset '{name}' downloaded from {url}.\n" + "The upstream file differs from the pinned SHA-256. If the lwdid " + "Stata package published a new data revision, verify the new file " + "and update the pinned checksum; otherwise treat the download as " + "untrusted." + ) + cache_path.write_bytes(content) + return content + + def clear_cache() -> None: """Clear the local dataset cache.""" if _CACHE_DIR.exists(): - for f in _CACHE_DIR.glob("*.csv"): - f.unlink() + for pattern in ("*.csv", "*.dta"): + for f in _CACHE_DIR.glob(pattern): + f.unlink() print(f"Cleared cache at {_CACHE_DIR}") @@ -750,6 +806,357 @@ def _construct_mpdta_data() -> pd.DataFrame: return df +def load_prop99(force_download: bool = False) -> pd.DataFrame: + """ + Load the California Proposition 99 smoking dataset (Lee-Wooldridge format). + + This dataset tracks per capita cigarette sales across 39 U.S. states + (California plus 38 never-treated donor states) from 1970 to 2000. + California passed Proposition 99, a large tobacco tax and control + program, effective in 1989. With a single treated unit, it is the + canonical setting for small-sample DiD inference and synthetic + control comparisons. + + Parameters + ---------- + force_download : bool, default=False + If True, re-download the dataset even if cached. + + Returns + ------- + pd.DataFrame + Panel dataset with columns: + - state : str - State name + - year : int - Year (1970-2000) + - first_year : int - Year treatment began (1989 for California, + 0 = never treated) + - lcigsale : float - Log per capita cigarette sales (packs) + - treated : int - 1 if treatment in effect, 0 otherwise + - cohort : int - Alias for first_year + + Notes + ----- + This is the cohort-format version of the Abadie, Diamond & + Hainmueller (2010) California tobacco data distributed (MIT license) + with the authors' Stata ``lwdid`` package by Hur, Lee and Wooldridge. + The donor pool excludes states with their own tobacco programs, + leaving exactly one treated state and 38 controls. + + Downloads are verified against a pinned SHA-256 and validated against + the source invariants (39 states, 1970-2000, single 1989 cohort). If + the real data cannot be obtained, a SYNTHETIC same-schema fallback is + returned with a ``UserWarning``; check ``df.attrs["source"]`` + (``"lwdid_ssc_ancillary"`` = real data, ``"synthetic_fallback"`` = + synthetic - never use the fallback for replication). + + References + ---------- + Lee, S. J., & Wooldridge, J. M. (2026). Simple Approaches to Inference + with Difference-in-Differences Estimators with Small Cross-Sectional + Sample Sizes. SSRN Working Paper No. 5325686. + + Abadie, A., Diamond, A., & Hainmueller, J. (2010). Synthetic Control + Methods for Comparative Case Studies: Estimating the Effect of + California's Tobacco Control Program. *Journal of the American + Statistical Association*, 105(490), 493-505. + + Examples + -------- + >>> from diff_diff.datasets import load_prop99 + >>> from diff_diff import DifferenceInDifferences + >>> + >>> prop99 = load_prop99() + >>> prop99["treated_state"] = (prop99["first_year"] > 0).astype(int) + >>> prop99["post"] = (prop99["year"] >= 1989).astype(int) + >>> + >>> did = DifferenceInDifferences() + >>> results = did.fit( + ... prop99, outcome="lcigsale", treatment="treated_state", time="post" + ... ) + """ + url = "http://fmwww.bc.edu/repec/bocode/l/lw_smoking.dta" + sha256 = "16c3ac1da351788817433fc890ec2f502a8bdfcb46cbc8d693653330e71d5a65" + + source = "lwdid_ssc_ancillary" + try: + content = _download_with_cache_binary(url, "prop99", sha256, force_download) + df = cast(pd.DataFrame, pd.read_stata(BytesIO(content))) + except RuntimeError as e: + # Fallback: construct synthetic data from documented patterns - NOT the + # real Prop 99 data; unsuitable for replication. + warnings.warn( + f"Could not obtain the real Prop 99 dataset ({e}). Returning a " + "SYNTHETIC fallback panel with the same schema. Do not use it for " + "replication; check `df.attrs['source']`.", + UserWarning, + stacklevel=2, + ) + source = "synthetic_fallback" + df = _construct_prop99_data() + + # Normalize dtypes (the .dta stores first_year as float32, 0 = never treated) + df["state"] = df["state"].astype(str) + df["year"] = df["year"].astype(int) + df["first_year"] = df["first_year"].astype(int) + df["lcigsale"] = df["lcigsale"].astype(float) + + if source == "lwdid_ssc_ancillary": + _validate_prop99(df) + + # Add convenience columns + if "cohort" not in df.columns: + df["cohort"] = df["first_year"] + + if "treated" not in df.columns: + df["treated"] = ((df["first_year"] > 0) & (df["year"] >= df["first_year"])).astype(int) + + df.attrs["source"] = source + return df + + +def _validate_prop99(df: pd.DataFrame) -> None: + """Validate the downloaded Prop 99 data against its source invariants.""" + problems = [] + if df.shape != (1209, 4): + problems.append(f"shape {df.shape} != (1209, 4)") + if df["state"].nunique() != 39: + problems.append(f"{df['state'].nunique()} states != 39") + if (df["year"].min(), df["year"].max()) != (1970, 2000): + problems.append("year range != 1970-2000") + if df.duplicated(["state", "year"]).any(): + problems.append("duplicate (state, year) rows") + if not (df.groupby("state")["first_year"].nunique() == 1).all(): + problems.append("first_year not constant within state") + if set(df.loc[df["first_year"] > 0, "first_year"].unique()) != {1989}: + problems.append("treated cohort != {1989}") + if df.loc[df["first_year"] > 0, "state"].nunique() != 1: + problems.append("treated state count != 1") + if df.loc[df["first_year"] == 0, "state"].nunique() != 38: + problems.append("never-treated state count != 38") + if problems: + raise RuntimeError( + "Downloaded Prop 99 data failed source validation: " + + "; ".join(problems) + + ". The upstream file may have changed - please report this." + ) + + +def _construct_prop99_data() -> pd.DataFrame: + """ + Construct a synthetic Prop 99-style dataset from documented patterns. + + This is a fallback when the online source is unavailable. + """ + rng = np.random.default_rng(2010) # Abadie-Diamond-Hainmueller publication year + + states = ["California"] + [f"State{i:02d}" for i in range(2, 40)] + + data = [] + for state in states: + first_year = 1989 if state == "California" else 0 + base = rng.uniform(4.3, 4.9) # log packs per capita + trend = rng.uniform(-0.020, -0.010) # secular decline + + for year in range(1970, 2001): + lcigsale = base + trend * (year - 1970) + rng.normal(0, 0.04) + # Treatment effect: gradual decline after 1989 (~ -0.4 by 2000) + if first_year > 0 and year >= first_year: + lcigsale -= 0.04 * min(year - first_year + 1, 10) + + data.append( + { + "state": state, + "year": year, + "first_year": first_year, + "lcigsale": round(lcigsale, 6), + } + ) + + return pd.DataFrame(data) + + +def load_walmart(force_download: bool = False) -> pd.DataFrame: + """ + Load the Walmart entry county panel (Lee-Wooldridge sample). + + This dataset tracks log retail and wholesale employment for 1,277 + U.S. counties from 1977 to 1999, with staggered first Walmart store + openings between 1986 and 1999 and 391 counties never receiving a + store. It is used to study the local labor-market effects of Walmart + entry under staggered treatment adoption. + + Parameters + ---------- + force_download : bool, default=False + If True, re-download the dataset even if cached. + + Returns + ------- + pd.DataFrame + Panel dataset with columns: + - cid : int - County identifier + - year : int - Year (1977-1999) + - first_year : int - Year of first Walmart opening (0 = never) + - log_retail_emp : float - Log county retail employment (outcome) + - log_wholesale_emp : float - Log county wholesale employment + - x1 : float - County poverty rate + - x2 : float - Share with high-school education + - x3 : float - Manufacturing employment share + - treated : int - 1 if a Walmart has opened, 0 otherwise + - cohort : int - Alias for first_year + + Notes + ----- + The panel derives from County Business Patterns data as constructed + by Brown & Butts, and is distributed (MIT license) with the authors' + Stata ``lwdid`` package by Hur, Lee and Wooldridge. The covariate + labels follow the Lee & Wooldridge application. + + Downloads are verified against a pinned SHA-256 and validated against + the source invariants (1,277 counties, 1977-1999, cohorts 1986-1999, + 391 never-treated). If the real data cannot be obtained, a SYNTHETIC + same-schema fallback (200 counties) is returned with a + ``UserWarning``; check ``df.attrs["source"]`` + (``"lwdid_ssc_ancillary"`` = real data, ``"synthetic_fallback"`` = + synthetic - never use the fallback for replication). + + References + ---------- + Lee, S. J., & Wooldridge, J. M. (2025). A Simple Transformation + Approach to Difference-in-Differences Estimation for Panel Data. + SSRN Working Paper No. 4516518. + + Brown, N., & Butts, K. (2025). Dynamic Treatment Effect Estimation + with Interactive Fixed Effects and Short Panels. *Journal of + Econometrics*. + + Examples + -------- + >>> from diff_diff.datasets import load_walmart + >>> from diff_diff import CallawaySantAnna + >>> + >>> walmart = load_walmart() + >>> cs = CallawaySantAnna(control_group="never_treated") + >>> results = cs.fit( + ... walmart, + ... outcome="log_retail_emp", + ... unit="cid", + ... time="year", + ... first_treat="first_year", + ... ) + """ + url = "http://fmwww.bc.edu/repec/bocode/l/lw_walmart.dta" + sha256 = "410885572143dceb9daa643a8097768f1bc3493f9437451a9e4d1d5dc1e18d14" + + source = "lwdid_ssc_ancillary" + try: + content = _download_with_cache_binary(url, "walmart", sha256, force_download) + df = cast(pd.DataFrame, pd.read_stata(BytesIO(content))) + except RuntimeError as e: + # Fallback: construct synthetic data from documented patterns - NOT the + # real Walmart panel (and much smaller: 200 counties vs 1,277); + # unsuitable for replication. + warnings.warn( + f"Could not obtain the real Walmart dataset ({e}). Returning a " + "SYNTHETIC fallback panel (200 counties, not the real 1,277) with " + "the same schema. Do not use it for replication; check " + "`df.attrs['source']`.", + UserWarning, + stacklevel=2, + ) + source = "synthetic_fallback" + df = _construct_walmart_data() + + # Normalize dtypes (the .dta stores identifiers as float32, 0 = never treated) + df["cid"] = df["cid"].astype(int) + df["year"] = df["year"].astype(int) + df["first_year"] = df["first_year"].astype(int) + for col in ("log_retail_emp", "log_wholesale_emp", "x1", "x2", "x3"): + df[col] = df[col].astype(float) + + if source == "lwdid_ssc_ancillary": + _validate_walmart(df) + + # Add convenience columns + if "cohort" not in df.columns: + df["cohort"] = df["first_year"] + + if "treated" not in df.columns: + df["treated"] = ((df["first_year"] > 0) & (df["year"] >= df["first_year"])).astype(int) + + df.attrs["source"] = source + return df + + +def _validate_walmart(df: pd.DataFrame) -> None: + """Validate the downloaded Walmart data against its source invariants.""" + problems = [] + if df.shape != (29371, 8): + problems.append(f"shape {df.shape} != (29371, 8)") + if df["cid"].nunique() != 1277: + problems.append(f"{df['cid'].nunique()} counties != 1277") + if (df["year"].min(), df["year"].max()) != (1977, 1999): + problems.append("year range != 1977-1999") + if df.duplicated(["cid", "year"]).any(): + problems.append("duplicate (cid, year) rows") + if not (df.groupby("cid")["first_year"].nunique() == 1).all(): + problems.append("first_year not constant within county") + cohorts = set(df.loc[df["first_year"] > 0, "first_year"].unique()) + if cohorts != set(range(1986, 2000)): + problems.append("treated cohorts != {1986, ..., 1999}") + if df.loc[df["first_year"] == 0, "cid"].nunique() != 391: + problems.append("never-treated county count != 391") + if problems: + raise RuntimeError( + "Downloaded Walmart data failed source validation: " + + "; ".join(problems) + + ". The upstream file may have changed - please report this." + ) + + +def _construct_walmart_data() -> pd.DataFrame: + """ + Construct a synthetic Walmart-entry-style county panel. + + This is a fallback when the online source is unavailable. + """ + rng = np.random.default_rng(2025) # Brown-Butts publication year, for reproducibility + + n_counties = 200 + years = range(1977, 2000) + # Roughly 30% never treated; the rest staggered over 1986-1999 + cohorts = [0] + list(range(1986, 2000)) + cohort_probs = [0.30] + [0.05] * 14 + + data = [] + for cid in range(1, n_counties + 1): + first_year = int(rng.choice(cohorts, p=cohort_probs)) + base_retail = rng.normal(7.5, 0.8) + base_wholesale = base_retail - rng.uniform(0.8, 1.5) + x1 = rng.uniform(0.05, 0.30) # poverty rate + x2 = rng.uniform(0.50, 0.85) # HS education share + x3 = rng.uniform(0.05, 0.40) # manufacturing share + + for year in years: + trend = (year - 1977) * 0.01 + te = 0.03 if (first_year > 0 and year >= first_year) else 0.0 + + data.append( + { + "cid": cid, + "year": year, + "first_year": first_year, + "log_retail_emp": round(base_retail + trend + te + rng.normal(0, 0.05), 6), + "log_wholesale_emp": round(base_wholesale + trend + rng.normal(0, 0.05), 6), + "x1": round(x1, 6), + "x2": round(x2, 6), + "x3": round(x3, 6), + } + ) + + return pd.DataFrame(data) + + def list_datasets() -> Dict[str, str]: """ List available real-world datasets. @@ -770,6 +1177,8 @@ def list_datasets() -> Dict[str, str]: "castle_doctrine": "Castle Doctrine laws - staggered adoption across states", "divorce_laws": "Unilateral divorce laws - staggered adoption (Stevenson-Wolfers)", "mpdta": "Minimum wage panel data - simulated CS example from R `did` package", + "prop99": "California Prop 99 smoking panel - single treated unit (Lee-Wooldridge format)", + "walmart": "Walmart entry county panel - staggered adoption (Lee-Wooldridge sample)", } @@ -805,6 +1214,8 @@ def load_dataset(name: str, force_download: bool = False) -> pd.DataFrame: "castle_doctrine": load_castle_doctrine, "divorce_laws": load_divorce_laws, "mpdta": load_mpdta, + "prop99": load_prop99, + "walmart": load_walmart, } if name not in loaders: diff --git a/diff_diff/guides/llms-full.txt b/diff_diff/guides/llms-full.txt index 235ee2ff..37ac917b 100644 --- a/diff_diff/guides/llms-full.txt +++ b/diff_diff/guides/llms-full.txt @@ -2215,6 +2215,7 @@ data = generate_synthetic_control_data(n_donors=20, n_pre=60, n_post=5, ```python from diff_diff import load_card_krueger, load_castle_doctrine, load_divorce_laws, load_mpdta +from diff_diff import load_prop99, load_walmart from diff_diff import load_dataset, list_datasets, clear_cache # List available datasets @@ -2229,6 +2230,8 @@ ck = load_card_krueger() # Card & Krueger (1994) minimum wage castle = load_castle_doctrine() # Castle Doctrine / Stand Your Ground laws divorce = load_divorce_laws() # Unilateral divorce laws (staggered) mpdta = load_mpdta() # Minimum wage panel (simulated, from R did package) +prop99 = load_prop99() # California Prop 99 smoking (single treated unit) +walmart = load_walmart() # Walmart entry county panel (staggered, 1,277 counties) # Force re-download data = load_card_krueger(force_download=True) diff --git a/diff_diff/guides/llms.txt b/diff_diff/guides/llms.txt index c34a2b52..2badbb9f 100644 --- a/diff_diff/guides/llms.txt +++ b/diff_diff/guides/llms.txt @@ -122,6 +122,6 @@ No R or Python package offers design-based variance estimation for modern hetero ## Optional - [Rust Backend](https://diff-diff.readthedocs.io/en/stable/benchmarks.html): Rust backend (bundled in released wheels; `maturin develop --release` for source builds) for order-of-magnitude speedups on compute-intensive estimators - 18-55x vs R synthdid on Synthetic DiD in the published benchmarks - plus TROP and bootstrap acceleration -- [Built-in Datasets](https://diff-diff.readthedocs.io/en/stable/api/datasets.html): Real-world datasets — Card & Krueger (1994), Castle Doctrine, divorce laws, MPDTA +- [Built-in Datasets](https://diff-diff.readthedocs.io/en/stable/api/datasets.html): Real-world datasets — Card & Krueger (1994), Castle Doctrine, divorce laws, MPDTA, California Prop 99 smoking (single treated unit), Walmart entry county panel (staggered) - [Visualization](https://diff-diff.readthedocs.io/en/stable/api/visualization.html): Event study plots, group effects, sensitivity plots, Bacon decomposition plots, power curves - [Data Preparation](https://diff-diff.readthedocs.io/en/stable/api/prep.html): Data generation, panel balancing, wide-to-long conversion, treatment/post indicator creation diff --git a/docs/api/_autosummary/diff_diff.load_prop99.rst b/docs/api/_autosummary/diff_diff.load_prop99.rst new file mode 100644 index 00000000..1a1589e7 --- /dev/null +++ b/docs/api/_autosummary/diff_diff.load_prop99.rst @@ -0,0 +1,7 @@ +diff\_diff.load\_prop99 +======================= + +.. currentmodule:: diff_diff + +.. autofunction:: load_prop99 + :no-index: \ No newline at end of file diff --git a/docs/api/_autosummary/diff_diff.load_walmart.rst b/docs/api/_autosummary/diff_diff.load_walmart.rst new file mode 100644 index 00000000..bc431bf2 --- /dev/null +++ b/docs/api/_autosummary/diff_diff.load_walmart.rst @@ -0,0 +1,7 @@ +diff\_diff.load\_walmart +======================== + +.. currentmodule:: diff_diff + +.. autofunction:: load_walmart + :no-index: \ No newline at end of file diff --git a/docs/api/datasets.rst b/docs/api/datasets.rst index 39a937e0..ccd69448 100644 --- a/docs/api/datasets.rst +++ b/docs/api/datasets.rst @@ -122,6 +122,61 @@ Example first_treat="first_treat" ) +load_prop99 +~~~~~~~~~~~ + +California Proposition 99 tobacco program study (Lee--Wooldridge cohort format +of the Abadie--Diamond--Hainmueller 2010 data). Log per capita cigarette sales +for 39 states (1970--2000) with a single treated unit (California, treated from +1989) -- the canonical small-sample DiD and synthetic control setting. + +.. autofunction:: diff_diff.load_prop99 + +Example +^^^^^^^ + +.. code-block:: python + + from diff_diff.datasets import load_prop99 + from diff_diff import DifferenceInDifferences + + prop99 = load_prop99() + prop99["treated_state"] = (prop99["first_year"] > 0).astype(int) + prop99["post"] = (prop99["year"] >= 1989).astype(int) + + did = DifferenceInDifferences() + results = did.fit( + prop99, outcome="lcigsale", treatment="treated_state", time="post" + ) + +load_walmart +~~~~~~~~~~~~ + +Walmart entry county panel (Lee & Wooldridge 2025 sample, derived from County +Business Patterns data as constructed by Brown & Butts). Log retail and +wholesale employment for 1,277 counties (1977--1999) with staggered first +store openings (1986--1999) and 391 never-treated counties. + +.. autofunction:: diff_diff.load_walmart + +Example +^^^^^^^ + +.. code-block:: python + + from diff_diff.datasets import load_walmart + from diff_diff import CallawaySantAnna + + walmart = load_walmart() + cs = CallawaySantAnna(control_group="never_treated") + results = cs.fit( + walmart, + outcome="log_retail_emp", + unit="cid", + time="year", + first_treat="first_year" + ) + Utility Functions ----------------- diff --git a/docs/api/index.rst b/docs/api/index.rst index 7d00504e..789cfc57 100644 --- a/docs/api/index.rst +++ b/docs/api/index.rst @@ -296,6 +296,8 @@ Built-in datasets for examples and testing: diff_diff.load_castle_doctrine diff_diff.load_divorce_laws diff_diff.load_mpdta + diff_diff.load_prop99 + diff_diff.load_walmart diff_diff.load_dataset diff_diff.list_datasets diff_diff.clear_cache diff --git a/docs/methodology/REGISTRY.md b/docs/methodology/REGISTRY.md index a56da0a4..60b89848 100644 --- a/docs/methodology/REGISTRY.md +++ b/docs/methodology/REGISTRY.md @@ -20,6 +20,7 @@ This document provides the academic foundations and key implementation requireme - [StackedDiD](#stackeddid) - [WooldridgeDiD (ETWFE)](#wooldridgedid-etwfe) - [LPDiD](#lpdid) + - [LWDiD](#lwdid) 3. [Advanced Estimators](#advanced-estimators) - [SyntheticDiD](#syntheticdid) - [SyntheticControl](#syntheticcontrol) @@ -2132,6 +2133,94 @@ The paper specifies no standard-error formula (Section 1 defers to "standard, we --- +## LWDiD + +**Primary sources:** +- [Lee, S.J. & Wooldridge, J.M. (2025). A Simple Transformation Approach to Difference-in-Differences Estimation for Panel Data. SSRN Working Paper No. 4516518 (61-page revision, cover page June 8, 2026; SSRN "last revised" 8 Jun 2026).](https://ssrn.com/abstract=4516518) — estimation core (rolling transformations, RA/IPW/IPWRA, multiplier bootstrap). DOI: 10.2139/ssrn.4516518. +- [Lee, S.J. & Wooldridge, J.M. (2026). Simple Approaches to Inference with Difference-in-Differences Estimators with Small Cross-Sectional Sample Sizes. SSRN Working Paper No. 5325686 (36 pages, cover page February 3, 2026; SSRN "last revised" 13 Jun 2026).](https://ssrn.com/abstract=5325686) — exact small-sample inference layer. DOI: 10.2139/ssrn.5325686. + +Exact reviewed artifacts (PDF SHA-256) and live-verified SSRN metadata are pinned in the paper-review headers below; stale SSRN caches/mirrors may still show the superseded December 2025 / January 2026 revisions. + +Full maintainer paper reviews (equation-level detail, replication targets): `docs/methodology/papers/lee-wooldridge-2025-review.md`, `docs/methodology/papers/lee-wooldridge-2026-review.md`. + +- **Note:** Registry entry authored with the paper reviews ahead of the implementation (PR #588, third-party contribution under maintainer revision). Checklist boxes are unchecked until the implementation lands; implementation-specific edge-case notes will be finalized in that PR. + +**Key implementation requirements:** + +*Assumption checks / warnings:* +- Treatment is absorbing (no reversibility), common timing (`1 < S <= T`) or staggered (cohorts `g in {S,...,T,infinity}`, mutually exclusive and exhaustive); at least one pre-treatment period. +- Common timing: **NAC** (no anticipation, eq. (2.7)), **CPTC** (conditional parallel trends, eq. (2.10)), **OVLC** (overlap, eqs. (2.14)-(2.15)); Theorem 2.1 identifies `tau_r`, `r = S,...,T`. +- Staggered: **CNAS** (eq. (4.4); X-conditioning droppable with NT-only controls), **CPTS** (eq. (4.6)), **OVLS** (eq. (4.10), control pool `A_{r+1} = D_{r+1} + ... + D_T + D_infinity`); Theorem 4.1. +- Heterogeneous linear trends: **CHT** (eq. (5.3)) — demeaning inconsistent under CHT; unit-specific detrending (Procedure 5.1) restores consistency under CNAS + CHT + OVLS. +- Minimum pre-periods: >= 1 per cohort for demeaning, >= 2 for detrending (rank condition, Appendix B); failing cells not estimable. +- Small-sample (exact) inference layer: classical linear model assumptions — conditional normality AND homoskedasticity of the collapsed cross-sectional error (2026 paper eqs. (2.7)-(2.9); with controls (2.18)-(2.19)). Sample-size guards: `N0 >= 1`, `N1 >= 1`, `N >= 3`; `N > K + 2` with K controls; interacted controls need `N0 > K + 1` AND `N1 > K + 1`; NT-only controls in the staggered case need `N_infinity >= 2`. +- IPWRA is doubly robust: consistent if either the propensity-score model or the outcome model is correct. + +*Transformations (LW 2025 eqs. (3.2)/(4.11), (5.6); as specified for implementation):* + + Demeaning: Y_dot_irg = Y_ir - (1/(g-1)) * sum_{s=1}^{g-1} Y_is + Detrending: Yddot_irg = Y_ir - Yhat_irg, Yhat from unit OLS of Y_it on (1, t), t < g (out-of-sample residuals) + +Event-study/placebo transformations over ALL periods (Appendix D): demeaning (D.1) excludes `t = g-1`; detrending (D.2) excludes `t in {g-2, g-1}` (anchor periods, event times `r = -1` / `r = -2, -1`). + +*Estimation (per (g, t) cell; LW 2025 eq. (E.1)):* +- Cross-sectional RA regression on the cell sample `A_{g,t}`: `Y_dot on 1, D_g, X, D_g (X - Xbar_g)`; ATT(g,t) = coefficient on `D_g`. No-covariate case reduces to plain DiD (eq. (3.4)); Theorem 3.1: common-timing per-period regressions are numerically equivalent to pooled OLS (3.6) (r = g reproduces the ETWFE estimand; `r > g` does not). +- IPWRA (workhorse): logit propensity score per cell + WLS with weights `w = D + (1-D) p/(1-p)`; IPW = special case without the outcome-regression component. +- Control pool at (g, r): `A_{r+1} = 1` (never-treated + not-yet-treated) by default; NT-only optional. Pre-treatment placebo cells use the Appendix D.3 rule `A_{g,t} = {G = g} ∪ {G = 0} ∪ {G > max(g,t)}`. + +*Aggregation:* +- Event-study: `WATT(r) = sum_{g in G_r} omega_{g,r} ATT(g, g+r)` with `omega_{g,r}` = (treated units of cohort g contributing at event time r) / (total treated units contributing at event time r) - the operative definition per LW 2025 Appendix E.1, required under unbalanced panels where a cohort's contributing count at r can differ from `N_g`. In balanced panels this simplifies to `N_g / N_{G_r}` (Sec. 6.2/D.3). Aggregated influence function `IF_{i,r} = sum_g omega_{g,r} IF_{i,g,g+r}`. +- Overall: composite-outcome single regression (LW 2026 eqs. (7.18)-(7.19)) — `tau_omega` with cohort-share weights `omega_g = N_g / N_treat`; automatically accounts for correlation among per-cohort effects and supports exact small-N inference. + +*Standard errors:* +- Large-N default: influence-function **multiplier bootstrap** (LW 2025 Algorithm 1): IFs per (g,t) from E.2 (RA, finite-sample exact), E.3 (IPWRA, stacked M-estimator with logit-score correction), E.4 (IPW, `psi - Gamma' IF_gamma` correction); centered IFs; **unit-level Rademacher multipliers** (one draw per unit across all cells — unit clustering by construction); sup-t simultaneous bands over the event-study path; B = 999 in the paper's application; anchor periods excluded. +- Small-N exact (LW 2026): usual OLS SE on the collapsed cross-sectional regression with exact `T_{N-2}` / `T_{N-K-2}` reference distribution; valid down to `N = 3` and a single treated unit (`N1 = 1` — the t statistic is the studentized residual; same for `N_g = 1` per cohort in (7.8)/(7.10)). +- Alternatives: HC3 when there are "at least a handful" of treated units; randomization inference for the sharp null (two-sided p = c / #permutations; Stata `lwdid` `ri` option); higher-level clustering and Conley SHAC SEs for larger cross sections (LW 2026 Sec. 8.2, citing Abadie-Athey-Imbens-Wooldridge 2023). + +*Edge cases:* +- Anchor periods: event-study omits `r = -1` (demeaning) / `r = -2, -1` (detrending); bootstrap excludes them. +- All units eventually treated (LW 2025 Sec. 4.3): drop `D_infinity`; effects defined relative to the last cohort; no effect estimable for the last cohort. +- Unbalanced panels (Sec. 4.4): transform observed data; per-cell observability requirements (1 pre-period demeaning / 2 detrending + outcome observed at r); selection may correlate with unit heterogeneity (and trends, under detrending) but not with shocks. +- Anticipation: drop periods just prior to the intervention from the pre-average/trend (LW 2025 Sec. 4.4; LW 2026 eq. (2.22) anchor `Ybar_{i,S0}`, `S0 < S-1`). +- Periods with no newly-treated units: no effects estimated there. +- Seasonality: seasonal dummies in the transformation step (LW 2026, p12). +- Complex nonlinear pre-trends: linear detrending may be inadequate — authors' caveat that the method "will not always work better than existing alternatives"; higher-order trend terms possible at a power cost. +- Per-period effect combinations: SEs for linear combinations of per-period estimates are NOT easily obtained (serial dependence); the composite regression (7.19) is the paper's device for valid aggregate SEs. +- CS normalization: rolling event studies are deviations from own pre-treatment averages, NOT normalized to `r = -1` as in Callaway-Sant'Anna — event-study points are not numerically comparable across the two. +- No incidental-parameters problem from unit-specific trend regressions with small T. + +*Procedures (verbatim transcriptions in the paper reviews):* +- LW 2026 Procedure 2.1 (unit demeaning, common timing) / Procedure 3.1 (unit detrending) — collapse to `{(Ybar_dot_i, D_i)}` and run the exact-t cross-sectional regression. +- LW 2025 Procedure 3.1 (rolling, common timing) / Procedure 4.1 (rolling, staggered; control pool `A_{r+1}`) / Procedure 5.1 (staggered detrending). +- LW 2025 Algorithm 1 (multiplier bootstrap, 7 steps): compute per-cell IFs -> aggregate WATT(r) -> center IFs -> B unit-level Rademacher draws -> bootstrap SEs -> sup statistics -> sup-t bands. + +**Reference implementation(s):** +- Stata: user-written `lwdid` (Hur, Lee and Wooldridge 2026; SSC) — implements the full procedure, multiplier-bootstrap inference, and randomization inference (`ri` option); ancillary datasets `lw_smoking.dta` / `lw_walmart.dta` (MIT) are the sources for `load_prop99()` / `load_walmart()`. +- R: none. + +**Replication targets (from the papers; datasets available via `diff_diff.datasets`):** +- Prop 99 (LW 2026 Table 3, 38-state donor pool): demeaning ATT = -0.422 (SE 0.121); detrending ATT = -0.227 (SE 0.094), exact p = 0.021 vs randomization-inference p = 0.020. +- Castle laws (LW 2026 Sec. 7): `tau_omega` = 0.092 (demeaning; OLS SE 0.057), 0.067 (detrending). +- Walmart entry (LW 2025 Tables A4/A5, 1,277 counties): per-relative-period WATT(r) with SEs for r = 0..13. + +**Requirements checklist:** +- [ ] Rolling demeaning (3.2)/(4.11) using ALL pre-g periods; detrending (5.6)/(D.2) via unit OLS with out-of-sample residuals +- [ ] Minimum pre-period enforcement (>= 1 demeaning / >= 2 detrending); failing cells dropped with warning +- [ ] Control pools: NT + NYT (`A_{r+1} = 1`) default, NT-only option (`N_infinity >= 2` guard); placebo cells per D.3 rule `G > max(g,t)` +- [ ] RA (E.1) with treated-cohort-centered interactions; IPWRA (logit + WLS); IPW special case +- [ ] Influence functions per E.2/E.3/E.4 including first-stage logit-score corrections; IFs centered +- [ ] WATT(r) event-study aggregation with contributing-treated-unit weights (E.1 definition; cohort-size weights `N_g / N_{G_r}` only as the balanced-panel simplification); anchor periods excluded (r = -1 / r = -2,-1) +- [ ] Algorithm 1 multiplier bootstrap: unit-level Rademacher, sup-t simultaneous bands +- [ ] Composite-outcome overall aggregation (7.18)/(7.19) with cohort-share weights +- [ ] Exact-t inference: `T_{N-2}` / `T_{N-K-2}`, valid to `N = 3`, `N1 = 1`, `N_g = 1`; sample-size guards enforced +- [ ] HC3 alternative; randomization inference (two-sided p = c / #permutations); higher-level clustering / SHAC for larger N +- [ ] Anticipation-robustness period dropping; seasonal dummies in the transformation step +- [ ] All-eventually-treated (Sec. 4.3) and unbalanced-panel (Sec. 4.4) support +- [ ] Common-timing no-covariate case reproduces plain DiD (3.4); Theorem 3.1 pooled-OLS equivalence (cross-estimator test vs `DifferenceInDifferences` / ETWFE at r = g) +- [ ] Prop 99 / castle-laws / Walmart replication targets pinned as tests + +--- + # Advanced Estimators ## SyntheticDiD diff --git a/docs/methodology/papers/lee-wooldridge-2025-review.md b/docs/methodology/papers/lee-wooldridge-2025-review.md index 8af69965..80fdcf3e 100644 --- a/docs/methodology/papers/lee-wooldridge-2025-review.md +++ b/docs/methodology/papers/lee-wooldridge-2025-review.md @@ -2,7 +2,8 @@ **Authors:** Soo Jeong Lee (Southern Illinois University Carbondale), Jeffrey M. Wooldridge (Michigan State University) **Citation:** Lee, S.J., & Wooldridge, J.M. (2025). A Simple Transformation Approach to Difference-in-Differences Estimation for Panel Data. SSRN Working Paper No. 4516518. First posted 27 Jul 2023; version reviewed dated April 26, 2026, last revised June 8, 2026. https://ssrn.com/abstract=4516518 -**PDF reviewed:** /private/tmp/claude-501/-Users-igerber-diff-diff-LWDiD/fa768cf3-1977-474a-9ddf-b98c87817229/scratchpad/ssrn-4516518.pdf +**PDF reviewed:** SSRN download of abstract 4516518 (61 pages; cover page dated June 8, 2026; SHA-256 `78460841def3f15fdac6a2c6b04bc0c80ecc192493b9aa441e465a81f6846ea0`). https://ssrn.com/abstract=4516518 | DOI: https://doi.org/10.2139/ssrn.4516518 +**SSRN metadata (verified live 2026-07-13):** 61 pages; Posted: 27 Jul 2023; Last revised: 8 Jun 2026; Date Written: April 26, 2026. Note: stale caches/mirrors of the SSRN page may still show the superseded December 25, 2025 revision (52 pages). **Review date:** 2026-07-11 --- @@ -87,7 +88,7 @@ ATT(g,t) = coefficient on `D_{i,g}` (`theta_hat_{g,t} = e_2' eta_hat_{g,t}`). IP WATT(r) = sum_{g in G_r} w(g,r) * ATT(g, g+r), w(g,r) = N_g / N_{G_r}, N_{G_r} = sum_{g in G_r} N_g -cohort-size-weighted average over cohorts `G_r` for which `ATT(g, g+r)` is identified at event time `r = t - g` (Appendix E.1 phrases the weight as the number of treated units in cohort g contributing at event time r over the total treated units contributing at that event time). Estimable event times: `r = -1` excluded under demeaning; `r = -2, -1` excluded under detrending. Aggregated IF: `IF_{i,r} = sum_g omega_{g,r} IF_{i,g,g+r}`. +contributing-treated-unit weighted average over cohorts `G_r` for which `ATT(g, g+r)` is identified at event time `r = t - g`: the operative weight (Appendix E.1) is the number of treated units in cohort g contributing at event time r over the total treated units contributing at that event time, which simplifies to the cohort-size form `N_g / N_{G_r}` above in balanced panels. Estimable event times: `r = -1` excluded under demeaning; `r = -2, -1` excluded under detrending. Aggregated IF: `IF_{i,r} = sum_g omega_{g,r} IF_{i,g,g+r}`. *Edge cases:* - Cohort with zero pre-treatment periods (demeaning) or fewer than two (detrending): transformation undefined -> cell not estimable; detrending rank condition `(J'_{g-1} J_{g-1})` invertible iff >= 2 distinct pre-treatment time points (Appendix B, Equations (B.1)-(B.3)). @@ -138,7 +139,7 @@ Algorithm 1 (Multiplier Bootstrap for Simultaneous Inference on WATT_hat(r); App - [ ] RA estimator (E.1) with treated-cohort-centered covariate interactions `D_{i,g}(X_i - Xbar_g)` - [ ] IPWRA: logit PS by ML per cell + WLS of (E.1) with weights `w_{i,g,t}`; IPW special case - [ ] Influence functions per E.2 (RA, exact), E.3 (IPWRA), E.4 (IPW) including first-stage logit-score corrections -- [ ] WATT(r) cohort-size-weighted aggregation over identified (g, g+r) cells +- [ ] WATT(r) contributing-treated-unit weighted aggregation over identified (g, g+r) cells (cohort-size weights in balanced panels) - [ ] Multiplier bootstrap (Algorithm 1): unit-level Rademacher draws, centered IFs, sup-t simultaneous bands; anchor periods excluded - [ ] Anchor-period exclusion: r = -1 (demeaning); r = -2, -1 (detrending) - [ ] Section 4.3 support: all-eventually-treated panels (drop D_infinity; last cohort as control at r = T; no effect for last cohort) diff --git a/docs/methodology/papers/lee-wooldridge-2026-review.md b/docs/methodology/papers/lee-wooldridge-2026-review.md index 4a54cbdf..2635ef27 100644 --- a/docs/methodology/papers/lee-wooldridge-2026-review.md +++ b/docs/methodology/papers/lee-wooldridge-2026-review.md @@ -2,7 +2,8 @@ **Authors:** Soo Jeong Lee (Southern Illinois University Carbondale), Jeffrey M. Wooldridge (Michigan State University) **Citation:** Lee, S.J., & Wooldridge, J.M. (2026). Simple Approaches to Inference with Difference-in-Differences Estimators with Small Cross-Sectional Sample Sizes. SSRN Working Paper No. 5325686, dated February 3, 2026. https://ssrn.com/abstract=5325686 -**PDF reviewed:** /private/tmp/claude-501/-Users-igerber-diff-diff-LWDiD/fa768cf3-1977-474a-9ddf-b98c87817229/scratchpad/ssrn-5325686.pdf +**PDF reviewed:** SSRN download of abstract 5325686 (36 pages; cover page dated February 3, 2026; SHA-256 `30b2b9bcd09ce63981671624daccc04deed9350cd98e06a6b402325c2eccc145`). https://ssrn.com/abstract=5325686 | DOI: https://doi.org/10.2139/ssrn.5325686 +**SSRN metadata (verified live 2026-07-13):** 36 pages; Posted: 27 Jun 2025; Last revised: 13 Jun 2026; SSRN "Date Written" field: January 03, 2026 (the author-entered metadata field lags the delivered PDF's cover date of February 3, 2026 - cite the cover date). **Review date:** 2026-07-11 --- diff --git a/docs/references.rst b/docs/references.rst index 5419b701..d9a0663e 100644 --- a/docs/references.rst +++ b/docs/references.rst @@ -283,6 +283,17 @@ Local Projections DiD Origin of the local-projections method that LP-DiD adapts to the difference-in-differences setting. +Rolling-Transformation DiD (Lee-Wooldridge) +------------------------------------------- + +- **Lee, S. J., & Wooldridge, J. M. (2025).** "A Simple Transformation Approach to Difference-in-Differences Estimation for Panel Data." SSRN Working Paper No. 4516518 (61-page revision, cover page June 8, 2026). https://ssrn.com/abstract=4516518 | https://doi.org/10.2139/ssrn.4516518 + + Estimation core for the forthcoming ``LWDiD`` estimator: unit-specific rolling demeaning/detrending converts panel DiD into per-(cohort, period) cross-sectional treatment-effects problems (RA, IPW, doubly robust IPWRA, matching), with contributing-treated-unit weighted event-study aggregation (simplifying to cohort-size weights in balanced panels) and influence-function multiplier-bootstrap inference. Paper review on file at ``docs/methodology/papers/lee-wooldridge-2025-review.md``. + +- **Lee, S. J., & Wooldridge, J. M. (2026).** "Simple Approaches to Inference with Difference-in-Differences Estimators with Small Cross-Sectional Sample Sizes." SSRN Working Paper No. 5325686 (36 pages, cover page February 3, 2026). https://ssrn.com/abstract=5325686 | https://doi.org/10.2139/ssrn.5325686 + + Exact small-sample inference layer: collapsed cross-sectional regressions with exact ``t`` reference distributions (valid down to a single treated unit), HC3 and randomization-inference alternatives, and the composite-outcome aggregate regression for staggered designs. Reference Stata package ``lwdid`` (Hur, Lee & Wooldridge; SSC s459672), whose MIT-licensed ancillary datasets back ``load_prop99()`` and ``load_walmart()``. Paper review on file at ``docs/methodology/papers/lee-wooldridge-2026-review.md``. + Changes-in-Changes / Distributional DiD --------------------------------------- diff --git a/tests/test_datasets.py b/tests/test_datasets.py index 28f2cf2a..a5fbfa51 100644 --- a/tests/test_datasets.py +++ b/tests/test_datasets.py @@ -5,7 +5,7 @@ including both the download/cache mechanism and the fallback data generation. """ -from unittest.mock import patch +from unittest.mock import MagicMock, patch import numpy as np import pandas as pd @@ -16,10 +16,14 @@ _construct_castle_doctrine_data, _construct_divorce_laws_data, _construct_mpdta_data, + _construct_prop99_data, + _construct_walmart_data, clear_cache, list_datasets, load_card_krueger, load_dataset, + load_prop99, + load_walmart, ) @@ -34,7 +38,14 @@ def test_returns_dict(self): def test_contains_expected_datasets(self): """list_datasets should contain all expected datasets.""" result = list_datasets() - expected = {"card_krueger", "castle_doctrine", "divorce_laws", "mpdta"} + expected = { + "card_krueger", + "castle_doctrine", + "divorce_laws", + "mpdta", + "prop99", + "walmart", + } assert set(result.keys()) == expected def test_descriptions_are_strings(self): @@ -56,6 +67,15 @@ def test_load_by_name(self): df = load_dataset("card_krueger") assert isinstance(df, pd.DataFrame) + def test_load_by_name_binary(self): + """load_dataset should dispatch to the binary (.dta) loaders.""" + with patch("diff_diff.datasets._download_with_cache_binary") as mock: + mock.side_effect = RuntimeError("No network") + for name in ("prop99", "walmart"): + with pytest.warns(UserWarning, match="SYNTHETIC"): + df = load_dataset(name) + assert isinstance(df, pd.DataFrame) + def test_invalid_name_raises(self): """load_dataset should raise ValueError for unknown datasets.""" with pytest.raises(ValueError, match="Unknown dataset"): @@ -222,6 +242,278 @@ def test_fallback_data_size(self): assert df["countyreal"].nunique() == 500 +class TestProp99: + """Tests for California Prop 99 smoking dataset.""" + + def test_fallback_data_structure(self): + """Fallback data should have expected structure.""" + df = _construct_prop99_data() + + # Check required columns + required_cols = {"state", "year", "first_year", "lcigsale"} + assert required_cols.issubset(set(df.columns)) + + # Check years + assert df["year"].min() == 1970 + assert df["year"].max() == 2000 + + def test_fallback_data_treatment(self): + """Fallback data should have a single 1989 cohort and zero-coded controls.""" + df = _construct_prop99_data() + + # Exactly one treated state, cohort 1989 + treated_states = df.loc[df["first_year"] > 0, "state"].unique() + assert len(treated_states) == 1 + assert set(df.loc[df["first_year"] > 0, "first_year"].unique()) == {1989} + + # Never-treated states coded 0 + assert df.loc[df["first_year"] == 0, "state"].nunique() == 38 + + def test_fallback_data_values(self): + """Fallback data should have reasonable log-scale values.""" + df = _construct_prop99_data() + + assert (df["lcigsale"] > 2.0).all() + assert (df["lcigsale"] < 6.0).all() + + def test_fallback_data_size(self): + """Fallback data should be a balanced 39 x 31 panel.""" + df = _construct_prop99_data() + + assert df["state"].nunique() == 39 + assert len(df) == 39 * 31 + + def test_load_uses_fallback_on_network_error(self): + """load_prop99 should warn and mark the frame when falling back.""" + with patch("diff_diff.datasets._download_with_cache_binary") as mock: + mock.side_effect = RuntimeError("Network error") + with pytest.warns(UserWarning, match="SYNTHETIC"): + df = load_prop99() + assert isinstance(df, pd.DataFrame) + assert df.attrs["source"] == "synthetic_fallback" + assert "treated" in df.columns + assert "cohort" in df.columns + # treated indicator consistent with first_year timing + in_effect = (df["first_year"] > 0) & (df["year"] >= df["first_year"]) + assert (df["treated"] == in_effect.astype(int)).all() + + +class TestWalmart: + """Tests for Walmart entry county panel.""" + + def test_fallback_data_structure(self): + """Fallback data should have expected structure.""" + df = _construct_walmart_data() + + # Check required columns + required_cols = { + "cid", + "year", + "first_year", + "log_retail_emp", + "log_wholesale_emp", + "x1", + "x2", + "x3", + } + assert required_cols.issubset(set(df.columns)) + + # Check years + assert df["year"].min() == 1977 + assert df["year"].max() == 1999 + + def test_fallback_data_treatment(self): + """Fallback data should have staggered 1986-1999 cohorts + never-treated.""" + df = _construct_walmart_data() + + cohorts = set(df.loc[df["first_year"] > 0, "first_year"].unique()) + assert cohorts.issubset(set(range(1986, 2000))) + + # A meaningful never-treated group coded 0 + assert df.loc[df["first_year"] == 0, "cid"].nunique() > 0 + + def test_fallback_data_values(self): + """Fallback data should have reasonable values.""" + df = _construct_walmart_data() + + # Covariates are shares/rates in (0, 1) + for col in ("x1", "x2", "x3"): + assert (df[col] > 0).all() + assert (df[col] < 1).all() + + def test_fallback_data_size(self): + """Fallback data should be a balanced counties x 23-year panel.""" + df = _construct_walmart_data() + + n_counties = df["cid"].nunique() + assert n_counties == 200 + assert len(df) == n_counties * 23 + + def test_load_uses_fallback_on_network_error(self): + """load_walmart should warn and mark the frame when falling back.""" + with patch("diff_diff.datasets._download_with_cache_binary") as mock: + mock.side_effect = RuntimeError("Network error") + with pytest.warns(UserWarning, match="SYNTHETIC"): + df = load_walmart() + assert isinstance(df, pd.DataFrame) + assert df.attrs["source"] == "synthetic_fallback" + assert "treated" in df.columns + assert "cohort" in df.columns + in_effect = (df["first_year"] > 0) & (df["year"] >= df["first_year"]) + assert (df["treated"] == in_effect.astype(int)).all() + + +class TestSourceValidation: + """Tests for the downloaded-data source validators.""" + + def test_prop99_valid_frame_passes(self): + """A frame matching all source invariants should validate silently.""" + from diff_diff.datasets import _validate_prop99 + + # The seeded fallback matches the real file's invariants exactly + _validate_prop99(_construct_prop99_data()) + + def test_prop99_duplicate_rows_rejected(self): + from diff_diff.datasets import _validate_prop99 + + df = _construct_prop99_data() + df = pd.concat([df, df.iloc[[0]]], ignore_index=True) + with pytest.raises(RuntimeError, match="duplicate"): + _validate_prop99(df) + + def test_prop99_nonconstant_cohort_rejected(self): + from diff_diff.datasets import _validate_prop99 + + df = _construct_prop99_data() + df.loc[df.index[0], "first_year"] = 1975 + with pytest.raises(RuntimeError, match="not constant"): + _validate_prop99(df) + + def test_prop99_multiple_treated_states_rejected(self): + from diff_diff.datasets import _validate_prop99 + + df = _construct_prop99_data() + df.loc[df["state"] == "State02", "first_year"] = 1989 + with pytest.raises(RuntimeError, match="treated state count"): + _validate_prop99(df) + + @staticmethod + def _valid_walmart_frame(): + """Minimal frame satisfying every real-Walmart validator invariant.""" + n_counties, years = 1277, list(range(1977, 2000)) + # 391 never-treated; remaining 886 cycle through cohorts 1986-1999 + cohort_by_cid = {cid: 0 for cid in range(1, 392)} + cohort_cycle = list(range(1986, 2000)) + for i, cid in enumerate(range(392, n_counties + 1)): + cohort_by_cid[cid] = cohort_cycle[i % len(cohort_cycle)] + rows = [ + { + "year": year, + "cid": cid, + "first_year": fy, + "log_retail_emp": 7.5, + "log_wholesale_emp": 6.5, + "x1": 0.1, + "x2": 0.7, + "x3": 0.2, + } + for cid, fy in cohort_by_cid.items() + for year in years + ] + return pd.DataFrame(rows) + + def test_walmart_valid_frame_passes(self): + from diff_diff.datasets import _validate_walmart + + _validate_walmart(self._valid_walmart_frame()) + + def test_walmart_wrong_panel_rejected(self): + from diff_diff.datasets import _validate_walmart + + # The 200-county synthetic fallback must NOT pass as the real panel + with pytest.raises(RuntimeError, match="counties != 1277"): + _validate_walmart(_construct_walmart_data()) + + def test_walmart_duplicate_rows_rejected(self): + from diff_diff.datasets import _validate_walmart + + df = self._valid_walmart_frame() + df.iloc[-1, df.columns.get_loc("year")] = df.iloc[-2]["year"] + with pytest.raises(RuntimeError, match="duplicate"): + _validate_walmart(df) + + def test_walmart_nonconstant_cohort_rejected(self): + from diff_diff.datasets import _validate_walmart + + df = self._valid_walmart_frame() + df.loc[df.index[0], "first_year"] = 1990 + with pytest.raises(RuntimeError, match="not constant"): + _validate_walmart(df) + + def test_walmart_missing_cohort_rejected(self): + from diff_diff.datasets import _validate_walmart + + df = self._valid_walmart_frame() + df["first_year"] = df["first_year"].replace(1986, 1987) + with pytest.raises(RuntimeError, match="treated cohorts"): + _validate_walmart(df) + + def test_walmart_never_treated_count_rejected(self): + from diff_diff.datasets import _validate_walmart + + df = self._valid_walmart_frame() + # Convert one never-treated county to a treated cohort + df.loc[df["cid"] == 1, "first_year"] = 1990 + with pytest.raises(RuntimeError, match="never-treated county count"): + _validate_walmart(df) + + +class TestBinaryDownloadIntegrity: + """Tests for the checksum-verified binary download helper.""" + + def test_checksum_mismatch_raises(self, tmp_path, monkeypatch): + """A fresh download that fails the pinned checksum should raise.""" + import diff_diff.datasets as datasets_mod + + monkeypatch.setattr(datasets_mod, "_CACHE_DIR", tmp_path) + + fake_response = MagicMock() + fake_response.read.return_value = b"tampered bytes" + fake_response.__enter__ = lambda self: self + fake_response.__exit__ = lambda self, *a: False + + with patch("diff_diff.datasets.urlopen", return_value=fake_response): + with pytest.raises(RuntimeError, match="Checksum mismatch"): + datasets_mod._download_with_cache_binary( + "http://example.invalid/x.dta", "x", sha256="0" * 64 + ) + # Tampered bytes must not be cached + assert not (tmp_path / "x.dta").exists() + + def test_stale_cache_triggers_redownload(self, tmp_path, monkeypatch): + """A cached file failing the checksum should be replaced by a re-download.""" + import hashlib + + import diff_diff.datasets as datasets_mod + + monkeypatch.setattr(datasets_mod, "_CACHE_DIR", tmp_path) + good = b"good bytes" + good_sha = hashlib.sha256(good).hexdigest() + (tmp_path / "x.dta").write_bytes(b"stale bytes") + + fake_response = MagicMock() + fake_response.read.return_value = good + fake_response.__enter__ = lambda self: self + fake_response.__exit__ = lambda self, *a: False + + with patch("diff_diff.datasets.urlopen", return_value=fake_response): + content = datasets_mod._download_with_cache_binary( + "http://example.invalid/x.dta", "x", sha256=good_sha + ) + assert content == good + assert (tmp_path / "x.dta").read_bytes() == good + + class TestClearCache: """Tests for cache management.""" @@ -233,6 +525,21 @@ def test_clear_cache_creates_directory(self): except Exception as e: pytest.fail(f"clear_cache raised unexpected exception: {e}") + def test_clear_cache_removes_csv_and_dta(self, tmp_path, monkeypatch): + """clear_cache should remove both .csv and .dta cached files.""" + import diff_diff.datasets as datasets_mod + + monkeypatch.setattr(datasets_mod, "_CACHE_DIR", tmp_path) + csv_file = tmp_path / "dummy.csv" + dta_file = tmp_path / "dummy.dta" + csv_file.write_text("a,b\n1,2\n") + dta_file.write_bytes(b"\x00\x01") + + clear_cache() + + assert not csv_file.exists() + assert not dta_file.exists() + class TestDatasetIntegration: """Integration tests verifying datasets work with estimators.""" @@ -301,3 +608,40 @@ def test_mpdta_with_cs(self): assert hasattr(results, "group_time_effects") assert len(results.group_time_effects) > 0 + + def test_prop99_with_did(self): + """Prop 99 data should work with DifferenceInDifferences.""" + from diff_diff import DifferenceInDifferences + + # Use fallback data + df = _construct_prop99_data() + df["treated_state"] = (df["first_year"] > 0).astype(int) + df["post"] = (df["year"] >= 1989).astype(int) + + did = DifferenceInDifferences() + results = did.fit(df, outcome="lcigsale", treatment="treated_state", time="post") + + assert hasattr(results, "att") + assert hasattr(results, "se") + assert not np.isnan(results.att) + # The synthetic DGP builds in a negative post-1989 effect for California + assert results.att < 0 + + def test_walmart_with_cs(self): + """Walmart data should work with CallawaySantAnna.""" + from diff_diff import CallawaySantAnna + + # Use fallback data + df = _construct_walmart_data() + + cs = CallawaySantAnna(control_group="never_treated") + results = cs.fit( + df, + outcome="log_retail_emp", + unit="cid", + time="year", + first_treat="first_year", + ) + + assert hasattr(results, "group_time_effects") + assert len(results.group_time_effects) > 0 From 8587b356ac3dd36cb0b422e56bb07817c85635fb Mon Sep 17 00:00:00 2001 From: igerber Date: Mon, 13 Jul 2026 13:47:27 -0400 Subject: [PATCH 3/3] fix(docs): unwrap load_prop99 docstring bullet (Sphinx -W) + extend doc-snippet mock loaders with prop99/walmart The doc-snippets harness replaces sys.modules['diff_diff.datasets'] with a mock module; the new loaders' snippets need mock counterparts there. The wrapped Returns bullet in load_prop99 broke the -W docs build (unexpected indentation). Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01FZK3FD9jxWGxBrPDw5APSg --- diff_diff/datasets.py | 3 +-- tests/test_doc_snippets.py | 46 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+), 2 deletions(-) diff --git a/diff_diff/datasets.py b/diff_diff/datasets.py index 58031217..1be02401 100644 --- a/diff_diff/datasets.py +++ b/diff_diff/datasets.py @@ -828,8 +828,7 @@ def load_prop99(force_download: bool = False) -> pd.DataFrame: Panel dataset with columns: - state : str - State name - year : int - Year (1970-2000) - - first_year : int - Year treatment began (1989 for California, - 0 = never treated) + - first_year : int - Treatment start year (1989 for California, 0 = never) - lcigsale : float - Log per capita cigarette sales (packs) - treated : int - 1 if treatment in effect, 0 otherwise - cohort : int - Alias for first_year diff --git a/tests/test_doc_snippets.py b/tests/test_doc_snippets.py index 7e346f46..e1d9e3fc 100644 --- a/tests/test_doc_snippets.py +++ b/tests/test_doc_snippets.py @@ -287,11 +287,51 @@ def _mock_load_mpdta(**kwargs): } ) + def _mock_load_prop99(**kwargs): + states = ["California"] + [f"State{i:02d}" for i in range(2, 11)] + years = list(range(1980, 1996)) + rows = [(s, y) for s in states for y in years] + fy = [1989 if r[0] == "California" else 0 for r in rows] + return pd.DataFrame( + { + "state": [r[0] for r in rows], + "year": [r[1] for r in rows], + "first_year": fy, + "lcigsale": rng.normal(4.6, 0.1, len(rows)), + "treated": [1 if f and r[1] >= f else 0 for f, r in zip(fy, rows)], + "cohort": fy, + } + ) + + def _mock_load_walmart(**kwargs): + counties = list(range(1, 21)) + years = list(range(1985, 1996)) + rows = [(c, y) for c in counties for y in years] + n = len(rows) + cohort_of = {c: (0 if c <= 8 else [1988, 1990, 1992][c % 3]) for c in counties} + fy = [cohort_of[r[0]] for r in rows] + return pd.DataFrame( + { + "cid": [r[0] for r in rows], + "year": [r[1] for r in rows], + "first_year": fy, + "log_retail_emp": rng.normal(7.5, 0.5, n), + "log_wholesale_emp": rng.normal(6.5, 0.5, n), + "x1": rng.uniform(0.05, 0.3, n), + "x2": rng.uniform(0.5, 0.85, n), + "x3": rng.uniform(0.05, 0.4, n), + "treated": [1 if f and r[1] >= f else 0 for f, r in zip(fy, rows)], + "cohort": fy, + } + ) + _dataset_dispatch = { "card_krueger": _mock_load_card_krueger, "castle_doctrine": _mock_load_castle_doctrine, "divorce_laws": _mock_load_divorce_laws, "mpdta": _mock_load_mpdta, + "prop99": _mock_load_prop99, + "walmart": _mock_load_walmart, } def _mock_load_dataset(name, **kwargs): @@ -305,6 +345,8 @@ def _mock_list_datasets(): "castle_doctrine": "Castle Doctrine laws - staggered adoption", "divorce_laws": "Unilateral divorce laws - staggered adoption", "mpdta": "Minimum wage panel data - simulated CS example", + "prop99": "California Prop 99 smoking panel - single treated unit", + "walmart": "Walmart entry county panel - staggered adoption", } # Inject mocks into namespace so `from diff_diff.datasets import ...` works @@ -315,6 +357,8 @@ def _mock_list_datasets(): mock_datasets_mod.load_castle_doctrine = _mock_load_castle_doctrine mock_datasets_mod.load_divorce_laws = _mock_load_divorce_laws mock_datasets_mod.load_mpdta = _mock_load_mpdta + mock_datasets_mod.load_prop99 = _mock_load_prop99 + mock_datasets_mod.load_walmart = _mock_load_walmart mock_datasets_mod.load_dataset = _mock_load_dataset mock_datasets_mod.list_datasets = _mock_list_datasets import sys @@ -327,6 +371,8 @@ def _mock_list_datasets(): ns["load_castle_doctrine"] = _mock_load_castle_doctrine ns["load_divorce_laws"] = _mock_load_divorce_laws ns["load_mpdta"] = _mock_load_mpdta + ns["load_prop99"] = _mock_load_prop99 + ns["load_walmart"] = _mock_load_walmart ns["load_dataset"] = _mock_load_dataset ns["list_datasets"] = _mock_list_datasets