A Counterfactual Journey Engine for American Express · Cross-Channel Journey Stitching
"Everyone else built a better rear-view mirror. We built a flight simulator for the customer relationship."
TimeWeave takes a customer's fragmented touches across four channels (app, web, call center, in-person), stitches them into one identity and one journey, explains why that customer is at risk, and then branches the journey into alternate futures ranked by outcome so an analyst can pick an action and fire it, with a human in the loop and an audit trail behind every decision.
Two things make it different from a dashboard:
- It captures the anonymous signal everyone misses. The pre-login web session where a customer searches "Cancel membership" is normally invisible, because it has no account id attached. TimeWeave resolves it back to the person, with a confidence score, a plain-English explanation, and a reversible un-merge.
- It is honest about causality. Every recommendation is labelled as an observed retention rate within a cohort of similar journeys, correlational and not a causal guarantee. Nothing in the UI or the API claims causal uplift.
The whole stack runs offline: embedded DuckDB, no servers, no API keys, no network calls required.
- Run it
- The Priya walkthrough (the worked example)
- Architecture and data flow
- Feature inventory
- 1. Synthetic data generation
- 2. Ingestion and the canonical event model
- 3. Identity resolution
- 4. Identity and journey graph
- 5. Timeline assembly and friction tagging
- 6. Churn model and per-driver attribution
- 7. The counterfactual cohort engine
- 8. LLM planner and explainer
- 9. Approve, execute, and close the loop
- 10. Population "fix the flow"
- 11. Persistence and live data entry
- 12. The frontend
- Every formula in the project
- Edge cases and how they behave
- API reference
- Database schema
- Configuration
- Offline by design (graceful degradation)
- Design principles (why it is built this way)
- Repository structure
- Version 2 capabilities (merged and shipping)
- Roadmap
Two commands, two terminals. Each script builds its environment on first run (Python venv plus pip
install, or npm install) and then starts the server. Start the backend first.
./scripts/run-backend.ps1 # http://localhost:8000 (interactive docs at /docs)
./scripts/run-frontend.ps1 # http://localhost:5173 (second terminal)If PowerShell blocks the scripts, either unblock once with
Set-ExecutionPolicy -Scope Process Bypass, or run the commands inline:
cd backend
python -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements.txt
.\.venv\Scripts\python.exe -m uvicorn main:app --reload --port 8000
# second terminal:
cd frontend; npm install; npm run dev./scripts/run-backend.sh # http://localhost:8000
./scripts/run-frontend.sh # http://localhost:5173 (second terminal)backend/.venv/Scripts/python scripts/seed.py # Windows
backend/.venv/bin/python scripts/seed.py # macOS / Linuxscripts/seed.py loads the population into an in-memory DuckDB (so it never fights a running server
for the file, DuckDB being single-writer), resolves identities, trains the churn model, and prints
Priya's normalized journey, her identity resolution, her churn plus attribution, the cohort
comparison, the edge-case behaviour, and the population totals.
The frontend talks to http://localhost:8000 by default; override with VITE_API_BASE (see
frontend/.env.example). If the backend is down, Priya's hero flow still renders from the static seed
in frontend/src/data/priya.ts, so the demo never hard-fails.
Deep links for screen recording:
| URL | View |
|---|---|
http://localhost:5173/ |
Identity |
http://localhost:5173/?view=journey |
Journey |
http://localhost:5173/?view=counterfactual |
Counterfactual |
http://localhost:5173/?view=population |
Population "fix the flow" |
Priya Sharma, golden-priya-001, Platinum, 6-year tenure, renewal in 8 days,
fee_sensitivity = 0. She is the authored hero journey in backend/data/seed_priya.py. Every number
below is what the running system actually produces, not an illustration.
| # | When | Channel | Raw schema | What happened | Outcome |
|---|---|---|---|---|---|
| 1 | 2026-07-11 10:12 | app | Adobe XDM ExperienceEvent | Redeem 60k points for travel, fails at the payment step | redemption_failed |
| 2 | 2026-07-11 10:19 | app | Adobe XDM | Retry, fails again | redemption_failed |
| 3 | 2026-07-12 21:40 | web | Adobe XDM (ECID, ambiguous) |
Reads "Is Amex Platinum worth it?" | none |
| 4 | 2026-07-12 21:43 | web | Adobe XDM (ECID, ambiguous) |
Views "Cancel membership", the signal everyone misses | none |
| 5 | 2026-07-12 21:47 | web | Adobe XDM (ECID, ambiguous) |
Reads "Points redemption problems" | none |
| 6 | 2026-07-14 09:05 | web | Adobe XDM (account and the same ECID, authenticated) |
Logs in, opens support case R123 | none |
| 7 | 2026-07-14 15:22 | call | Amazon Connect Contact Trace Record | 14-minute call, disposition unresolved, callback promised |
call_unresolved |
| 8 | 2026-07-15 15:22 | system | system-derived | Callback SLA breached, the promised call never happened | callback_broken |
| 9 | 2026-07-16 13:30 | in_person | ISO 8583 (MTI 0200, tokenized PAN) |
Centurion lounge check-in, a positive but disconnected touch | positive |
Events 3, 4 and 5 carry no account id. In a normal stack they are orphans. That is the problem TimeWeave solves.
Four channel streams collapse into one golden identity:
| Stream | Identifier | Method | Confidence |
|---|---|---|---|
| app | acct-priya-001 |
deterministic | 1.00 |
| web | ECID-77f3a1 |
fuzzy | 0.91 |
| call | +1-•••-4821 (masked) |
deterministic | 1.00 |
| in_person | tok_9f2a… (tokenized) |
deterministic | 1.00 |
The fuzzy merge explains itself verbatim:
Merged at 0.91: same device fingerprint (0.4), IP geolocation match (0.2), cookie later resumed as authenticated account (0.31).
That is 0.40 × 1.0 + 0.20 × 1.0 + 0.31 = 0.91, above the 0.85 auto-merge threshold. The decisive
piece of evidence is the third one: the exact cookie (ECID ECID-77f3a1) that browsed anonymously on
2026-07-12 later authenticated as her account on 2026-07-14. Absence of that continuation is
exactly why the look-alike session in the same city fails to merge (see
edge cases).
Click the edge in the UI and you get the confidence bar against the 0.55 / 0.85 thresholds, the weighted matched-feature breakdown, the method, the status, a "reversible" badge, and an Un-merge button. Un-merging drops the anonymous web track from her journey and is recorded in the append-only merge history.
The nine events render as a cross-channel swim-lane timeline (rows are channels, columns are a shared
time axis) with a connector weaving between lanes, so every vertical jump is a channel handoff.
Nodes are tagged drop_off, escalation, unresolved_issue and anonymous_signal.
The trained churn model scores her:
churn risk: 66.2 % model: logistic-regression (scikit-learn), training accuracy 0.717
Attribution, computed as leave-one-out marginals in probability space:
+26.4 Broken callback promise
+19.6 Redemption failure (x2)
+13.7 Anonymous "cancel" research
-3.1 Tenure 6y (protective)
This ordering is the whole point of the demo. The risk is a broken promise, not price. Her
fee_sensitivity is 0 and the fee never appears in her attribution at all, which is what makes the
obvious retention play (a fee discount) the wrong answer.
The generative world model that produced the labels agrees, which is the honesty check:
z = -1.42 + 0.30x2 + 0.85 + 0.50 + 0.25 + 0.22 + (-0.04 x 6) + 0.10x0 = 0.76
P(churn | do nothing) = sigmoid(0.76) = 0.681 -> 68 %
The trained classifier independently lands at 66.2 %, so the model recovered the true drivers rather than being told them.
Her matched cohort is:
Customers with {redemption failure + broken callback + cancel-page research + unresolved call} within days of renewal · N = 361
Three branches, with the observed cohort rate, the value-personalized projection, and the 95 % Wilson band:
| Branch | Observed cohort rate | Projected for Priya | Projected churn | 95 % band | Lift over do-nothing | ROI rank |
|---|---|---|---|---|---|---|
| Do nothing | 33.0 % retained | 33.8 % | 66.2 % | 29.1 to 38.8 % | 0.0 pts | 3 |
| Fee discount ($200) | 59.3 % retained | 60.1 % | 39.9 % | 54.9 to 65.0 % | +26.3 pts | 2 |
| Senior callback + restore points ($600) | 78.7 % retained | 79.5 % | 20.5 % | 75.0 to 83.4 % | +45.7 pts | 1 |
The gauge drains from 66.2 % down to 20.5 % along the winning branch. The fee discount carries an explicit "obvious · weak" badge because a clearly better option exists whose confidence band sits entirely above its own. Every card renders the mandatory basis line verbatim:
Projected from this member's current retention (33.8 %) plus the lift observed across 361 similar journeys, correlational, not a causal guarantee.
The plan (templated, or LLM-drafted if a key is present) is concrete:
Senior specialist to call Priya Sharma within 24h. Acknowledge the failed points redemption and the missed callback, restore the points, and honor the original value. Restore points: 60,000. Outreach: "We saw the trouble redeeming your points, and that we didn't call you back as promised. We've restored your points and a senior specialist will see this through personally."
Because the senior callback costs $600 and the auto-execute ceiling is $250, Approve & Execute
records a human approval rather than firing autonomously. It writes an Intervention node onto her
graph and shows predicted vs observed: predicted 79.5 %, and then "Close the loop" reveals her
synthetic ground truth, retained_under.senior_callback = True, so retained.
Apply the winning intervention across her whole cohort:
| Metric | Value |
|---|---|
| Cohort N | 361 |
| Do-nothing retention | 33.0 % |
| Recommended retention | 78.7 % |
| Churn prevented | 165 members retained |
| Repeat calls avoided (est.) | 380 fewer contacts |
| Value preserved (est.) | $2,475,196 (about $2.48M) |
Labelled: "Projected across 361 similar synthetic journeys, correlational, not a causal guarantee", plus "Validated at scale requires a controlled rollout / experiment."
raw source schemas canonical resolution intelligence
------------------ --------- ---------- ------------
Adobe XDM (web + app) ─┐
Amazon Connect CTR ────┤ ┌──────────────┐ ┌──────────────────┐ ┌────────────────────┐
ISO 8583 (in-person) ──┼──►│ normalize.py │──►│ deterministic │──►│ features.py │
system (SLA breach) ───┘ │ CanonicalEvent│ │ + fuzzy matcher │ │ churn.py (logistic)│
└──────────────┘ │ review queue │ │ attribution │
│ │ un-merge + audit │ │ cohort.py (Wilson) │
▼ └──────────────────┘ │ journey.py timeline│
┌──────────────┐ │ └────────────────────┘
│ DuckDB file │◄───────────┘ │
│ events │ ▼
│ ground_truth │ ┌───────────────┐
│ profiles │ │ llm/planner │
└──────────────┘ └───────────────┘
│ │
└──────────────► FastAPI (main.py) ◄─────────┘
│
▼
React + Vite + Tailwind + react-flow + framer-motion
Identity ──► Journey ──► Counterfactual ──► Population (4-step story)
On first boot, EventStore.load() runs: generator produces raw events, ingestion normalizes them,
_deterministic_resolve links anonymous ECIDs by cookie continuation, features.extract_features
derives the feature vector per customer, outcomes.draw_ground_truth draws each customer's potential
outcome under every intervention, and everything is bulk-loaded into DuckDB and checkpointed. On every
later boot it hydrates from the file instead of regenerating.
backend/data/generator.py, backend/data/seed_priya.py
-
835 customers, 4,213 events across five channel labels:
app 1275 · call 953 · web 827 · in_person 743 · system 415. -
Events are emitted in real industry schemas, not a bespoke format. Map real Amex feeds onto these shapes and the pipeline runs unchanged:
xdm: Adobe XDM ExperienceEvent, with anidentityMapcarryingaccount/ECID/devicenamespaces and per-namespaceauthenticatedStateofauthenticated/ambiguous/anonymous.ctr: Amazon Connect Contact Trace Record, withContactId,InitiationTimestamp,DisconnectTimestamp,CustomerEndpoint.Address,Queue,Agent,Attributes.disposition,Attributes.callback_promised, and a transcript snippet.iso8583: ISO 8583 message, withMTI 0200, a tokenizedPAN(tok_xxxx),amount,MCC,terminalId,transmissionDateTime.system: system-derived events, such assystem.sla_breachfor a broken callback promise.
-
Identities are realistic (Faker), seeded at a fixed value (
n_seed = 42) so every run is byte-identical. No wall-clock is used anywhere, so results are reproducible. -
Four cohort journey shapes, sized so the hero cohort is the largest and its rates are therefore tight and believable:
cohort_keyMembers Journey shape redemption_failed+callback_broken@renewal361 (360 generated + Priya) two failed redemptions, anonymous cancel research, unresolved call, SLA breach, lounge check-in redemption_failed@renewal180 one or two failed redemptions, a card touch serial_caller@midcycle91 (90 + Marcus Lee) 5 to 7 unresolved calls healthy@midcycle202 (200 + the two household members) successful redemption, a card touch unresolved1 the false-positive-trap session -
Per-customer feature noise. Non-hero customers get extra risk drivers at controlled rates: an authenticated cancel-page view at 24 %, an SLA breach at 12 %, a third failed redemption at 18 %, an extra unresolved call at 12 %. Rationale: without this the drivers are perfectly correlated inside a cohort and the churn model can only learn "which cohort", not "which driver". The noise decorrelates them so each coefficient is individually identified, which is what makes the attribution ordering meaningful. Each noise event is a real, deterministically resolvable event that also appears in the timeline, so nothing is invisible.
-
Renewal timing variance. The hero cohort is pinned to 4 to 12 days from renewal (it is defined as "within days of renewal"); other renewal-stage customers span 3 to 40 days. That spread is what gives the
near_renewalfeature the variance the model needs.
backend/ingestion/normalize.py, backend/schemas/canonical.py
One normalizer function per raw schema, dispatched on the _source tag, all producing the same
CanonicalEvent pydantic model:
| Field | Meaning |
|---|---|
event_id |
stable id from the source |
customer_ref |
the raw source identifier before resolution: ECID, phone, account key, or tokenized PAN |
resolved_golden_id |
filled by identity resolution; null for anonymous touches until the resolver runs |
channel |
app / web / call / in_person / system |
event_type |
e.g. application.redemption, web.webpagedetails.pageViews, call.contact, lounge_checkin, system.sla_breach |
timestamp |
ISO-8601, assumed UTC |
authenticated, auth_state |
authenticated / ambiguous / anonymous, derived from the XDM identityMap |
attributes |
per-schema payload, flattened (page name, points, disposition, MCC, device fingerprint, IP geo, and so on) |
outcome |
redemption_failed, call_unresolved, callback_broken, redemption_succeeded, positive, or null |
source_schema |
human label of the raw schema it came from, shown in the UI detail strip |
Channel derivation: XDM eventType starting with application is the app channel, everything else
XDM is web. customer_ref prefers the account id when authenticated and falls back to the ECID or
device id otherwise, which is precisely the split that leaves anonymous touches unresolved for the
fuzzy pass to pick up.
backend/resolution/matcher.py, backend/resolution/resolver.py
Pass 1, deterministic (store._deterministic_resolve plus normalize._resolved): authenticated
account keys, phones on file, tokenized PANs and authenticated device ids resolve at confidence
1.00. Anonymous web touches are additionally linked by ECID cookie continuation: if the same
ECID later appears in an authenticated event, every earlier touch on that ECID is attributed to that
golden identity and tagged _resolution = ecid_continuation. This is exact, not probabilistic, and it
is the honest half of the story.
Pass 2, fuzzy (RapidFuzzMatcher): anonymous sessions are scored against a candidate identity on a
weighted evidence sum over three features.
confidence = min( 0.99 ,
0.40 x similarity(session.fingerprint, target.fingerprint)
+ 0.20 x similarity(session.geo, target.geo)
+ 0.31 x [ cookie later authenticated as this account ] )
Weights and rationale:
| Feature | Weight | Why that weight |
|---|---|---|
| device fingerprint | 0.40 | the strongest single passive signal; the same physical browser |
| IP geolocation | 0.20 | corroborating but weak on its own; a whole metro shares it |
| cookie continuation | 0.31 | the decisive evidence: this exact cookie later authenticated as the account. Its absence is why look-alikes fail |
similarity is rapidfuzz.fuzz.ratio / 100 when rapidfuzz is installed, and
difflib.SequenceMatcher(...).ratio() from the standard library otherwise. Both are in [0, 1], exact
matches short-circuit to 1.0, and an empty value scores 0.0. The 0.99 cap exists so the system
never claims certainty from probabilistic evidence.
Thresholds (matcher.AUTO_MERGE, matcher.REVIEW_FLOOR):
confidence >= 0.85 -> active (auto-merge)
0.55 <= conf < 0.85 -> pending_review (a human steward decides)
confidence < 0.55 -> rejected (held back, never merged)
Rationale: a hard auto-merge floor keeps precision high on identity, which is the one thing a financial institution cannot get wrong. The 0.55 to 0.85 middle band is deliberately routed to a human rather than guessed, and everything below 0.55 is surfaced as a guardrail ("we correctly did not merge this") rather than hidden.
Every merge is a first-class, auditable record (MergeRecord):
merge_id, source_label, source_id, source_type (account / phone / card_token / ecid /
device / session), target_golden_id, target_label, confidence, method
(deterministic / fuzzy), status, reversible, merged_at, a plain-English explanation, and the
weighted matched_features breakdown.
Additional resolution features:
- Identifier masking. Phones render as
+1-•••-4821, card tokens astok_9f2a…. Nothing sensitive is displayed in full even though the data is synthetic and already tokenized. - Shared-PAN household detection. If one tokenized PAN maps to more than one golden id, the deterministic card link is downgraded into a single household merge at 0.71 · pending_review, because the cardmember names differ. Individual-versus-household ambiguity is a real resolution failure mode and the system refuses to resolve it silently.
- Unresolved-candidate scoring. Anonymous sessions that never linked deterministically are still scored against their nearest candidate, so the system can prove it held one back.
- Reversibility.
POST /unmergesets a merge tounmerged;POST /remergerestores it toactiveorpending_reviewdepending on whether its confidence clears 0.85. The DuckDB rows are never mutated, so the operation is truly non-destructive. - Un-merge propagates.
resolver.suppressed_source_ids(gid)lists the source identifiers whoseSAME_ASedge is currently un-merged./journeydrops their events from the timeline, andstore.live_features_forrecomputes features from the surviving events, so the churn score and the attribution both update. Detach Priya's anonymous session andcancel_page_viewgoes to 0, so her risk genuinely falls instead of going stale. - Append-only merge history. Every
created,unmerged,remerged,review:approveandreview:rejectaction is appended with actor and timestamp, and is exported as JSON from the population view. - Review queue.
GET /review-queuelists pending merges;POST /review/{merge_id}withapprove/rejectresolves them. The UI has a pulsing badge and an approve-or-reject card per item. - Swappable matcher.
Matcheris a one-method interface (score_session). Theversion2branch ships a learned Fellegi-Sunter implementation behind it; a real Splink model slots in the same way.
backend/graph/graph.py
A temporal identity plus journey graph, built per identity on demand, backed by networkx when installed and by an equivalent zero-dependency structure otherwise.
- Node types:
Identity,SourceIdentifier,Event,Outcome,Intervention. - Edge types:
SAME_AS(carrying confidence, method, status and explanation),PERFORMED,LED_TO(the temporal chain between consecutive events),HAS_OUTCOME,HAS_INTERVENTION. GET /graph/{gid}returns the subgraph as nodes and edges;GET /graph-statsreturns node-type and edge-type counts across the whole population;to_networkx(gid)materializes a realnetworkx.MultiDiGraphfor graph algorithms.- Rejected merges are excluded from the graph, so a held-back look-alike never contaminates a journey.
- Approving an intervention adds an
Interventionnode with predicted retention, observed outcome, basis, approver and execution timestamp.
backend/intelligence/journey.py
Turns a stitched identity's canonical events into a time-ordered journey with four friction flags:
| Flag | Fires when |
|---|---|
drop_off |
outcome is redemption_failed |
escalation |
a second redemption failure, or any call_unresolved / callback_broken |
unresolved_issue |
outcome is call_unresolved or callback_broken |
anonymous_signal |
the event is anonymous / ambiguous or was linked by cookie continuation, and it is a web event or its page name contains "cancel" |
Each timeline item also carries a sequence number, a human title, the source schema label, the raw
event type, a relative time string, the auth state, cleaned attributes (internal _-prefixed keys are
stripped before the UI sees them), and the outcome.
Relative time is computed against the journey's own latest event, not the wall clock:
days = (latest.date - event.date).days
label = "T-{days}d HH:MM" if days > 0
"T-0 HH:MM" otherwise
Rationale: the demo must read identically today and in six months, so the timeline is anchored to the data rather than to "now".
Titles are humanized per event type: a first 60k-point redemption reads "Redeem 60k pts", the second
reads "Retry redemption", system.sla_breach reads "Promised callback never happened", a
lounge_checkin reads "Centurion lounge check-in", and a web view renders its page name in quotes.
backend/data/features.py, backend/intelligence/logreg.py,
backend/intelligence/sklearn_model.py, backend/intelligence/churn.py
One feature definition, used twice. features.extract_features is the single source of truth. Both
the synthetic world model that draws the ground truth and the classifier that learns to predict
it read features through this one function. Rationale: this guarantees there is no drift between
"what happened" and "what we learned", so a recovered coefficient is evidence and not a coincidence.
The seven features, in the fixed order the coefficient vector is aligned to:
| Feature | Type | Extracted from |
|---|---|---|
redemption_failures |
count | number of redemption_failed outcomes |
callback_broken |
0/1 | any callback_broken outcome |
cancel_page_view |
0/1 | any event whose page name contains "cancel" |
unresolved_call |
0/1 | any call_unresolved outcome |
near_renewal |
0/1 | renewal_in_days <= 14 |
tenure_years |
numeric | profile |
fee_sensitivity |
0/1 | profile |
Features are extracted from the stitched journey (post identity resolution), which is exactly the journey a human analyst sees.
The classifier. A genuinely trained logistic regression over the whole population, labelled with
churned_no_action. Two interchangeable backends behind one interface:
sklearn_model.SklearnLogistic(preferred when scikit-learn is importable),solver="lbfgs",C = 100.0,max_iter = 5000. The largeCmeans light regularization, chosen so the coefficients stay close to the near-unregularized pure-Python fit and Priya's calibration is preserved.logreg.LogisticRegression, a pure-Python full-batch gradient-descent trainer with L2, zero third-party dependencies:lr = 0.3,iters = 4000,l2 = 1e-3. Deterministic: same data in, same weights out.
Both standardize features first (z = (x - mean) / std, with std falling back to 1.0 when a column
is constant), which is what makes a single fixed learning rate behave across features on wildly
different scales.
Live training accuracy: 0.717, surfaced at /system/status and in the UI.
Attribution. Per-driver contributions are computed in raw feature space, then converted to percentage points via a leave-one-out marginal. The full derivation is in Every formula; the short version is:
contribution_j = (w_j / std_j) x x_j # raw-space log-odds contribution
points_j = ( sigmoid(full_logit) - sigmoid(full_logit - contribution_j) ) x 100
Rationale for this design, in three parts:
- Why raw space and not standardized space. A standardized contribution is
w_j x (x_j - mean_j)/std_j, which is an effect measured relative to the population average. That makes a customer with zero of a bad thing look protective, and it distorts the ordering. Folding standardization back out givesw_raw_j x x_j, the true per-driver effect times its actual value, which recovers the generative ordering. The identitysum(contributions) + raw_intercept == logit(x)holds exactly. - Why leave-one-out marginals and not raw log-odds. Log-odds are not interpretable to a business audience. "Removing the broken callback drops predicted churn by 26.4 points" is. The marginal is evaluated in probability space, which correctly accounts for the sigmoid's nonlinearity: the same log-odds contribution is worth more near p = 0.5 than in the tails.
- Why not SHAP. The seed explicitly allowed a simple additive contribution model. For a logistic model with no interaction terms, the leave-one-out marginal is the exact per-feature effect, so SHAP would add a dependency and an approximation without adding information.
Two display rules keep the story clean without hiding anything material: features whose value is 0
contribute nothing and are dropped (with tenure_years exempt, since it is the one protective factor
and is always present), and drivers under +0.5 points are suppressed as collinearity noise. Tenure is
only shown when it is meaningfully protective (below -0.5 points). This is why Priya shows four
drivers rather than seven; unresolved_call is real but lands under the noise floor once the broken
callback (which it always co-occurs with) is accounted for.
backend/data/outcomes.py, backend/intelligence/cohort.py, store.cohort_query
This is the honest core of the product. It is not a causal model, and it never claims to be.
The generative world model. Instead of inventing cohort retention numbers at display time, one generative model defines how features map to outcomes:
no-action churn log-odds: z = intercept + sum_i ( w_i x feature_i )
intervention: P(churn | arm) = sigmoid( z - delta(features, arm) )
ground truth: retained_under[arm] ~ Bernoulli( 1 - P(churn | arm) )
Calibrated weights, and why each one is what it is:
| Feature | Weight | Rationale |
|---|---|---|
| intercept | -1.42 | sets the base rate so an unremarkable customer is low risk |
callback_broken |
+0.85 | the top driver: a broken promise is a trust event, not a service event |
cancel_page_view |
+0.50 | the anonymous signal everyone misses; intentionally large, because surfacing it is the product's reason to exist |
redemption_failures |
+0.30 each | compounding: each retry that fails adds risk |
unresolved_call |
+0.25 | real, but the SLA breach that follows it matters more |
near_renewal |
+0.22 | timing pressure: a decision point is imminent |
tenure_years |
-0.04 per year | protective: loyalty buys patience |
fee_sensitivity |
+0.10 | deliberately small, so the fee genuinely barely matters and the "obvious but weak" story is a property of the data rather than a script |
Intervention effects, as a reduction in the churn log-odds (larger means it helps more):
delta(do_nothing) = 0
delta(fee_discount) = 1.20 # flat, moderate, indiscriminate
delta(senior_callback) = 0.70
+ 0.50 if callback_broken
+ 0.40 if redemption_failures > 0
+ 0.30 if unresolved_call
Rationale: the fee discount is a flat bribe, so it helps everyone the same amount regardless of what is actually wrong. The senior callback is conditionally stronger: it helps extra precisely when it repairs a driver the customer actually has. That single asymmetry is what makes "understand the journey, then fix the actual problem" beat "throw money at it", and it is encoded in the data-generating process rather than asserted in the UI.
For Priya: delta(fee_discount) = 1.20, but
delta(senior_callback) = 0.70 + 0.50 + 0.40 + 0.30 = 1.90, because her journey has all three
repairable drivers.
Two properties fall out of this, both real properties of the generated data:
- A classifier trained on the drawn labels recovers these weights, so its predicted risk and its attribution ordering are honest rather than hardcoded.
- Aggregating drawn outcomes over a behavioural cohort yields observed retention rates, which is the correlational "outcome rate for N similar journeys" the demo shows.
Cohort matching. A customer's features are binarized into five flags
(redemption_failure, callback_broken, cancel_page_view, unresolved_call, near_renewal), and
store.cohort_query runs one DuckDB aggregate for the exact-match cohort:
SELECT COUNT(*),
AVG(CASE WHEN retained_do_nothing THEN 1.0 ELSE 0.0 END),
AVG(CASE WHEN retained_fee_discount THEN 1.0 ELSE 0.0 END),
AVG(CASE WHEN retained_senior_callback THEN 1.0 ELSE 0.0 END)
FROM ground_truth
WHERE (redemption_failures > 0) = ? AND callback_broken = ?
AND cancel_page_view = ? AND unresolved_call = ? AND near_renewal = ?The flags are derived from the customer's live features, so an un-merge changes which cohort is
selected. Any flag may also be passed as None, meaning "do not constrain on this dimension", which
is what the back-off below uses to assemble the predicate dynamically.
Cohort back-off, so a rate can never degenerate. An exact-match cohort can shrink to a handful of
members, at which point a rate reads as a meaningless 0 % or 100 %. If the cohort falls under
MIN_COHORT = 25, the least-defining flags are relaxed in a fixed order
(cancel_page_view, then unresolved_call, then near_renewal) until it is large enough again. The
churn drivers, redemption failure and broken callback, always stay constrained, so the cohort still
means something. The response reports exactly which dimensions were widened in cohort_widened, and
the UI says so on screen. Full detail in
section F.
Ranking by expected value, not by retention.
LTV_by_tier = { Platinum: 15000, Gold: 6000, Green: 2500, unknown: 8000 }
value_score = retention_rate x LTV - cost
Interventions are sorted by value_score descending, each gets a roi_rank, and the top one is the
winner. Rationale: the highest retention rate is not automatically the right business decision.
Ranking on retention alone would always pick the most expensive option. Ranking on expected value can
correctly choose a cheaper action when the difference in outcome is not worth the money, and that is
what makes the recommendation defensible to a business audience rather than just to a data scientist.
Uncertainty on every number. Each rate carries a 95 % Wilson score interval (formula in the
reference section). Rationale: the normal approximation to a binomial proportion is badly behaved
near 0 and 1 and can produce intervals outside [0, 1]. Wilson is well behaved at the extremes and at
small N, and it stays honest when a cohort is small, which is exactly when a demo is most tempted to
overclaim.
"Obvious but weak" is decided dynamically, not hardcoded. The fee discount gets the badge only when some other option's lower confidence bound sits entirely above the fee's upper bound (a real, non-noise gap) and the fee is not itself the winner. Rationale: the label must be earned by the data on the customer currently on screen. Hardcoding it to the hero narrative would make it a lie the moment an analyst clicks a different customer, and it also prevents the contradictory state of an option being both "recommended" and "weak".
Honesty on statistical ties. If the winner has a lower point estimate than another option but their 95 % bands overlap, the reason text says so explicitly:
(Statistically tied with "Senior callback + restore points", their 95 % confidence bands overlap, so it is recommended on cost/value.)
Personalized projection. The cohort is matched on the full stitched journey, which is a stable population fact and does not collapse to N = 1. The per-branch numbers are then anchored onto this customer's live risk:
shift = baseline_retention - cohort_do_nothing_retention
projected = clamp( observed_cohort_rate + shift , 0 , 99.9 )
with baseline_retention = 100 - live_churn_risk, and the confidence bands shifted by the same amount.
Rationale, and the tradeoff being made: the lift and therefore the ranking, the winner and the
band widths are all unchanged; only the absolute level moves. This buys three things. First, the
do-nothing branch projects to exactly the customer's churn risk, so the gauge in step 2 and the
do-nothing branch in step 3 agree instead of contradicting each other. Second, an un-merge, which
lowers individual risk, flows through to every branch rather than leaving step 3 stale. Third, when
nothing is un-merged the baseline is already approximately the cohort's do-nothing rate, so the shift
is near zero (for Priya: 33.8 - 33.0 = +0.8) and the hero numbers are preserved. The tradeoff, stated
plainly: this is a projection anchored on an individual model score, not a second independent
measurement, and the response flags it with personalized: true and rewrites the basis string to say
so.
backend/llm/planner.py
- The LLM is a planner and explainer, never a journey summarizer. It converts the churn attribution plus the cohort result into a concrete, human-approvable plan: a senior-agent callback script, a points-restoration amount, a proactive outreach message, and a one-sentence "why" grounded in the numbers. Rationale: summarizing a journey is the one job where a language model can hallucinate a fact that the rest of the system is specifically built to get right. Drafting an action from numbers that are already computed is the job where it adds real value and cannot invent evidence.
- Privacy guardrail. Only
golden_id, tier, churn risk, the top three driver names, the recommended action id, cohort N and the two retention rates are sent. No name, email, phone or PAN ever enters a prompt. - The prompt forbids causal language and states that the retention figure is an observed cohort rate.
- Graceful degradation, three layers deep. No
ANTHROPIC_API_KEYmeans templated text; the SDK not being importable means templated text; any exception at call time means templated text plus anllm_errorfield. The response always carriessource: "llm" | "template"andllm_enabled, so the UI can label which path produced what. The demo cannot break because of the network. - The templated path is not a stub. It produces the full plan, branching on which intervention won, and it is what the offline fallback bundle ships.
POST /approve in backend/main.py
-
A real cost guardrail, not a disclaimer.
AUTO_EXECUTE_COST_LIMIT = 250.requires_approval = intervention.cost > 250 auto_executed = not requires_approvalSo the $600 senior callback requires a recorded human approval, while the $200 fee credit and the $0 do-nothing can auto-execute. Rationale: autonomy should scale with blast radius. A cheap, reversible action does not need a human in the path; an expensive one does. Encoding that as an actual threshold rather than a UI note is what makes the guardrail claim credible.
-
Mock execution adds an
Interventionnode to the customer's graph, carrying the predicted retention, the observed outcome, the basis, the approver and the execution timestamp. -
Predicted versus observed. The response returns the predicted retention alongside the observed outcome read from that customer's synthetic ground truth (
retained_under[intervention_id]), gated behind a "Close the loop" reveal in the UI, and labelled: "Synthetic ground-truth reveal, in production this fills when the real outcome lands."
GET /population/fix-the-flow
Turns one customer into a business case by applying the winning intervention across the whole hero cohort:
retained_delta = ( senior_callback_rate - do_nothing_rate ) x N
churn_prevented = round(retained_delta)
calls_avoided_est = round(retained_delta x 2.3) # illustrative repeat contacts per retained member
value_preserved_est = round(retained_delta x 15000) # illustrative retained LTV
Live output: N = 361, 33.0 % to 78.7 %, 165 members retained, 380 calls avoided,
$2,475,196 preserved. Both multipliers are explicitly flagged as illustrative, and the response
carries basis ("correlational, not a causal guarantee") and roadmap_note ("Validated at scale
requires a controlled rollout / experiment").
The same view offers the regulator-ready audit export: a JSON download of every merge with its confidence, matched features, method, status and the full append-only history.
backend/data/store.py
- File-backed DuckDB, embedded, no server. Default
backend/timeweave.duckdb(gitignored), overridable withTIMEWEAVE_DB, andTIMEWEAVE_DB=:memory:for an ephemeral run. - Hydrate, do not regenerate. If
ground_truthalready has rows, the store rebuilds its in-memory caches from the file. If hydration throws (schema drift, a partial write), it falls through and regenerates rather than starting up broken. - Thread-safe by construction. DuckDB connections are not thread-safe, and FastAPI serves each request on a threadpool thread while the frontend fires several calls in parallel. All database access is serialized through a single reentrant lock, which is what fixed the "signal is aborted" and laggy-UI failures.
- Fast bulk load. The population is inserted via a temp-file
COPY ... FROM ... (FORMAT CSV)rather thanexecutemany. Rationale: DuckDB is columnar; binding every value through Python row-by-row takes roughly 15 seconds for the full population, while a vectorized CSV copy takes about 0.1 seconds, with no pandas or pyarrow dependency. This is what removed the cold-start that made customers fail-then-load in the UI. GET /admin/dbproves persistence: file path,persistentflag, and row counts.POST /admin/customerwrites a brand new customer (profile plus raw source-schema events) into the database, then rebuilds resolution and retrains the churn model so it appears live in the UI.POST /admin/eventsappends activity to an existing customer, recomputes their features, re-draws their ground truth with a stable per-customer seed, upserts the ground-truth row, rebuilds resolution and retrains, so the timeline, the gauge and the interventions all move together.POST /admin/regenerate?n_seed=wipes and rebuilds the whole population.- Determinism everywhere. Fixed seeds for Faker and every RNG, stable per-customer seeds for
incremental draws (
random.Random(f"add:{gid}")), and fixed timestamp constants instead ofdatetime.now(). Same input, same output, every run.
frontend/src/ · React 18 + Vite 5 + TypeScript + Tailwind 3 + react-flow 11 + framer-motion 11
The four-step story, driven by a header stepper and forward buttons inside each view
(1 · Identity, 2 · Journey, 3 · Counterfactual, 4 · Population).
Identity view (IdentityView.tsx, MergeCard.tsx, ReviewQueue.tsx)
- A react-flow canvas: one node per channel stream on the left, the golden identity on the right.
Deterministic edges are thin and grey and labelled
deterministic; fuzzy edges are thick and purple and labelledSAME_AS · 91%. - Clicking a fuzzy edge opens the merge card: confidence against the 0.55 / 0.85 scale, weighted matched-feature bars, the plain-English explanation, method / status / reversible chips, and Un-merge or Re-merge.
- Un-merge is optimistic: the edge instantly goes dashed and faded, then the backend call fires and the bundle refetches so the timeline reflects the detached track. If the call fails the optimistic state stands, so the demo survives an offline backend.
- The review queue badge pulses with the pending count and opens approve-or-reject cards.
- A guardrail panel shows look-alike sessions the matcher correctly held back, with their score and the explanation of what was missing.
- The graph re-fits its viewport 300 ms after the view transition settles, keyed on the customer.
Rationale: calling
fitViewduring the enter animation measures a moving container and lands at a slightly different zoom on each visit, which made the framing look unstable across revisits.
Journey view (JourneyView.tsx, Timeline.tsx, ChurnGauge.tsx, AttributionList.tsx)
- A cross-channel swim-lane timeline: rows are channels in the order app, web, call, system,
in-person (only lanes that have events are rendered), columns are the ordered events on a shared time
axis, with day-group headers across the top. An SVG connector weaves between lanes in journey order,
drawn as smooth cubic curves measured from live element geometry via
ResizeObserver, and it turns purple and dashed on any segment touching an anonymous signal. - Every chip is colour-coded by outcome, carries flag dots, and shows an
anontag where relevant. Clicking one opens a detail strip with the title, the channel, the relative time, the source schema, the flag chips, the outcome chip, and the three most telling raw attributes. - The churn gauge: a 270-degree SVG arc starting at 135 degrees, with the number tweening over 1.4 s on a custom cubic-bezier ease, and the colour stepping red (>= 55) to orange (>= 40) to yellow (>= 30) to green.
- The attribution list: horizontal bars normalized to the largest absolute contribution, red for
risk drivers, green for protective factors, and deliberately grey and de-emphasized for
fee_sensitivity, closing with "The fee itself barely registers, the risk is a broken promise, not price."
Counterfactual view (CounterfactualView.tsx, InterventionCard.tsx)
- A branch diagram: the last four events of the trunk, then the journey splitting into one branch per intervention, each animating in on a spring with a staggered delay, coloured green for the winner and yellow or red otherwise, with a trophy on the recommendation.
- The gauge drains: it initializes at the customer's current churn (so step 2 and step 3 agree), then animates down to the selected branch after 550 ms, and updates immediately on later clicks.
- Three intervention cards, each with the retention percentage, the 95 % band rendered as a
positioned bar, the ROI rank, the contextual reason, the value estimate, and the mandatory
Basis:line rendered verbatim and never softened. - Approve & Execute reveals the action plan (labelled
LLM plannerortemplated), the points to restore, the outreach message, the intervention-node confirmation, and the predicted-versus-observed panel with the "Close the loop" reveal.
Population view (PopulationView.tsx)
- Three stat tiles (churn prevented, repeat calls avoided, value preserved), the retention-lift comparison, the roadmap note, and the merge audit JSON export.
- Gracefully tells you to start the backend if it is unreachable, rather than rendering an empty page.
Customer picker (CustomerPicker.tsx)
- Live roster from the API, sorted so Priya is always first and the rest by event count descending.
- An edge-cases-only toggle that filters to the demonstrative cases.
- The
false_positive_trapis deliberately excluded from the selectable roster, because it is an unresolved session with no golden journey; it is demonstrated as a rejected look-alike inside Priya's identity view instead. - A live-versus-offline indicator in the footer.
API layer (api/client.ts, api/bundle.ts, api/fallback.ts, api/types.ts)
- Typed client with a 12-second timeout (generous enough for a cold backend still warming up) and one automatic retry after 600 ms, so a transient failure self-heals.
loadBundle(gid)fetches identity, journey and recommendation in parallel and returns one bundle.- If the backend is unreachable and the customer is Priya, it returns the static offline seed bundle whose shapes match the live API exactly, so no component knows or cares which source it got. For any other customer it surfaces the error honestly.
useTimeWeaveshows the full-screen loader only when the customer changes; a refresh after an un-merge updates in place with no flash.- Switching customers resets the story to step 1, because a half-told story about a different person is confusing.
A consolidated reference. Each entry gives the formula, where it lives, and why it is that way.
confidence = min(0.99, 0.40 x sim(fingerprint) + 0.20 x sim(geo) + 0.31 x [cookie continuation])
sim(a, b) = 1.0 if a == b
= rapidfuzz.fuzz.ratio(a, b) / 100 if rapidfuzz installed
= difflib.SequenceMatcher(None,a,b).ratio() otherwise
= 0.0 if either side is empty
backend/resolution/matcher.py. A weighted evidence sum, transparent by design so the merge card can
show each term's contribution as a bar. The 0.99 cap prevents ever claiming certainty from
probabilistic evidence. Priya: 0.40 + 0.20 + 0.31 = 0.91. Look-alike trap: about 0.18 + 0.20 + 0 =
0.38.
status = active if conf >= 0.85
= pending_review if 0.55 <= conf < 0.85
= rejected if conf < 0.55
matcher.AUTO_MERGE, matcher.REVIEW_FLOOR, resolver._status_for. High precision on identity, a
human in the middle band, and held-back cases surfaced as a guardrail rather than hidden.
mean_j = (1/n) sum_i X[i][j]
std_j = sqrt( (1/n) sum_i (X[i][j] - mean_j)^2 ) or 1.0 if that is zero
Z[i][j] = (X[i][j] - mean_j) / std_j
logreg.LogisticRegression.fit. Puts every feature on a comparable scale so one learning rate works
across counts, binaries and tenure-in-years. The or 1.0 guard avoids dividing by zero on a constant
column.
sigmoid(z) = 1 / (1 + exp(-z)) for z >= 0 (clamped to 1.0 above z = 700)
= exp(z) / (1 + exp(z)) for z < 0 (clamped to 0.0 below z = -700)
per iteration, for i in 1..n:
lin_i = b + sum_j w_j x Z[i][j]
err_i = sigmoid(lin_i) - y_i
gw_j += err_i x Z[i][j]
gb += err_i
w_j := w_j - lr x ( gw_j / n + l2 x w_j )
b := b - lr x ( gb / n )
lr = 0.3 , iters = 4000 , l2 = 1e-3
backend/intelligence/logreg.py. Full-batch gradient descent on the standard cross-entropy gradient,
with L2 on the weights but not on the intercept (regularizing the intercept would bias the base
rate). The two-branch sigmoid with clamps is the numerically stable form: exp(-z) on a large positive
z overflows, so each branch uses the arithmetic that stays in range. Deterministic because it is
full-batch with a zero initialization, no sampling and no shuffling.
sklearn.linear_model.LogisticRegression(C = 100.0, max_iter = 5000, solver = "lbfgs")
backend/intelligence/sklearn_model.py. A drop-in subclass: it computes the same mean and std, fits on
the same standardized matrix, and writes back w and b in the identical layout, so every
downstream formula is unchanged and only the fitting is delegated. C = 100.0 is light
regularization, chosen so coefficients stay close to the near-unregularized pure-Python fit and
Priya's calibration is preserved.
logit(x) = b + sum_j w_j x z_j where z_j = (x_j - mean_j) / std_j
P(churn) = sigmoid( logit(x) )
contribution_j = ( w_j / std_j ) x x_j
raw_intercept = b - sum_j ( w_j x mean_j / std_j )
identity: sum_j contribution_j + raw_intercept == logit(x)
logreg.raw_contributions. This is algebra, not an approximation: it folds standardization back out so
each contribution is w_raw_j x x_j, the true per-driver effect times its actual value. Standardized
contributions measure an effect relative to the population average, which distorts the ordering and
can make "zero of a bad thing" read as protective. Raw space recovers the generative ordering.
full_logit = raw_intercept + sum_j contribution_j
points_j = ( sigmoid(full_logit) - sigmoid(full_logit - contribution_j) ) x 100
backend/intelligence/churn.py. Answers a question a business audience can act on: "how many points of
predicted churn does this one event account for?" Evaluated in probability space so the sigmoid's
nonlinearity is respected. For a logistic model with no interaction terms this is the exact per-feature
effect, which is why SHAP is not needed.
Display filters: drop features whose value is 0 (except tenure_years), drop positive drivers under
+0.5 points as collinearity noise, and show tenure_years only when it is below -0.5 points. Sorted
descending.
Priya: +26.4 broken callback, +19.6 redemption failure (x2), +13.7 anonymous cancel research, -3.1 tenure 6y protective.
accuracy = ( count of rows where (P(churn) >= 0.5) == bool(y) ) / n
Live value 0.717. Reported openly at /system/status. It is not near-perfect, and it should not
be: the labels are Bernoulli draws, so the Bayes-optimal classifier is also well short of 1.0. A
suspiciously high number here would mean the label leaked.
z = -1.42
+ 0.30 x redemption_failures
+ 0.85 x callback_broken
+ 0.50 x cancel_page_view
+ 0.25 x unresolved_call
+ 0.22 x near_renewal
- 0.04 x tenure_years
+ 0.10 x fee_sensitivity
P(churn | arm) = sigmoid( z - delta(features, arm) )
backend/data/outcomes.py. Weight rationale is in
section 7. Priya: z = 0.76, so P(churn | do nothing) = 0.681,
which is the seed's 68 % target; the independently trained classifier lands at 66.2 %.
delta(do_nothing) = 0.00
delta(fee_discount) = 1.20
delta(senior_callback) = 0.70 + 0.50 x [callback_broken]
+ 0.40 x [redemption_failures > 0]
+ 0.30 x [unresolved_call]
The fee discount is a flat, indiscriminate bribe. The senior callback is conditionally stronger,
helping extra exactly when it repairs a driver the customer actually has. That asymmetry is the
product thesis, encoded in the data-generating process rather than asserted in the UI. Priya:
delta(senior_callback) = 1.90 versus delta(fee_discount) = 1.20.
for each arm in { do_nothing, fee_discount, senior_callback }:
retained_under[arm] = ( rng.random() >= P(churn | arm) )
churned_no_action = not retained_under[do_nothing]
Each customer carries a potential outcome under every intervention, drawn from a seeded RNG. This is what makes the cohort aggregate a genuine observed rate rather than a display-time invention, and it is what the predicted-versus-observed loop reveals.
rate[arm] = AVG( CASE WHEN retained_<arm> THEN 1.0 ELSE 0.0 END )
over all ground_truth rows matching the five binarized flags
n = COUNT(*) of that same set
One DuckDB aggregate, store.cohort_query. Computed in the database rather than in Python because that
is what a columnar engine is for and it keeps the "real analytical query" claim literal.
Priya's cohort: N = 361, do-nothing 33.0 %, fee 59.3 %, senior callback 78.7 %.
z = 1.96
denom = 1 + z^2 / n
center = ( p + z^2 / (2n) ) / denom
margin = ( z x sqrt( p(1-p)/n + z^2/(4n^2) ) ) / denom
band = [ max(0, center - margin) , min(1, center + margin) ]
cohort._wilson. Chosen over the normal (Wald) approximation because Wald misbehaves near 0 and 1 and
can produce bounds outside [0, 1], which is exactly the failure mode a small cohort would trigger.
Wilson is well behaved at the extremes and at small N. Priya's bands, after the projection shift:
do-nothing 29.1 to 38.8, fee 54.9 to 65.0, senior callback 75.0 to 83.4.
LTV = { Platinum: 15000, Gold: 6000, Green: 2500, unknown: 8000 }[tier]
value = retention_rate x LTV - cost
ranking = sort by value descending -> roi_rank, and rank 1 is the winner
cohort.compare. Ranking on retention alone always picks the most expensive option. Ranking on
expected value can correctly choose a cheaper action when the outcome difference is not worth the
money, which is what makes the recommendation defensible commercially and not just statistically.
Priya (Platinum, LTV 15000): senior callback rank 1, fee discount rank 2, do nothing rank 3.
lift_vs_nothing = retention - do_nothing_retention
Priya: fee +26.3 pts, senior callback +45.7 pts.
clearly_better(item) = exists o != item, o != do_nothing, such that o.retention_lo > item.retention_hi
is_obvious_but_weak = clearly_better(fee_discount) and not fee_discount.is_winner
Requires a real, non-overlapping gap in the confidence bands, not just a lower point estimate, so
the badge is never awarded on noise. The not is_winner clause makes the "recommended" and
"obvious, weak" labels mutually exclusive. Decided per customer at request time, so it stays true when
an analyst clicks someone else.
if winner is not the highest-retention option
and winner.retention_hi >= top_retention.retention_lo:
append "(Statistically tied with X, their 95 % confidence bands overlap,
so it is recommended on cost/value.)"
Prevents the system from quietly presenting a cost-driven choice as an outcome-driven one.
baseline_retention = 100 - live_churn_risk
shift = baseline_retention - cohort_do_nothing_retention
projected = clamp( observed_cohort_rate + shift , 0 , 99.9 )
projected_churn = 100 - projected
band = [ clamp(lo + shift, 0, 100) , clamp(hi + shift, 0, 100) ]
Anchors the cohort's observed lift onto the individual's live risk. The lift, the ranking, the
winner and the band widths are unaffected; only the level moves. This makes the step-2 gauge and the
step-3 do-nothing branch agree, and makes an un-merge propagate. Priya: shift = 33.8 - 33.0 = +0.8,
so 33.0/59.3/78.7 becomes 33.8/60.1/79.5. Flagged in the response as personalized: true with a
rewritten basis string.
MIN_COHORT = 25
relax order = [ cancel_page_view , unresolved_call , near_renewal ]
while cohort_n < MIN_COHORT and a flag remains to relax:
set that flag to None # drop it from the SQL predicate
re-run the cohort query
Runs before the projection above, on the live feature flags. Guarantees a rate is never quoted
from a near-empty cohort, which is exactly the failure an un-merge could otherwise cause. The churn
drivers stay constrained so the cohort remains meaningful, and cohort_widened reports what was
dropped. Priya un-merged: N would collapse, so cancel_page_view is relaxed and N = 362.
requires_approval = intervention.cost > 250
auto_executed = not requires_approval
AUTO_EXECUTE_COST_LIMIT in backend/main.py. Autonomy scales with blast radius. $600 senior
callback: approval required. $200 fee credit and $0 do-nothing: auto-executable.
retained_delta = ( senior_callback_rate - do_nothing_rate ) x N
churn_prevented = round(retained_delta)
calls_avoided_est = round(retained_delta x 2.3)
value_preserved_est = round(retained_delta x 15000)
Both multipliers are illustrative and labelled as such. N = 361, delta = (0.787 - 0.330) x 361 = 165.0, so 165 retained, 380 calls avoided, $2,475,196 preserved.
days = (latest_event.date - event.date).days
label = "T-{days}d HH:MM" if days > 0 else "T-0 HH:MM"
Anchored to the journey's own latest event rather than the wall clock, so the demo reads identically at any future date.
START = 135 degrees , SWEEP = 270 degrees
circumference = 2 x pi x r
arcLen = (270 / 360) x circumference
progress = clamp(value, 0, 100) / 100
strokeDashoffset = arcLen x (1 - progress)
colour = red #E0304F if value >= 55
= orange #E08A00 if value >= 40
= yellow #C9A100 if value >= 30
= green #12A66B otherwise
frontend/src/components/ChurnGauge.tsx. A dash-offset arc rather than an animated path, so the sweep
is a single cheap CSS-driven property. The number tweens over 1.4 s on ease [0.22, 1, 0.36, 1].
dx = max(28, (x2 - x1) / 2)
path = M x1 y1 C (x1+dx) y1 , (x2-dx) y2 , x2 y2
frontend/src/components/Timeline.tsx. A cubic bezier from one chip's right edge to the next chip's
left edge, with the control-point offset floored at 28 px so vertically-adjacent chips still get a
readable curve instead of a kink. Coordinates come from live getBoundingClientRect measurements under
a ResizeObserver, so the connector stays correct on resize.
All four are authored deliberately, and all four behave correctly in the running system.
| Case | What it is | Live behaviour |
|---|---|---|
priya |
the hero journey, nine events across four channels | anonymous session merges at 0.91, risk 66.2 %, senior callback wins |
false_positive_trap (golden-trap-01, "Anonymous session #4471") |
an anonymous session with a similar device fingerprint (fp-visitor-3c9 versus Priya's fp-priya-e91) in the same metro (San Francisco, CA), but a different person and a different ECID |
scores 0.38, status rejected, not merged. Surfaced inside Priya's identity view as a guardrail panel explaining that the decisive evidence, cookie continuation, is absent. Excluded from the selectable roster because it has no golden journey |
shared_household (golden-hh-0 Ada Okafor, golden-hh-1 Chidi Okafor) |
two people on one tokenized PAN (tok_household_okafor) |
the deterministic card link is downgraded into a household merge at 0.71 · pending_review, with the explanation "same card token across 2 accounts, but the cardmember names differ, likely an authorized user / shared household, NOT a confident individual-identity merge. A human steward decides." Lands in the review queue |
serial_caller (golden-9001, Marcus Lee) |
5 to 7 unresolved calls mid-cycle, no renewal pressure | resolves normally, but produces a different journey shape, a different cohort (serial_caller@midcycle, N = 91) and therefore a different recommendation, which proves the pipeline is not hardcoded to the hero |
FastAPI, version 0.5.0, interactive docs at http://localhost:8000/docs. CORS is a wildcard, which
is safe here because there is no auth and no cookies, and it avoids the classic "Vite picked port 5174"
or "I opened the LAN URL" failure that shows up in the UI as Failed to fetch.
| Method | Path | Returns |
|---|---|---|
| GET | /health |
{status, phase} |
| GET | / |
service name, version, Priya's golden id, the endpoint index |
| GET | /system/status |
active matcher backend, graph backend, churn backend, training accuracy, whether the LLM is enabled |
| Method | Path | Notes |
|---|---|---|
| GET | /customers?edge_cases_only= |
roster, Priya first, then by event count descending |
| GET | /events?golden_id= |
the raw canonical events for one identity |
| GET | /journey/{gid} |
profile plus the tagged timeline, with un-merged sources' events removed |
| Method | Path | Notes |
|---|---|---|
| GET | /identity/{gid} |
per-channel streams, every merge record, the primary fuzzy merge |
| GET | /graph/{gid} |
the identity plus journey subgraph as nodes and edges |
| GET | /graph-stats |
node-type and edge-type counts across the population |
| GET | /merges |
every merge plus the append-only history (this is the audit export) |
| GET | /review-queue |
pending low-confidence merges |
| POST | /unmerge |
{merge_id, by}, reversible, audited |
| POST | /remerge |
{merge_id, by}, restores to active or pending_review by confidence |
| POST | /review/{merge_id} |
{decision: "approve" | "reject", by} |
| Method | Path | Notes |
|---|---|---|
| GET | /churn/{gid} |
risk, probability, attribution, model backend, training accuracy |
| GET | /interventions/{gid} |
the full cohort comparison with bands, ROI ranks and the winner |
| GET | /recommendation/{gid} |
one call for the whole counterfactual view: churn plus cohort plus plan plus uplift |
| GET | /uplift/{gid} |
optional T-learner individual uplift estimate, labelled not-causal |
| GET | /cost/{gid} |
per-journey cost to serve, split by channel, with the avoidable share |
| Method | Path | Notes |
|---|---|---|
| POST | /approve |
{golden_id, intervention_id, approved_by}, applies the cost guardrail, adds the graph node, returns predicted versus observed, and logs the outcome to the closed loop |
| Method | Path | Notes |
|---|---|---|
| GET | /population/fix-the-flow |
the cohort-wide business case |
| GET | /population/stats |
customer and event counts, per-channel and per-cohort breakdowns, per-cohort retention rates by intervention |
| GET | /population/emerging-friction |
near-real-time spike alerts per friction flow (z-score and rate-ratio) |
| GET | /population/cohort-diff |
prior versus recent friction mix, and which flow drove the shift |
| GET | /population/cost-to-serve |
population cost split by channel and by cohort |
| GET | /monitoring/merge-drift |
live resolver snapshot plus a PSI drift trend on match scores |
| GET | /resolution/model |
the learned Fellegi-Sunter weights plus two worked examples |
| GET | /learning/loop |
observed retention per intervention, with the Wilson band tightening |
| POST | /learning/simulate?n= |
log N observed outcomes so the loop's tightening is demonstrable |
| Method | Path | Notes |
|---|---|---|
| GET | /admin/db |
database path, persistence flag, row counts |
| POST | /admin/customer |
add a new customer, then rebuild resolution and retrain |
| POST | /admin/events |
append events to an existing customer, recompute and retrain |
| POST | /admin/regenerate?n_seed= |
wipe and regenerate the whole population |
Adding your own customer:
curl -X POST http://localhost:8000/admin/customer -H "Content-Type: application/json" -d '{
"profile": {"golden_id":"golden-demo-9","name":"Demo Customer","tier":"Platinum",
"tenure_years":7,"renewal_in_days":5,"fee_sensitivity":0,
"cohort_key":"redemption_failed+callback_broken@renewal"},
"raw_events": [
{"_source":"xdm","_id":"d9-app-1","timestamp":"2026-07-11T10:00:00Z",
"eventType":"application.redemption",
"identityMap":{"account":[{"id":"acct-d9","authenticatedState":"authenticated"}]},
"_attributes":{"points":60000},"_outcome":"redemption_failed"},
{"_source":"system","_id":"d9-sla","_customer_ref":"acct-d9",
"timestamp":"2026-07-15T15:00:00Z","eventType":"system.sla_breach",
"_attributes":{"breached":true},"_outcome":"callback_broken"}
]
}'raw_events use exactly the source schemas the generator emits (xdm / ctr / iso8583 / system).
Three tables in an embedded DuckDB file. Open backend/timeweave.duckdb in any DuckDB client and
query them directly.
CREATE TABLE events (
event_id TEXT, customer_ref TEXT, resolved_golden_id TEXT,
channel TEXT, event_type TEXT, timestamp TIMESTAMP,
authenticated BOOLEAN, auth_state TEXT, outcome TEXT,
source_schema TEXT, attributes JSON
);
CREATE TABLE ground_truth (
golden_id TEXT, cohort_key TEXT, near_renewal BOOLEAN,
churned_no_action BOOLEAN, retained_do_nothing BOOLEAN,
retained_fee_discount BOOLEAN, retained_senior_callback BOOLEAN,
redemption_failures INTEGER, callback_broken BOOLEAN,
cancel_page_view BOOLEAN, unresolved_call BOOLEAN,
tenure_years INTEGER, fee_sensitivity BOOLEAN
);
CREATE TABLE profiles (golden_id TEXT, profile JSON);ground_truth carries both the drawn potential outcomes and the binarized features, which is what lets
the cohort query be a single indexed aggregate instead of a join.
DuckDB is single-writer. Run one backend instance against a given database file at a time. This is why
scripts/seed.pyuses:memory:.
| Variable | Where | Default | Effect |
|---|---|---|---|
TIMEWEAVE_DB |
backend | backend/timeweave.duckdb |
database file path; set to :memory: for an ephemeral run |
ANTHROPIC_API_KEY |
backend | unset | enables the live LLM planner path; without it the planner uses deterministic templates |
VITE_API_BASE |
frontend | http://localhost:8000 |
where the UI looks for the API (copy frontend/.env.example to .env.local) |
Optional Python accelerators, all commented out in backend/requirements.txt and all safe to install:
rapidfuzz==3.10.1, networkx==3.4.2, scikit-learn==1.5.2, anthropic>=0.39.0.
Every optional dependency has a zero-dependency fallback with identical behaviour, and the system
auto-upgrades if the library is importable. GET /system/status reports which path is live, and the UI
header shows it.
| Capability | Accelerator | Standard-library fallback |
|---|---|---|
| Fuzzy string similarity | rapidfuzz |
difflib.SequenceMatcher |
| Identity and journey graph | networkx |
equivalent in-memory structure |
| Churn classifier | scikit-learn (lbfgs) |
pure-Python logistic regression (gradient descent, L2) |
| Intervention planner | Anthropic LLM | deterministic templates |
| The whole frontend | live backend | static Priya seed bundle in frontend/src/data/priya.ts |
Nothing about the demo depends on the network. There is no external service, no hosted database and no required API key.
- Never claim causality. Recommendations are observed retention rates within a cohort of similar
journeys, labelled correlationally everywhere they appear, in the UI text and in the API payloads
alike. Every response carries a
roadmap_notestating that validated causal uplift requires controlled experimentation. This survives an ML-literate reviewer, which a claimed uplift number on synthetic data would not. - Ground every number in the generated data. Nothing is invented at display time. One generative world model produces the outcomes; the classifier learns from those outcomes; the cohort aggregates those outcomes. The seed's target numbers are hit because the weights were calibrated, not because the display was hardcoded.
- Explain every automated decision. Every merge carries a confidence, a weighted matched-feature breakdown, a method and a plain-English explanation. Every recommendation carries a contextual reason and a basis line.
- Make everything reversible and audited. Merges un-merge and re-merge, the DuckDB rows are never mutated by either, and every action is appended to a history that exports as JSON.
- Keep a human in the loop where it matters. Low-confidence merges go to a steward, and expensive actions require a recorded approval against an actual cost threshold.
- Show the uncertainty. Wilson bands on every cohort number, honest disclosure of statistical ties, and a training accuracy of 0.717 reported openly rather than hidden.
- Be deterministic. Fixed seeds, no wall-clock, relative times anchored to the data. The same inputs give the same outputs, so a number quoted today is the number tomorrow.
- Never let the demo hard-fail. A stdlib fallback for every accelerator, a static seed bundle for the hero, optimistic UI updates, a generous timeout with one retry, and views that explain what to start rather than rendering blank.
- Privacy by construction. Synthetic data only, tokenized card identifiers, masked display of
phones and tokens, internal
_-prefixed attributes stripped before the UI sees them, and never any raw PII in an LLM prompt.
amex-hackathon/
├─ frontend/ React 18 + Vite 5 + TS + Tailwind 3 + react-flow + framer-motion
│ ├─ src/api/
│ │ ├─ client.ts typed client, 12s timeout, one retry
│ │ ├─ bundle.ts parallel loader for one customer's full bundle
│ │ ├─ fallback.ts offline static bundle for Priya + fallback roster
│ │ └─ types.ts response types shared with the components
│ ├─ src/hooks/useTimeWeave.ts roster, system status, review queue, selected bundle
│ ├─ src/components/
│ │ ├─ CustomerPicker.tsx left rail, edge-case filter, live/offline badge
│ │ ├─ IdentityView.tsx react-flow identity graph
│ │ ├─ MergeCard.tsx confidence, matched features, un-merge / re-merge
│ │ ├─ ReviewQueue.tsx steward approve / reject
│ │ ├─ JourneyView.tsx timeline + gauge + attribution
│ │ ├─ Timeline.tsx cross-channel swim lanes + SVG connector + detail strip
│ │ ├─ ChurnGauge.tsx 270-degree animated arc gauge
│ │ ├─ AttributionList.tsx per-driver bars, fee de-emphasized
│ │ ├─ CounterfactualView.tsx branch diagram, gauge drain, approve, close the loop, uplift
│ │ ├─ InterventionCard.tsx retention, Wilson band, ROI rank, mandatory basis
│ │ ├─ PopulationView.tsx fix-the-flow stats + audit export + the six v2 panels
│ │ ├─ EmergingFrictionPanel.tsx near-real-time spike alerts (also exports PanelShell)
│ │ ├─ CohortDiffPanel.tsx prior vs recent friction mix
│ │ ├─ CostToServePanel.tsx population cost by channel and cohort
│ │ ├─ ClosedLoopPanel.tsx observed outcomes, band tightening, simulate
│ │ ├─ ResolutionModelPanel.tsx learned linkage weights + worked examples
│ │ └─ MergeDriftPanel.tsx resolver snapshot + PSI drift trend
│ ├─ src/data/priya.ts static Phase-1 seed (offline fallback source)
│ └─ src/ui.ts channel / outcome / flag visual metadata
├─ backend/ FastAPI + embedded persistent DuckDB, fully offline
│ ├─ main.py every endpoint, the cost guardrail, wiring
│ ├─ timeweave.duckdb persisted data file (gitignored, created on first run)
│ ├─ schemas/
│ │ ├─ canonical.py CanonicalEvent, Channel, GroundTruth
│ │ └─ api.py request / response models
│ ├─ data/
│ │ ├─ generator.py synthetic population in real source schemas
│ │ ├─ seed_priya.py the authored hero journey
│ │ ├─ features.py THE single feature definition
│ │ ├─ outcomes.py the generative world model + potential-outcome draws
│ │ └─ store.py DuckDB store, persistence, thread safety, cohort queries
│ ├─ ingestion/normalize.py XDM / CTR / ISO 8583 / system -> canonical
│ ├─ resolution/
│ │ ├─ matcher.py weighted fuzzy matcher + thresholds (swappable interface)
│ │ ├─ fellegi_sunter.py learned log2(m/u) linkage weights, Splink-shaped
│ │ └─ resolver.py merges, review queue, un-merge, audit history
│ ├─ graph/graph.py identity + journey graph (networkx optional)
│ ├─ intelligence/
│ │ ├─ logreg.py pure-Python logistic regression
│ │ ├─ sklearn_model.py scikit-learn backend, same interface
│ │ ├─ churn.py scoring + leave-one-out attribution
│ │ ├─ cohort.py cohort comparison, back-off, Wilson bands, ROI, projection
│ │ ├─ journey.py timeline assembly + friction flags
│ │ ├─ monitoring.py emerging friction, cohort diffs, merge-drift (PSI)
│ │ ├─ learning.py closed loop: logged outcomes, tightening Wilson band
│ │ ├─ cost.py per-journey and population cost to serve
│ │ └─ uplift.py optional T-learner, labelled not-causal
│ └─ llm/planner.py optional planner / explainer, degrades to templates
├─ scripts/
│ ├─ run-backend.ps1 / .sh one-command backend (creates venv on first run)
│ ├─ run-frontend.ps1 / .sh one-command frontend
│ └─ seed.py end-to-end pipeline check with no server
├─ seed.md · strategy.md the project spec
├─ amex-hackathon-winning-strategy.md
├─ timeweave-coverage-audit.md honest vision-versus-code audit
├─ timeweave-journey-walkthrough.md
└─ Implementztion.md
Everything below was developed on the version2 branch and is now merged into master and
running. It closes the gaps called out in timeweave-coverage-audit.md. Backend version is
0.5.0.
Every one of these is exercised by the verification suite: the hero figures are unchanged, all seven subsystems respond, and an un-merge propagates through both the cohort matching and the projection.
backend/resolution/fellegi_sunter.py, GET /resolution/model, UI ResolutionModelPanel.tsx
The current matcher uses hand-set weights (0.40 / 0.20 / 0.31). This replacement does not hardcode
them, it learns them, the Fellegi-Sunter way, from match versus non-match agreement rates. For each
comparison field k:
m_k = P( fields agree | the pair is a true match )
u_k = P( fields agree | the pair is a non-match )
w_agree_k = log2( m_k / u_k ) # evidence FOR a match, in bits
w_disagree_k = log2( (1 - m_k) / (1 - u_k) ) # evidence AGAINST, in bits
total_bits = prior_logodds + sum_k ( w_agree_k if agrees_k else w_disagree_k )
P(match) = 1 / ( 1 + 2^(-total_bits) )
m and u are estimated by counting over 10,000 labelled pairs generated at a fixed seed
(P(match) = 0.20 among candidate pairs), with agreement rates that are realistic for device and
cookie linkage: for a true match fingerprint 0.96 / geo 0.82 / cookie 0.88, for a non-match
0.02 / 0.22 / 0.03. An epsilon of 1e-4 clamps the estimates away from 0 and 1 so the logs stay
finite.
Why this matters: it is exactly the model Splink fits, via EM on unlabelled data or on
clerical-review labels. Estimating m and u by counting makes the whole procedure transparent and keeps
it fully offline. Swapping real Splink in means replacing _learn() and nothing else, because the
Matcher interface is unchanged. The honest caveat is stated in the module and in the API response:
the labelled pairs are synthetic, and it is the mechanism that is production-real.
It ships inspectable but not active by default (TIMEWEAVE_MATCHER=fs activates it), so the hero
merge stays at the familiar 0.91. /resolution/model returns the learned per-field m, u and bit
weights, the prior, and two worked examples: Priya's session and the look-alike trap, so a reviewer can
watch the learned weights actually make the decision.
backend/intelligence/learning.py, a new approvals table, GET /learning/loop,
POST /learning/simulate?n=, UI ClosedLoopPanel.tsx
Today's predicted-versus-observed is a single reveal. Version 2 persists every approved action with its observed outcome and re-derives the estimate from the accumulating evidence:
observed_rate_k = ( retained observations in the first k ) / k
halfwidth_k = ( wilson_hi(rate_k, k) - wilson_lo(rate_k, k) ) / 2
The panel plots halfwidth_k shrinking as k grows, converging on the cohort prior. Why it is
framed this way: no model magically "improves". What is shown is the honest, non-hand-wavy version,
an observed rate and its Wilson interval visibly tightening with N, which is the seed of an RL
policy without pretending to be one. On this data the observations are the customers' synthetic ground
truth, so the live estimate converges to the cohort prior, and that consistency is precisely the point:
in production the identical code path consumes real observed outcomes.
POST /learning/simulate logs a batch of observed outcomes across hero-cohort customers, clearly
labelled source: "simulated", so the tightening is demonstrable without hand-clicking approve sixty
times.
backend/intelligence/monitoring.py
1. Emerging-friction detection (GET /population/emerging-friction,
UI EmergingFrictionPanel.tsx). Buckets the population's events into daily windows and flags any
friction flow whose most recent window spikes above its trailing baseline:
b_mean, b_std = mean and std of the flow's count over all baseline windows
z = (recent_count - b_mean) / b_std
ratio = recent_count / b_mean
alert fires when recent_count >= 3 AND ( z >= 2.0 OR ratio >= 1.5 )
severity = "high" if ( z >= 4 or ratio >= 3 ) else "elevated"
trend = "rising" if recent/baseline >= 1.25 , "falling" if <= 0.8 , else "flat"
Two deliberate refinements. Partial windows are dropped: a trailing bucket holding less than
max(5, 0.15 x median daily volume) is still filling and would otherwise trigger a false "declining"
read or a false alert, and never alerting on an incomplete window is standard practice. Near-zero
baselines are phrased honestly: when the baseline average is below 1.0, a rate ratio of "2070x" is
technically true but reads as noise, so the message says "up from effectively zero" instead, and the
z-score is capped at 99.9 for display. Fully real, computed from the actual event timestamps in DuckDB.
2. Cohort diffs over time (GET /population/cohort-diff, UI CohortDiffPanel.tsx). Splits the
event stream into a prior half and a recent half and diffs the friction mix:
share(flow, period) = count(flow, period) / total_friction(period)
delta_share(flow) = share(flow, recent) - share(flow, prior)
Sorted to surface the biggest riser and the biggest faller, with a headline such as "the failure is moving downstream from self-service into service recovery". Shares rather than raw counts, because a mix shift is the interesting signal and raw counts conflate it with volume. Fully real.
3. Merge-drift and resolution-quality monitoring (GET /monitoring/merge-drift,
UI MergeDriftPanel.tsx). The live snapshot is real, read straight from the resolver: total merges, a
breakdown by status and by method, mean fuzzy confidence, auto-merge rate and review rate. The weekly
trend runs PSI, the Population Stability Index, the industry-standard score-drift test, over the
fuzzy match-score distribution binned at [<0.55, 0.55-0.70, 0.70-0.85, 0.85-0.95, >=0.95]:
PSI = sum_bins ( current_prop - baseline_prop ) x ln( current_prop / baseline_prop )
PSI < 0.10 -> stable
PSI >= 0.10 -> watch
PSI >= 0.20 -> significant drift
Honest labelling, stated in the response note: the detection is production-real, only the
weekly history is deterministic synthetic telemetry (fixed seed 2026, stable baseline, mass drifting
toward low confidence over the final three weeks), because a five-day demo snapshot cannot hold months
of resolution logs. In production the monitor reads the resolution audit log instead. The alert
explains what drift means operationally: the auto-merge rate falling while the review queue grows means
the resolver is seeing systematically weaker evidence, so recalibrate the thresholds or retrain the
matcher before precision degrades.
backend/intelligence/cost.py, GET /cost/{gid}, GET /population/cost-to-serve,
UI CostToServePanel.tsx plus a strip in the journey view
Assigns an illustrative unit cost per event by channel, sums it per journey and aggregates it across the population:
call : duration_min (default 9) x $5.50 per minute
app / web : $0.12 , plus $1.50 if the outcome is redemption_failed
in_person : lounge_checkin $25.00 · travel_booking $10.00 · pos $0.02
system : $0.00
avoidable_cost = sum of ( all call costs ) + ( failed self-service costs )
Why this angle: Amex's moat is high-touch service, and its most watched cost is the premium human call minute, which dwarfs a self-service page view by more than two orders of magnitude. This turns the counterfactual into dollars in a second, independent way: the same broken flow that drives churn also drives expensive repeat calls, so fixing it saves retention and cost. The population view then surfaces the most expensive cohorts, which turn out to be exactly the repeat-call and broken-flow ones. Costs are labelled illustrative everywhere; plug in real per-channel unit costs and the arithmetic is unchanged.
backend/intelligence/uplift.py, GET /uplift/{gid}, surfaced as a dashed optional card in the
counterfactual view
A T-learner: one outcome model per treatment arm.
mu_0(x) = P( retain | do_nothing , x )
mu_t(x) = P( retain | arm t , x )
uplift_t(x) = ( mu_t(x) - mu_0(x) ) x 100 # estimated individual treatment effect, in points
It is trained on the per-arm potential outcomes every synthetic customer already carries, using the same backend selection as the churn model, so it also runs fully offline.
This ships deliberately secondary and deliberately caveated. The original vision's causal-uplift engine was cut precisely because, on synthetic data, a model that claims causal uplift is circular: the ground truth is drawn from a known generative model, so any learner just recovers the constant that was injected. Version 2 includes the T-learner to demonstrate the technique and to add an individual-level, heterogeneous estimate that a cohort average cannot express, while stating the caveat everywhere it surfaces. The UI card makes the circularity the lesson: it shows the T-learner uplift beside the cohort lift and points out that the close agreement is the point, because on synthetic data the learner recovers the same generative signal and therefore is not independent evidence. Because the arms are drawn independently per customer (an effectively randomized synthetic experiment), the ITE matches the cohort difference in expectation, which is exactly why it is honest here and why validating it in the wild would require an RCT.
backend/intelligence/cohort.py
An exact-match cohort can get too small to be meaningful, and a lone customer reads as a degenerate 0 % or 100 %. Version 2 adds cohort widening:
MIN_COHORT = 25
relax order = [ cancel_page_view , unresolved_call , near_renewal ]
while cohort_n < 25 and flags remain to relax:
drop the next least-defining flag from the WHERE clause
re-run the cohort query
The churn drivers (redemption failure and broken callback) stay constrained; only the least-defining
flags relax. store.cohort_query gains a None sentinel meaning "do not constrain on this dimension",
so the SQL predicate is assembled dynamically. The response reports cohort_widened: [...] and the
label appends "(widened to a comparable cohort for a stable estimate)", and the UI shows exactly which
dimensions were dropped.
How this composes with the projection. The version2 branch originally replaced the
personalized projection with this back-off, because matching the cohort on live features already makes
an un-merge move the analysis. Master had meanwhile solved the same problem the other way, with the
projection. The merge keeps both, applied in order (see CohortEngine.compare):
- Binarize the live features and select the matched cohort, so an un-merge changes the cohort.
- Widen that cohort if it is under
MIN_COHORT, so rates can never degenerate. - Project the cohort's observed lift onto the member by anchoring do-nothing to their own risk.
These do not double-count, because the shift in step 3 is always measured relative to whichever cohort steps 1 and 2 selected. Two properties follow, and both are covered by the verification suite:
- With nothing un-merged, the baseline is already ≈ the cohort's do-nothing rate, so the shift is ≈ 0 and the hero's figures are exactly preserved (Priya: cohort 361, shift +0.8, giving 33.8 / 60.1 / 79.5 as every submitted artifact states).
- After un-merging Priya's anonymous session, both engage: her churn falls 66.2 % to 52.5 %, the
cohort widens by dropping
cancel_page_view(N = 362), and do-nothing still projects to exactly her live churn, so step 2 and step 3 of the story stay consistent.
- Un-merge and re-merge now propagate all the way through churn, attribution, cohort and uplift,
because every one of them recomputes from
store.live_features_for. Detaching Priya's anonymous session actually lowers her risk instead of leaving the downstream analysis stale. - The identity graph re-fits its viewport when the merge set changes, which fixes a case where the canvas could be left collapsed and render blank after an un-merge.
/system/statusgainsactive_matcher, so the swap is visible./journey/{gid}gains an embeddedcost_to_serveblock./recommendation/{gid}gains an embeddedupliftblock, so the counterfactual view still loads in one call.- The population view grows six new panels under three labelled dividers: Population observability (emerging friction, cohort diffs, cost to serve), Closed-loop learning, and Resolution quality (learned linkage weights, merge drift). Every panel fails silently and independently, rendering nothing if its endpoint is unavailable, so an older backend cannot break the page.
items/gains the submission deck, the round-one documents, and screenshots of all four views.
Still open after version2, in the order they would pay off:
| Item | Why it is next |
|---|---|
| Agent copilot at point-of-contact | the most Amex-native feature and the least developed: surface the stitched journey, the risk drivers and the recommended action at the moment an agent picks up the call |
| A policy that changes its recommendation | version 2 logs outcomes and tightens the estimate; the next step is a bandit or RL policy that actually reallocates spend as evidence accrues |
| Natural-language analyst console | English to a DuckDB or graph query, so an analyst can ask a question the UI did not anticipate |
| Real Splink swap-in | the interface and the learned-weights model are already in place; this replaces _learn() with Splink's EM on real labelled pairs |
| Transformer or LLM tie-breaker for genuinely hard merges | the current LLM is a planner; a dedicated tie-breaker would target the 0.55 to 0.85 review band specifically |
| Multi-agent intervention debate | competing agents argue for different actions before a human decides |
| Real causal inference and controlled experimentation | the honest path to the causal claim the product deliberately refuses to make today |
Deliberately not on the near-term roadmap (the right call under time pressure, kept as a clean architecture story rather than a fragile half-build): real streaming ingestion (Kafka or Redpanda), privacy-preserving record linkage and clean rooms, consent and data-residency toggles, federated learning, differential privacy, a multi-node graph database, and real channel integrations.
For completeness, and because every one of these was a judgement call worth defending:
| Vision | Shipped | Why |
|---|---|---|
| Causal uplift engine | honest cohort outcome comparison | on synthetic data a claimed causal number is circular; correlational plus labelled survives scrutiny and keeps almost all of the visual impact |
| Kafka / streaming | DuckDB batch, embedded | always-offline beats a fragile live pipeline in a demo |
| Neo4j | NetworkX, with a stdlib fallback | no server, a small graph, zero demo-day risk |
| Splink | rapidfuzz weighted sum by default | fewer moving parts; the interface is swap-ready and version2 ships the learned model behind it |
| Transformer sequence model | logistic classifier | deterministic, fast, and explainable on stage; the attribution is exact rather than approximated |
| Full agent autonomy | human approval plus a real cost threshold | autonomy should scale with blast radius, and the threshold is actual logic, not a disclaimer |
| Production privacy hardening | synthetic data, tokenized identifiers, masked display, no PII in prompts | demo-grade privacy that is genuinely sufficient to tell the story honestly |