Release v1.18.0

.gitleaks.toml

@@ -5,6 +5,14 @@ useDefault = true
 [allowlist]
 description = "Accepted false positives"
 regexTarget = "line"
+# Archived third-party source pages in the research record (checksummed, reviewed on archive):
+# hex example IDs in the p2pool.observer API docs and page-chrome tokens in saved GitHub HTML
+# pattern-match crypto-token rules. The archives must stay byte-identical to their recorded
+# checksums, so the redaction that would normally fix a finding is impossible by design.
+paths = [
+  # both path forms exist in history: pre-move (research/...) and current (docs/research/...)
+  '''(docs/)?research/xvb-delivery-study/data/sources/.*''',
+]
 # curl auth assembled from shell ENV VARS (e.g. `-u "${USER:-}:${PASS:-wallet}"`) is not a hardcoded
 # secret — the upstream `curl-auth-user` rule can't distinguish `${VAR}` from a literal credential.
 # Both sides must be `${VAR:-default}` expansions (default optional), so a real `-u "admin:hunter2"`

.markdownlint-cli2.jsonc

@@ -16,6 +16,7 @@
     "**/node_modules/**",
     "**/.venv/**",
     "build/dashboard/mining_dashboard/client/tari/generated/**",
-    "docs/dev/test-inventory.md" // generated by `make test-inventory`; not hand-edited
+    "docs/dev/test-inventory.md", // generated by `make test-inventory`; not hand-edited
+    "docs/research/**" // verbatim research records (numbered audit trails, quoted material) — reviewed as research, not house prose
   ]
 }

CHANGELOG.md

@@ -9,6 +9,49 @@ Pithead ships as **one product, one version** — the version lives in the top-l
 [`VERSION`](VERSION) file and every released image is tagged with it. Releases are cut
 per the process in [`docs/dev/releasing.md`](docs/dev/releasing.md).
 
+## [1.18.0] - 2026-08-13
+
+### Added
+
+- **The XvB calculator is a decision table with measured estimates (#900).** One row per tier:
+  the odds of a win per day at your hashrate, the donation cost, what XvB's published bonus
+  figure implies, and a study estimate that tempers the published figure by what winners
+  measurably receive on-chain — 33% of face value (95% CI 28–39%), from a 25-round delivery
+  study run against this stack's own wallet and the public winners feed. The yearly net shows
+  as a band, and a tier is coloured profitable or unprofitable only when the whole band agrees.
+  Once the dashboard has confirmed enough local wins, your own measured realization replaces
+  the study band. The expected-vs-actual card's XvB expectation is tempered the same way.
+- **The dashboard stands down XvB when it is disabled.** The hero raffle KPIs, the Overview
+  tiles, and the stats panel disappear instead of showing dashes. When XvB is on, the win
+  forecast follows the tier the donation actually targets, not only the tier already held.
+- **The XvB delivery study ships in the repo
+  ([docs/research/xvb-delivery-study](docs/research/xvb-delivery-study/PAPER.md)).** The full
+  paper — methods, per-round data, figures, analysis scripts, and checksummed source archives —
+  documents how the measured band was derived and how to reproduce it.
+
+### Fixed
+
+- **The donation cap holds during catch-up (#898).** After an under-tier hour, catch-up aimed
+  at the tier target even when the configured donation cap could never reach it, so the
+  algorithm donated at the cap ceiling indefinitely. Catch-up now aims at the achievable
+  ceiling — the lesser of the tier target and the cap times stable hashrate — so a cap set
+  below the tier target holds.
+- **The e2e upgrade test seeds from the live release bundle (#880),** not the canonical
+  checkout, so the pre-cut bench run exercises the same starting state an operator upgrades
+  from.
+
+### Changed
+
+- **The repository is reorganized (#907).** The research record lives under `docs/research/`,
+  root-level files are pruned or moved to their subject directories, and the dev docs got an
+  accuracy pass — release branch mechanics, withdrawing a bad release, the operator-run
+  pre-cut e2e gate, service and config-key counts.
+
+### Dependencies
+
+- Python (dashboard): diff-cover 10.5.0, hypothesis 6.165.2, ruff refresh. Docker: refreshed
+  the pinned `ubuntu` base digest in the monero, p2pool, and xmrig-proxy images.
+
 ## [1.17.0] - 2026-08-02
 
 ### Added

CONTRIBUTING.md

@@ -29,7 +29,8 @@ runs `ruff` (plus a few hygiene hooks) on your changed files. If you change depe
 ## Development workflow
 
 1. Fork the repo and create a branch off `develop` (the integration branch; `main` holds released
-   commits only).
+   commits only — how releases move `develop` → `main` and back is in
+   [Releasing › Branch mechanics](docs/dev/releasing.md#branch-mechanics)).
 2. Make your change. Keep it focused: one logical change per PR.
 3. Run the full test suite locally:
 

Makefile

@@ -44,7 +44,7 @@ lint: lint-sh lint-py lint-js lint-yaml lint-md lint-docs-voice lint-operator-st
 lint-sh: ## shellcheck + shfmt over the CLI, build/* container scripts, release + test scripts
 	shellcheck --severity=warning pithead pithead-completion.bash scripts/*.sh build/*/*.sh tests/stack/run.sh tests/stack/test_compose.sh \
 		tests/inventory.sh tests/integration/*.sh tests/integration/mini-stack/*.sh
-	shfmt -i 4 -d pithead pithead-completion.bash $(shell git ls-files '*.sh')
+	shfmt -i 4 -d pithead pithead-completion.bash $(shell git ls-files '*.sh' | grep -v '^docs/research/')
 
 lint-py: ## ruff lint + format check on all repo Python (ruff runs via uv from the locked dev extra)
 	uv run --locked --project build/dashboard --extra dev ruff check .

README.md

@@ -46,7 +46,7 @@ a Tor daemon. The `pithead` script renders config, provisions Tor, and drives do
   [The Dashboard](docs/dashboard.md).
 - ⚙️ **One config, tuned to your setup.** A local or remote Monero node, pruned or full; the P2Pool
   tier (`main`, `mini`, or `nano`); XvB donation strategy; per-worker power and API settings; four
-  alert channels; timezone, memory limits, and every privacy toggle — around 94 keys across 13
+  alert channels; timezone, memory limits, and every privacy toggle — around 113 keys across 13
   sections, all in one `config.json` and validated on every `apply`. Most have defaults you'll never
   touch. See [Configuration](docs/configuration.md).
 - 💡 **Energy-aware earnings.** Set your electricity cost and coin prices — typed in, or fetched
@@ -119,7 +119,7 @@ Full walkthrough: [docs/getting-started.md](docs/getting-started.md)
 | **[Configuration](docs/configuration.md)** | Every `config.json` key, applying changes safely, reusing an existing node, and remote Monero nodes. |
 | **[The Dashboard](docs/dashboard.md)** | Sync Mode, a tour of the live operational view, and the opt-in control channel: editing config, one-click upgrades, and the audit logs from the browser. |
 | **[Connecting Miners](docs/workers.md)** | Point any existing rig at the stack, or spin up a tuned miner with [RigForge](https://github.com/p2pool-starter-stack/rigforge). |
-| **[Architecture](docs/architecture.md)** | The nine services, the privacy model, and the algorithmic XvB switching engine. |
+| **[Architecture](docs/architecture.md)** | The eleven services, the privacy model, and the algorithmic XvB switching engine. |
 | **[Privacy & Network Egress](docs/privacy.md)** | Every off-box connection: what's Tor-routed, what's clearnet today, and how to harden it. |
 | **[Operations & Maintenance](docs/operations.md)** | Full command reference, upgrades, backups, and troubleshooting. |
 
@@ -129,10 +129,10 @@ Browse the full index at **[docs/](docs/README.md)**.
 
 ## 🏗️ How it works
 
-The stack orchestrates nine services via Docker Compose: a Monero full node, P2Pool, a Tari base
+The stack orchestrates eleven services via Docker Compose: a Monero full node, P2Pool, a Tari base
 node, an XMRig proxy (your single worker endpoint), Tor for anonymity, the dashboard plus switching
-engine, a read-only Docker socket proxy (plus a tiny start/stop-only control proxy), and Caddy for
-HTTPS.
+engine, a read-only Docker socket proxy (plus a tiny start/stop-only control proxy), Caddy for
+HTTPS, and two opt-in view-only wallets that confirm payouts on-chain.
 
 ```mermaid
 flowchart TB

VERSION

@@ -1 +1 @@
-1.17.0
+1.18.0

build/dashboard/mining_dashboard/client/xmrig_proxy_client.py

@@ -1,4 +1,3 @@
-import json
 import logging
 
 import requests
@@ -166,41 +165,3 @@ def update_config(self, config_data):
         if response.status_code == 204 or not response.content:
             return {}
         return response.json()
-
-
-if __name__ == "__main__":
-    # Configuration
-    # Ensure xmrig-proxy is running with API enabled:
-    # ./xmrig-proxy --http-port=8080 --http-access-token=SECRET
-
-    HOST = "127.0.0.1"
-    PORT = 8080
-    TOKEN = "SECRET"  # noqa: S105 — placeholder for this __main__ usage example, not a real secret
-
-    client = XMRigProxyClient(HOST, PORT, TOKEN)
-
-    try:
-        # 1. Get Summary
-        print("--- Summary ---")
-        summary = client.get_summary()
-        print(json.dumps(summary, indent=4))
-
-        # 2. Get Workers
-        print("\n--- Worker Details ---")
-        workers = client.get_workers()
-        print(json.dumps(workers, indent=4))
-
-        # 3. Get Config
-        print("\n--- Current Config ---")
-        config = client.get_config()
-        print(json.dumps(config, indent=4))
-
-        # 4. Update Config (Example: changing donate level)
-        # print("\n--- Updating Config ---")
-        # updated_config = client.update_config({"donate-level": 1})
-        # print(json.dumps(updated_config, indent=4))
-
-    except requests.exceptions.RequestException as e:
-        print(f"HTTP Request failed: {e}")
-    except Exception as e:
-        print(f"An error occurred: {e}")

build/dashboard/mining_dashboard/client/xvb_client.py

@@ -101,6 +101,47 @@ def parse_winners(text, wallet_address):
     return wins
 
 
+def parse_round_stats(text):
+    """Aggregate EVERY round in ``winners_recent_full_pub.txt`` — not just ours (#866/#872).
+
+    The file's ``players`` column is the qualifier count for each round, which makes two honest
+    figures computable that XvB's published estimates gloss over: how often a tier is drawn
+    (rounds of that type per day) and against how many qualifiers (players average). Returns
+    ``{"types": {round_type: {"rounds": n, "players_avg": x}}, "span_days": d}`` — an empty
+    ``types`` dict for a garbage/empty file, never a raise (same tolerance and input bounds as
+    ``parse_winners``; the two parse the SAME fetched body, one request)."""
+    types = {}
+    first_ts = last_ts = None
+    for line in (text or "").splitlines()[:_WINNERS_MAX_LINES]:
+        fields = line.split()
+        if len(fields) != _WINNERS_FIELD_COUNT:
+            continue
+        try:
+            ts = (
+                datetime.strptime(f"{fields[1]} {fields[2]}", "%Y-%m-%d %H:%M:%S")
+                .replace(tzinfo=UTC)
+                .timestamp()
+            )
+            players = int(fields[7])
+        except ValueError:
+            continue
+        if players <= 0:
+            continue
+        first_ts = ts if first_ts is None else max(first_ts, ts)
+        last_ts = ts if last_ts is None else min(last_ts, ts)
+        agg = types.setdefault(fields[8], {"rounds": 0, "players_sum": 0})
+        agg["rounds"] += 1
+        agg["players_sum"] += players
+    span_days = (first_ts - last_ts) / 86400.0 if first_ts is not None else 0.0
+    return {
+        "types": {
+            t: {"rounds": a["rounds"], "players_avg": a["players_sum"] / a["rounds"]}
+            for t, a in types.items()
+        },
+        "span_days": span_days,
+    }
+
+
 # register() outcomes (#263). The endpoint returns plaintext "ERROR: ..." with a 422 for the error
 # cases (not a 200/JSON contract), so success/failure is classified from status + body, not status
 # alone. The caller maps these to dashboard state + retry behaviour.
@@ -206,10 +247,11 @@ def get_reward_estimates(self):
             return None
 
     def get_recent_wins(self):
-        """Fetch XvB's public raffle-winners log over Tor and return THIS wallet's wins,
-        oldest first.
+        """Fetch XvB's public raffle-winners log over Tor — ONE request, both parses.
 
-        An empty list is a normal, successful read (no wins in the file's ~4-day window);
+        Returns ``{"wins": [...], "round_stats": {...}}``: ``wins`` is THIS wallet's wins oldest
+        first (``parse_winners``; an empty list is a normal read — no wins in the file's ~4-day
+        window), ``round_stats`` the all-rounds aggregate (``parse_round_stats``, #866/#872).
         ``None`` means the fetch failed — same "None means keep what you have" contract as
         ``get_stats``, so the caller writes nothing and retries later. The file carries only
         masked wallets and the request carries none, so there's no IP<->wallet correlation
@@ -228,7 +270,10 @@ def get_recent_wins(self):
                     f"XvB winners fetch failed with status code: {response.status_code}"
                 )
                 return None
-            return parse_winners(response.text, self.wallet_address)
+            return {
+                "wins": parse_winners(response.text, self.wallet_address),
+                "round_stats": parse_round_stats(response.text),
+            }
         except requests.RequestException as e:
             self.logger.error(f"Network error while fetching XvB winners: {e}")
             return None

build/dashboard/mining_dashboard/helper/utils.py

@@ -94,42 +94,31 @@ def format_disk_size(used_gb, total_gb):
     return f"{used:.1f}", f"{total:.1f}", "GB"
 
 
-def format_xmr(amount):
-    """Format an XMR amount with magnitude-adaptive precision — 4 decimal places at >= 1 XMR,
-    6 at >= 0.001, 8 below that — so a small daily estimate isn't truncated to zeros.
-
-    Mirrors ``formatXmr`` in ``web/static/logic.mjs`` (the dashboard earnings card) so the same
-    estimate reads the same on every surface (#387). Returns "0 XMR" for zero and an em dash for
-    non-numeric input.
-    """
+def _format_coin(amount, symbol):
+    """Magnitude-adaptive coin formatting — 4 decimal places at >= 1, 6 at >= 0.001, 8 below —
+    so a small daily estimate isn't truncated to zeros. Mirrors ``formatXmr``/``formatXtm`` in
+    ``web/static/logic.mjs`` so the same estimate reads the same on every surface (#387).
+    Returns "0 <symbol>" for zero and an em dash for non-numeric input."""
     try:
         val = float(amount)
     except (ValueError, TypeError):
         return "—"
     if not math.isfinite(val):
         return "—"
     if val == 0:
-        return "0 XMR"
+        return f"0 {symbol}"
     dp = 4 if val >= 1 else 6 if val >= 0.001 else 8
-    return f"{val:.{dp}f} XMR"
+    return f"{val:.{dp}f} {symbol}"
 
 
-def format_xtm(amount):
-    """Format an XTM amount, the Tari sibling of :func:`format_xmr` — same magnitude-adaptive
-    precision, same "0 XTM" / em-dash edge cases.
+def format_xmr(amount):
+    """XMR face of :func:`_format_coin`."""
+    return _format_coin(amount, "XMR")
 
-    Mirrors ``formatXtm`` in ``web/static/logic.mjs`` so a confirmed Tari total reads identically
-    in the bot and on the dashboard card (#387)."""
-    try:
-        val = float(amount)
-    except (ValueError, TypeError):
-        return "—"
-    if not math.isfinite(val):
-        return "—"
-    if val == 0:
-        return "0 XTM"
-    dp = 4 if val >= 1 else 6 if val >= 0.001 else 8
-    return f"{val:.{dp}f} XTM"
+
+def format_xtm(amount):
+    """XTM (Tari) face of :func:`_format_coin`."""
+    return _format_coin(amount, "XTM")
 
 
 def format_duration(seconds):

build/dashboard/mining_dashboard/service/algo_service.py

@@ -427,10 +427,17 @@ def _dwell_should_end(
         # effective split far past the computed fraction. A *changed* decision
         # still ends the dwell — only the avg-driven override pauses.
         target_hr = self._get_target_donation_hr(stable_hr)
+        # Catch up only toward what the donation cap allows (#898). Against an explicit target
+        # the fleet cannot sustain, the raw target comparison reads avg_1h as permanently
+        # "under tier", ends every p2pool dwell at its first check tick, and the actuated
+        # donation pins near 100% of allowed time — the cap never binds (measured live:
+        # ~94% of the fleet routed under a 0.65 cap). The achievable donation is the ceiling
+        # worth catching up to; beyond it, cutting dwells short only burns the p2pool side.
+        achievable_hr = min(target_hr, stable_hr * self.max_donation_fraction)
         under_tier = (
             not self._stats_are_stale(xvb_stats)
-            and target_hr > 0
-            and xvb_stats.get("avg_1h", 0) < target_hr
+            and achievable_hr > 0
+            and xvb_stats.get("avg_1h", 0) < achievable_hr
         )
         return decision != held_decision or under_tier
 

build/dashboard/mining_dashboard/service/data_service.py

@@ -888,11 +888,16 @@ async def _sync_xvb_winners(self):
         now = time.time()
         if now - self._last_xvb_winners_sync < _XVB_WINNERS_SYNC_SEC:
             return
-        wins = await asyncio.to_thread(self.xvb_client.get_recent_wins)
-        if wins is None:
+        result = await asyncio.to_thread(self.xvb_client.get_recent_wins)
+        if result is None:
             return  # fetch failed — retry next eligible poll; don't stamp the gate
         self._last_xvb_winners_sync = now
-        new_wins = await asyncio.to_thread(self.state_manager.add_raffle_wins, wins)
+        # Same fetched body, second parse (#866/#872): the all-rounds aggregate that makes win
+        # odds and realized-reward figures computable. Written only when it parsed to something,
+        # so a format change degrades to stale (detectable) rather than an empty-implied-fresh.
+        if (result.get("round_stats") or {}).get("types"):
+            await asyncio.to_thread(self.state_manager.set_xvb_round_stats, result["round_stats"])
+        new_wins = await asyncio.to_thread(self.state_manager.add_raffle_wins, result["wins"])
         for win in new_wins:
             logger.info(
                 f"XvB raffle WIN: {win['tier']} round won at "

build/dashboard/mining_dashboard/service/storage_service.py

@@ -105,6 +105,9 @@ def __init__(self, db_path: str = None):
         # losing it on restart just costs one refetch. ``last_update`` bumps only on a genuine fetch,
         # so the same staleness check as the stats (``xvb_stats_are_stale``) applies to it (#311).
         self._xvb_rewards = {"estimates": {}, "last_update": 0.0}
+        # All-rounds aggregate from the same winners file (#866/#872): round-type frequencies +
+        # qualifier counts. Same memory-only, refetch-on-restart, staleness-by-last_update rules.
+        self._xvb_round_stats = {"stats": {}, "last_update": 0.0}
 
         # Initialize persistent DB connection
         # check_same_thread=False allows the connection to be used by multiple threads
@@ -1374,6 +1377,20 @@ def set_xvb_reward_estimates(self, estimates: dict[str, float]):
         with self._lock:
             self._xvb_rewards = {"estimates": dict(estimates or {}), "last_update": time.time()}
 
+    def get_xvb_round_stats(self) -> dict[str, Any]:
+        """The cached all-rounds raffle aggregate (#866/#872):
+        ``{"stats": {"types": {...}, "span_days": d}, "last_update": ts}``."""
+        with self._lock:
+            return {
+                "stats": dict(self._xvb_round_stats["stats"]),
+                "last_update": self._xvb_round_stats["last_update"],
+            }
+
+    def set_xvb_round_stats(self, stats: dict[str, Any]):
+        """Replace the cached round aggregate and stamp ``last_update`` (only on a genuine fetch)."""
+        with self._lock:
+            self._xvb_round_stats = {"stats": dict(stats or {}), "last_update": time.time()}
+
     def update_xvb_stats(
         self,
         mode: str | None = None,