Make cache writes visible so query previews stop reading as permanent…

[Robbie1977]

…ly pending

Every get_term_info preview had been coming back blank. The cause was not in the preview logic: cache_result posted to Solr with commit=false and relied on an autoSoftCommit the cache core does not have (autoSoftCommit.maxTime: -1, autoCommit.openSearcher: false). Writes were accepted, durable, and returned HTTP 200 — and never became searchable. So every read-back missed, every call was cold, every cold call took the two-phase fast path that blanks previews and warms them in the background, and the warmed answer went into the same invisible hole. count: -1, a state meant to last seconds, became permanent.

Six changes, in the order they were found:

1. commitWithin on both write paths. The core's updateHandler sets commitWithin: {softCommit: true}, so the POST still returns immediately while a searcher reopens inside the window. VFBQUERY_SOLR_COMMIT_WITHIN_MS tunes it (non-positive values clamp to the 10s default rather than reinstating "never"); VFBQUERY_SOLR_WRITE_COMMIT=true restores the old blocking hard commit for bulk warming. The expired-document delete had the same bug with a nastier shape: an invisible delete leaves the expired doc in place to be read back and re-expired forever.

2. status/message on preview_results. -1 has always meant "not counted yet", which is not 0 ("no matches") — documented in CACHING.md and honoured by the v2 frontend — but nothing in the payload said so. The key is additive and optional, and absence has to keep meaning complete: the cache holds three months of entries written before it existed.

3. preview_is_resolved() replaces the literal count >= 0 test in all three validators. That test also condemned a complete preview whose exact total exceeded COUNT_CAP, where -1 means "many" and the rows are final. Any term with a large query was therefore refused by the cache and recomputed on every single request, forever, silently. Nobody had noticed.

4. ha_api honours X-Force-Refresh. The v3-cached nginx layer already defines that header as "bypass the edge and overwrite the canonical slot", but this service read no headers, so the header refreshed the edge from an unrefreshed upstream. The obvious workaround is worse: &force_refresh=true changes $request_uri and therefore the nginx cache key, so it writes a different slot and can never heal the canonical one.

5. Docs: schema.md gains a "Query previews and the -1 count" section and the status key; CACHING.md and docs/http-api.md record the fix and the header.

6. Tests: tests/test_preview_status.py and tests/test_force_refresh_header.py. Two stale expectations in tests/test_param_validation.py are fixed in passing — a stub whose signature had fallen behind _dispatch_to_pool, and a source-text assertion tripped by a comment mentioning the symbol it forbids.

Verified end to end against the real cache core under a private VFBQUERY_CACHE_NAMESPACE: a lone commit=false write was still unsearchable after 75s, while cache_result's write was searchable in ~12s and round-tripped back through get_cached_result.

Patch bump only (1.22.36 -> 1.22.37); no @with_solr_cache bucket renames, so the existing cache survives.

Claude-Session: https://claude.ai/code/session_01Y8ZAhRM7gh8arDx9SuSWcT