[HWORKS-2895] Document archiving tag history

[jimdowling]

Documents tag attachment history (HWORKS-2895, hopsworks-ee#3110) in the tags
user guide.

Adds an "Archive tag history" section covering what the flag does, where it is
set, and the two things worth knowing before turning it on: recording starts when
the flag is set, since a tag keeps only its current value and earlier changes are
not recoverable; and existing attachments are backfilled at their attach time,
which is unknown for attachments predating the created_on column.

The section sits after upstream's "When a tag was attached", which it builds on:
the attachment time is what the backfill uses for its starting point. Both are
unnumbered, so the numbered Step 1 to Step 3 walkthrough is unchanged.

---

Since the last body update

The published interval query was the broken one (review round 5). The dashboards and the design
note were corrected in round 4; this page was not, so the query a reader is most likely to copy was
the only copy still wrong — in both ways:

• It filtered to OPENED in the same query that evaluates LEAD. SQL applies WHERE before window
  functions, so the window never saw a CLOSED row and anything ending without a successor, a
  detached tag or a deleted artifact, read as still current with its duration growing forever.
• It ordered by id, which cannot break the tie between the two halves of a value change: they
  share one event_time by design and are not written in the order they were built.

Both are shown correctly now, each with a note, because both produce numbers that look entirely
reasonable.

The archive flag's UI claim was corrected too. The guide said it could be set "when the schema
is created or afterwards" in Cluster settings; there is no afterwards in the UI, so the guide now
gives the admin REST call. It also documents what turning archiving off does — it ends the
intervals still open and keeps what was recorded.

Testing

test-docs-build: pass.

[jimdowling]

Review round 5 — 8 findings: 4 fixed, 4 accepted

All eight verified against the code and the cluster before acting.

Fixed

#3 (High) — two granted tables held secrets and fed them to the model provider. alert_receiver.config is the raw notification configuration (API keys, webhook URLs, routing keys); trino_queries holds query text, prepared statements, plans and the principal that ran them. Neither is read by any dashboard. Granting a table is what makes okr-dashboards mount it, and mounting samples rows through infer_metadata. A scan of the schema found 79 sensitive-looking columns cluster-wide and exactly these two granted tables carrying any. Removed from both lists (80 → 78 grants, 78 → 76 mounts) and applied to jim-dashboard, so the live account no longer reaches either.

#1 (High) — disabling the feature on a create_secrets=false cluster revoked nothing. It printed a comment while reconciliation stopped, so existing connections kept working against the documented promise. Now revokes rather than drops, since the account may predate the feature. Note for reviewers: REVOKE ... IF EXISTS does not cover a missing account — it still raises 1269 — so a bare revoke would have failed the migration on every cluster that never enabled this. It is conditional on mysql.user, and both paths were executed against RonDB 26.02.9.

#7 (Medium) — the published SQL was still the broken one. A miss from round 4: the dashboards and design note were fixed then, the user guide was not, leaving the copy a reader is most likely to paste as the only one still wrong — in both ways.

#4 (Medium) — schema delete queued no reindex for models or deployments. The overloads existed; the model-registry loop only closed history. Fixed.

Accepted as they stand (owner decision)

#	Finding	Why it stands
2	demoted analytics owner keeps metadata access	the deeper revocation deletes user data; already residual D9
5	same-millisecond event_id collision	the fix removes retry idempotency, trading a loud rare failure for silent double counting
6	wizard runs instructions from an unpinned checkout	instructions move faster than a pin allows; first-party repo, same org
8	unquoted sed for passwords	pre-existing, shared with two passwords every cluster's init needs

Each is recorded in the code, not just here — #5 had already been raised twice, and #2/#6/#8 are the kind that come back.

Also fixed in passing

.github/test_lint.sh was failing on the parent README, stale since this branch's kyverno commit regenerated the subchart README but not the parent's. Regenerated; lint exits 0.

Verification

hopsworks-common 3345 tests / 0 failures · helm template 78 grants with neither sensitive table · rendered REVOKE verified against a present and an absent account on RonDB 26.02.9 · test_lint.sh exit 0 · mount parse 76 tables, 0 unmatched.

[Copilot]

Pull request overview

Adds documentation to the Hopsworks Feature Store tags user guide describing the Archive tag history capability, including how it’s enabled, what it records, and how to query the resulting history data from the metadata database.

Changes:

• Adds an “Archive tag history” section explaining behavior when toggling history archiving and what is (and isn’t) recoverable.
• Documents where the history is stored and provides an example SQL query to derive added_on / removed_at intervals from events.
• Notes important query pitfalls (filter placement and ordering for same-timestamp OPEN/CLOSE events).

Suppressed comments (6)

docs/user_guides/fs/tags/tags.md:258

• Documentation style: lines are split mid‑sentence and multiple sentences are combined on the same line. Rewrap to keep each sentence on its own line (and avoid breaking a sentence across lines).

The flag is set per schema, in `Cluster settings` > `Tag schemas`, when the schema is created. To
turn it on or off for a schema that already exists, a cluster administrator calls
`PUT /hopsworks-api/api/tags/{name}/archive?value=true`. It applies to every artifact the tag is
attached to: feature groups, feature views, training datasets, jobs, models and deployments.

docs/user_guides/fs/tags/tags.md:264

• Documentation style: this paragraph breaks sentences across lines and groups multiple sentences per line. Rewrap so each sentence is on its own line.

Turning it off ends every interval the tag still has open, at the moment you turn it off, and keeps
everything already recorded. The recorded rows stay because they are still true; the open intervals
have to be ended because nothing would ever end them once recording stops, and the last value of
every artifact would otherwise read as current forever. Turning it back on starts a fresh interval
at that moment rather than pretending the gap was observed.

docs/user_guides/fs/tags/tags.md:272

• Documentation style: the list items and following paragraph are wrapped mid‑sentence. Repo convention is one sentence per line; rewrap so each sentence is on its own line (with proper list indentation).

- **History starts when you turn it on.** Changes made before that are not recoverable, because the
  live tag keeps only its current value. Attachments that already exist are backfilled with the
  state they are in, timed from when they were attached.
- **Turning it off stops recording but keeps what was recorded.** The rows already written are still
  true, and the tag is still attached, so nothing is deleted.

docs/user_guides/fs/tags/tags.md:289

• Documentation style: these paragraphs are wrapped mid‑sentence. Rewrap so each sentence is on its own line (and avoid splitting a sentence across multiple lines).

It is read with SQL rather than through the tag APIs, which continue to return the current value.
On a cluster with the `hopsworks_analytics` project enabled, its Superset connection can query the
table directly, and
`create_tag_history_dashboard.py` in the `okr-dashboards` repository builds a "Tag Lifecycle"
dashboard over it: time spent in each state, whether that is increasing, what is in each state now,
and what has been in one state longest.

docs/user_guides/fs/tags/tags.md:316

• Documentation style: the bullets here split sentences across multiple lines. Rewrap so each sentence is on its own line (with list continuation lines properly indented).

Two details in that query are easy to get wrong and produce numbers that look reasonable:

- The `OPENED` filter has to be in the outer query. SQL applies `WHERE` before window functions, so
  filtering inside would hide every `CLOSED` row from `LEAD`, and anything that ended without a
  successor, a detached tag or a deleted artifact, would report as still current with its duration
  growing forever.

docs/user_guides/fs/tags/tags.md:323

• Documentation style: these paragraphs contain multiple sentences on one line and are wrapped mid‑sentence. Rewrap so each sentence is on its own line.

A `removed_at` of `NULL` means the artifact is still in that state. An `added_on` of `NULL` means the
tag was attached before Hopsworks began recording attachment times, so the start is unknown; it is
left empty rather than filled with a guess.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.