Send ETag and Last-Modified with rewritten npm and Composer metadata

[simonchrz]

Problem

Rewritten npm and Composer metadata goes out with only a Content-Type header. Without a validator, clients can't revalidate their copy, so every metadata request downloads the whole document again. With npm_full_metadata on, that can be megabytes per package.

Both client families already revalidate when they can:

• npm and pnpm send If-None-Match with the stored ETag. pnpm's resolver also caches a packument's etag, and treats a registry without one as unable to answer 304.
• Composer 2 sends If-Modified-Since with the stored Last-Modified for p2 metadata, and never sends If-None-Match.

Changes

ETag (rewrite_cache.go)

• The SHA-256 of the rewritten body. It's computed once per rewrite and kept with the rewrite cache entry, so the fast path from Cut memory use of npm metadata rewriting #413 serves it without hashing anything.
• When cooldown is on, the rewrite cache is bypassed and every request rewrites. The ETag is then derived from what fixes the output instead of hashing it: the upstream document's digest, the proxy URL, the package name, the versions kept after cooldown and denylist filtering, and a per-process nonce standing for the startup configuration. Without a known digest (metadata caching off) it falls back to hashing the body.

Last-Modified

• The second the package started serving this rewrite. The cache records which rewrite each package is currently serving and since when. When the package switches to another rewrite, including back to one that's still cached (upstream A → B → A), the new one is dated at the switch. A client holding any earlier representation therefore has an earlier date and never gets 304 for this one.
• Upstream's date can't be used: a rewrite also depends on configuration read at startup (proxy URL, denylist), and clamping upstream dates to the process start gave different documents the same date.
• Never ahead of the clock, so dates hold across restarts. A rewrite switched to in a second that already dated another rewrite of the package goes undated until the next second, since the date it would get is one a client may already hold. Nothing is dated in the second the cache starts, the last second the previous process may have dated anything in.
• A package's record is dropped when its current rewrite is evicted, so memory stays bounded by the cache. The next rewrite is then dated at once, unless the record dropped was dated in the current second; in that case new packages wait for the next second too.
• When there is no safe date the response has no Last-Modified, so the client fetches the full body next time instead of risking a false 304.
• The ordering holds within one process. Replicas behind a load balancer keep separate records.
• Omitted with cooldown on: versions leaving cooldown change the rewrite without upstream changing, so a date would make Composer keep an outdated copy.
• Omitted with the rewrite cache off: nothing records when an uncached rewrite was served.

Conditional requests

• Answered with an empty 304 that carries the validators, on both the rewrite fast path and the fetch path.
• If-None-Match decides when present (weak comparison, so ETags weakened by a gzip-ing reverse proxy still match). If-Modified-Since is used only without it, as RFC 9110 specifies.

Unchanged

• Metadata proxied unrewritten because rewriting failed carries no validators.

Verification with real clients

Built proxy, cache_metadata and npm_full_metadata on, two consecutive runs with a shared client cache:

client	first run	second run
composer update (Composer 2.8, monolog + symfony/console)	11 × 200	11 × 304, 0 bytes
npm install --package-lock-only --prefer-online	4 × 200	4 × 304, 0 bytes

pnpm 12.9.1 stores the ETag in its metadata cache. In this setup it resolved from its store on the second run without contacting the registry, so it sent no conditional request to observe.

Tests

metadata_validators_test.go, for both npm and Composer:

• ETag: present on the first fetch, on the rewrite fast path and with TTL 0. Strong, weak and listed If-None-Match values all give an empty 304 that carries the ETag. A new upstream document gives a new ETag.
• Last-Modified: the second the rewrite was first served. If-Modified-Since gives 304 at or after it and 200 before it. A mismatched If-None-Match overrides a matching If-Modified-Since.
• Refresh regression: after upstream changes to a newer document that it also dates before the process started, If-Modified-Since from the first response gets 200 with the new body and a later Last-Modified.
• Returning document regression: upstream A, then B, then A again. A's cached rewrite gets a date later than B's, and If-Modified-Since with B's date gets 200 with A.
• Same second: a document replacing another in the second it was dated gets no Last-Modified, and If-Modified-Since with the first one's date gets 200. It is dated the next second.
• Restart regression: two documents served in one second, then a fresh rewrite cache at the same clock time, then a changed document. If-Modified-Since with the latest date the client got gives 200, no date is ever ahead of the clock, and a date is given from the next second on.
• Rewrite cache dates (fake clock): undated in the cache's first second, the same date while a package keeps serving a rewrite, a new date when it switches back, undated after a same-second switch until the next second, no effect on other packages, and records dropped on eviction.
• Cooldown on: an ETag but no Last-Modified. The ETag changes when a version leaves cooldown and comes from the rewrite's inputs when the digest is known.
• Rewrite cache off: an ETag but no Last-Modified.
• Rewrite failed: no validators.
• Rewrite cache: the stored ETag matches the output.

go test -race ./... and golangci-lint run ./... (v2.13.1) pass.

Not in this PR

ProxyCached (used by APK and other ecosystems) still reads the whole stored document before checking conditional headers, and looks up the metadata row twice per hit. It also lets If-Modified-Since produce a 304 even when a present If-None-Match doesn't match. A follow-up could move it onto the same notModified check.

Rebased onto main after #426 merged. Routed packages' cache keys carry through the metadata lookups and rewrite cache keys.

[montehurd]

I tried this with clients that send back the validators they were given (If-None-Match for npm, If-Modified-Since for Composer), and the 304s work nicely:

Revalidation, cooldown off	main	This PR
npm	6.1 to 7.5 s, all 200	2.6 to 2.7 s, all 304
Composer	5.5 to 7.5 s, all 200	2.5 to 2.6 s, all 304
CPU per request	0.20 ms	0.12 ms

With cooldown on I noticed a cost, though. npm still gets its 304s, a big saving since full packuments averaged about 330 KB here. But every npm metadata request got about 20% more expensive, conditional or not: 4.5 to 4.8 ms of CPU against 3.8 to 4.0 on main. My guess is the SHA-256 over the whole rewritten body on each request.

Could the ETag come from the inputs instead? With cooldown on, the output is fixed by the upstream document's digest, the versions left after cooldown and denylist filtering (which also settles the dist-tags), and the startup config. Hashing those would stay exact without touching the full body. Happy to rerun the numbers if you give it a try.

[simonchrz]

Thanks for measuring this. The ETag with cooldown on now comes from the inputs, as you suggested. It's the SHA-256 of the upstream document's digest, the proxy URL, the package name, the versions kept after cooldown and denylist filtering, and a per-process nonce that stands for the startup configuration.

The digest isn't hashed again per request. It's the one the metadata cache already recorded when it stored the document, and it's passed along with the body. When there's no digest (metadata caching off), it falls back to hashing the body. With cooldown off nothing changes: the body hash is still computed once per cached rewrite.

In a local benchmark on a 1.4 MB, 300-version packument, the ETag went from about 340 µs (body hash) to about 19 µs. I'd be glad if you could rerun your numbers.

[montehurd]

Nice, that fixed it! With cooldown on, npm metadata now costs about the same CPU as main without this PR, and revalidations are even cheaper than on main while still getting 304s. Same test as my earlier comment:

Cooldown on, npm, CPU per request	main	This PR, hashing the body	This PR, ETag from inputs
Normal requests	3.8 ms	4.7 to 4.8 ms	3.9 ms
Revalidations	3.8 ms, all 200	4.3 ms, all 304	3.6 to 3.7 ms, all 304

With cooldown off nothing changed, as you said. In the later rounds, 10 of each tester's 15,380 revalidations came back 200 instead of 304, the same with both versions of the PR. I think that's a version crossing the cooldown boundary mid-test, so it looks right too. Thanks for the quick turnaround!

[andrew]

The Last-Modified calculation can return a false 304 after refreshing cached metadata. I reproduced this through both the npm and Composer HTTP handlers.

[andrew]

Clamping every upstream date before startup to processStartedAt gives different documents the same Last-Modified. For example, a restart can serve an older persisted cache entry that is still within its TTL, then fetch a newer upstream document whose modification date also predates the restart. Both responses receive the startup date, so a client sending the first response's If-Modified-Since gets 304 even though the body changed. A handler-level reproduction for npm and Composer produced a changed body and ETag but HTTP 304. Track modification time per representation, or omit the date when freshness cannot be established, and add regression coverage for refreshing an older cache entry after restart.

[simonchrz]

Thanks, you're right: clamping to the process start collapsed different documents onto one date. Fixed in 8ac4ce2.

Last-Modified no longer comes from upstream at all. It's the time this process first made the rewrite, kept with the rewrite cache entry. Every new rewrite of a package gets a date strictly later than the last one it got, including within the same second and after its entry was evicted, so no two representations share a date. With the rewrite cache off there's no Last-Modified, since nothing records when the rewrite was made. Cooldown still sends none.

Regression coverage is TestRewrittenMetadataLastModifiedFollowsRefresh, through the npm and Composer handlers, with TTL 1h (row expired) and 0. Upstream changes to a newer document that it also dates before the process started, and If-Modified-Since from the first response must get 200 with the new body. It failed with your 304 before the fix. TestRewriteCacheLastModified covers same-second rewrites and eviction with a fake clock.

The cost is one unneeded 200 per package after a restart or eviction, never a false 304. I also rebased onto main to resolve the conflict with #418.

[andrew]

The startup-date clamping issue is fixed, but two false-304 cases remain in the revised Last-Modified handling. Both were reproduced through the npm and Composer HTTP handlers.

[andrew]

Reusing a cached rewrite also reuses its old Last-Modified, even if another representation was served in between. With upstream documents A, then B, then A again, the final response carries A's original date. A client holding B and sending B's If-Modified-Since gets 304 despite the changed body and ETag. I reproduced this through both handlers with the clock advancing ten seconds between changes. Track when the current representation changes, including transitions to a previously cached body, or omit the date when that cannot be established. Add an A-to-B-to-A handler regression test.

[simonchrz]

Confirmed and fixed in ab0f057. A rewrite was dated when it was made, and that date travelled with its cache entry, so a cached A could come back older than B.

The cache now records which rewrite each package is serving and since when. Last-Modified is the second the package switched to the rewrite, so A coming back after B is dated at the switch, later than anything B was dated. TestRewrittenMetadataLastModifiedReturningDocument covers A → B → A through both handlers, with and without a TTL, and the clock moves ten seconds between changes. It got 304 with B's date before the fix.

[andrew]

Incrementing the timestamp for same-second rewrites issues Last-Modified dates ahead of the clock. That history is process-local, so after a restart a changed document can receive a date earlier than one the client already holds, and notModified returns a false 304. I reproduced this through both handlers by serving two different documents at the same clock time, replacing the rewrite cache with a fresh instance to model restart, then serving changed metadata with the second response's If-Modified-Since. Avoid manufactured future dates and add restart coverage; return the body when the timestamp cannot safely validate the client's representation.

[simonchrz]

Confirmed and fixed in ab0f057. The +1s bump is gone, and no date is ever ahead of the clock.

• When a package switches rewrites in a second that already dated another of its rewrites, the new one goes out with no Last-Modified until the next second. The client then has no date to send and gets the full body.
• A new rewrite cache dates nothing in the second it starts, since the previous process may have dated a rewrite in that second.
• Dropping a package's record on eviction follows the same rule. If the record was dated in the current second, new records wait for the next second.

TestRewrittenMetadataLastModifiedAcrossRestart follows your reproduction through both handlers. It serves two documents at the same clock time, replaces the rewrite cache to model the restart, then sends the latest date the client got with changed metadata. It checks that the response is a 200, that no date was ahead of the clock, and that dating resumes the next second. TestRewrittenMetadataLastModifiedSameSecond covers the undated window. Both failed before the change.

The guarantee is per process: replicas behind a load balancer keep separate records. The branch is also rebased onto main after #426.

[andrew]

The previous single-process findings are fixed, but process-local dates still allow false 304 responses across replicas. The Helm chart supports multiple replicas, and sharing database and storage does not share the rewrite cache's serving map.

I reproduced this through both npm and Composer handlers with two proxy instances sharing database and storage, TTL 0, and synchronized clocks one second after startup:

1. Instance A serves the original document with Last-Modified: Sat, 10 Oct 2026 12:00:01 GMT.
2. Upstream changes the document in that second.
3. A conditional request to instance B carrying that date returns 304 with a different ETag. The client retains the old document.

Please make date validation safe across replicas, or omit it when that cannot be guaranteed. A small safe fallback in rewrite_cache.go is:

func (c *rewriteCache) validators(e *rewriteEntry) metadataValidators {
    return metadataValidators{etag: e.etag}
}

The existing serveRewrittenMetadata and notModified paths already omit Last-Modified and decline If-Modified-Since validation when the date is zero. ETag revalidation still works. This would defer the Composer date-based bandwidth saving, so another option is an explicit single-instance opt-in for date validation, disabled by default. Merely omitting the response header while still evaluating the internal date would not fix it.

I checked the ETag-only hint in an isolated copy: the two-instance regression returns 200 with changed metadata, and the existing ETag revalidation tests still pass. Please add a two-instance handler regression that changes upstream between requests in the same second and verifies the old date cannot suppress the new body, including dates held by clients from before the fix.

[simonchrz]

Confirmed, thanks for the two-instance reproduction. Fixed in f68b87e and 171a60e by moving the dates into the shared database, so date validation stays on and is safe across replicas.

• New table metadata_dates, one row per ecosystem, proxy URL and package: the ETag being dated and its date in Unix seconds. Migration 011 creates it on SQLite and Postgres.
• A package's date is the second it started serving its current representation. Every replica reads the same row, so they all serve the same date.
• A switch to another representation is only recorded in a second later than the stored date. The write is a compare-and-swap on the old ETag and date, so when two replicas switch at once, one wins and the other re-reads. Until a later second the new representation goes out without Last-Modified, and If-Modified-Since isn't evaluated.
• A process dates nothing in the second it started. Dates clients hold from before this change were never later than the clock of the process that gave them out, so every date recorded afterwards is later.
• A date ahead of the local clock, written by a replica whose clock is ahead, isn't served until the clock catches up.
• The serving map, lastModified() and forget() are gone. The rewrite cache only keeps ETags, so dates work with the cache off too. Cooldown still sends no Last-Modified.

Regression tests, through the npm and Composer handlers:

• TestRewrittenMetadataLastModifiedAcrossReplicas follows your setup: two proxies sharing database and storage, TTL 0, the same clock one second after startup. A dates the document, upstream changes in that second, and If-Modified-Since with A's date gets 200 with the new body and a new ETag from B and from A. A second later both replicas serve the same, later date, and both answer 304 to it.
• TestRewrittenMetadataLastModifiedFromBeforeTheFix covers dates clients hold from before the change, on both replicas.
• TestRewrittenMetadataLastModifiedWithClockSkew runs one replica two seconds behind.
• The database tests cover the compare-and-swap and UTC handling on SQLite and Postgres.

I checked that each rule is needed: removing the same-second check fails the two-replica test, removing the start-second check fails the pre-fix test, and serving dates ahead of the clock fails the skew test. I also ran two proxy binaries against one Postgres. Both served the same Last-Modified from one row, and B answered 304 to A's date.

Two limits:

• During a rolling upgrade, replicas still on the old code keep handing out in-memory dates, so the guarantee only covers dates from processes that stopped before the new ones started.
• Each served rewrite now costs one extra database read, plus a write when a package switches representation.

The branch is rebased onto main.

[andrew]

Please add upgrade regression coverage for migration 011 on both SQLite and PostgreSQL, starting with the previous schema and representative existing npm/Composer metadata.

TestMetadataDatesTableCreatedByMigration currently creates an empty SQLite database, drops metadata_dates, removes the migration record, and checks only that the table is recreated. It does not verify that existing cached metadata survives the upgrade or that the npm and Composer handlers serve correct validators and conditional responses afterward.

Seed cached documents before applying the migration, verify the existing rows and stored bytes are preserved, then exercise the handlers after upgrade, including a changed document and conditional requests from two replicas.

[simonchrz]

Re review: added upgrade regression coverage for migration 011 on SQLite and PostgreSQL in 9f42425:

• TestMetadataDatesMigrationKeepsCachedMetadata (database): rolls back to the pre-011 schema, seeds cached npm/Composer rows, migrates, checks the rows are unchanged and dates can be recorded.
• TestRewrittenMetadataAfterMetadataDatesMigration (handler): caches npm/Composer metadata before the upgrade, checks the rows and stored bytes survive it, then has two replicas serve the cached document with matching ETag/Last-Modified and 304 for both conditional headers. After the upstream document changes, neither replica answers the old validators with 304, and both agree on the new ones.

The PostgreSQL cases run when PROXY_DATABASE_URL is set (CI doesn't set it); the handler test creates its own database so it doesn't collide with the database package tests.