From ad25037264ee885c4a6c06c4183b729a4d5caaec Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=EC=B5=9C=EC=98=81=EA=B8=B0?= Date: Wed, 15 Jul 2026 15:19:29 +0900 Subject: [PATCH 1/2] knowledge: ingest 3 verified insight(s) --- .dev-loop/INGEST_REPORT.md | 77 +++++++++++++++++++ log.md | 3 + .../containers/pid1-entrypoint-log-flush.md | 73 ++++++++++++++++++ wiki/infrastructure/index.md | 2 + .../observability/kubelet-resource-metrics.md | 74 ++++++++++++++++++ .../observability/logs-metrics-signals.md | 2 +- .../shells/portable-shell-scripts.md | 2 +- wiki/platforms/tools/bsd-vs-gnu-cli.md | 8 +- 8 files changed, 238 insertions(+), 3 deletions(-) create mode 100644 .dev-loop/INGEST_REPORT.md create mode 100644 wiki/infrastructure/containers/pid1-entrypoint-log-flush.md create mode 100644 wiki/infrastructure/observability/kubelet-resource-metrics.md diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md new file mode 100644 index 0000000..b3415c1 --- /dev/null +++ b/.dev-loop/INGEST_REPORT.md @@ -0,0 +1,77 @@ +# Knowledge flush — 3 insight(s) + +Drained 3 queued `★ Insight` candidates. Each was independently researched and +verified against primary sources before ingest. Two new pages created, one +existing page extended. No auto-merge — this PR is for review. + +## Verified best-practice + +### 1. Empty cAdvisor `container_*` metrics on a lightweight kubelet → scrape `/metrics/resource` +- **Claim:** With kube-prometheus-stack, pod CPU/memory series are empty though + kubelet targets are UP; fix by enabling `kubelet.serviceMonitor.resource: true` + and overriding `resourcePath: /metrics/resource`; mixin dashboards hide the + resource series via `image!=""`. +- **Sources checked (real):** + - kps chart values.yaml — confirms both keys, `resource` default **false**, and `resourcePath` default is the stale **`/metrics/resource/v1alpha1`**: https://github.com/prometheus-community/helm-charts/blob/main/charts/kube-prometheus-stack/values.yaml + - k8s metrics docs — `/metrics/resource` is the current STABLE endpoint; series carry `namespace`/`pod`/`container` only (no `image`): https://kubernetes.io/docs/reference/instrumentation/metrics/ , https://kubernetes.io/docs/concepts/cluster-administration/system-metrics/ + - k8s PR#86282 — the `v1alpha1` path was removed (404 on kubelet ≥1.20): https://github.com/kubernetes/kubernetes/pull/86282 + - kubernetes-mixin — Compute Resources dashboards select cAdvisor series with `image!=""` **and** `job="cadvisor"`/`metrics_path="/metrics/cadvisor"`: https://github.com/kubernetes-monitoring/kubernetes-mixin +- **Verification result:** mechanism + fix **VERIFIED** by primary sources. **Correction folded in:** `image!=""` is not the *only* reason the resource series are excluded — the mixin also pins `job=cadvisor`/`metrics_path=/metrics/cadvisor`, so a custom panel is required regardless (the page states this, not "just drop the image filter"). The OrbStack-specific root cause (cAdvisor emitting only `machine_*`) is **field-observed**, not documented upstream — labeled as such in the page and log. +- **Confidence: verified** (fix is kubelet-endpoint-level and independent of why cAdvisor is empty). + +### 2. Container PID 1 entrypoint loses piped logs on exit → wait for tee in an EXIT trap +- **Claim:** `exec > >(tee -a file)` in a PID 1 bash entrypoint loses log output on + container exit; capture `TEE_PID=$!` and `wait` it in an EXIT trap after closing + fds; add `trap 'exit 143' TERM`. +- **Sources checked (real):** + - Greg's Wiki — process substitution is an **unwaited** background subshell; bash sets `$!` to its PID so you can `wait`: https://mywiki.wooledge.org/ProcessSubstitution + - Docker stop — SIGTERM then SIGKILL after grace period: https://docs.docker.com/reference/cli/docker/container/stop/ + - pipe(7) — a read sees EOF only when **all** write-end fds are closed: https://www.man7.org/linux/man-pages/man7/pipe.7.html + - bash manual — signal-killed command exits 128+N (SIGTERM 15 → 143): https://www.gnu.org/software/bash/manual/html_node/Exit-Status.html + - PID 1 signal write-up — PID 1 ignores SIGTERM without a handler; orphans reaped only if PID 1 waits: https://petermalmgren.com/signal-handling-docker/ +- **Verification result:** all four mechanisms **VERIFIED**. **Correction folded in:** capturing the process-substitution PID via `$!` is **bash 4.4+** (2016), NOT 5.1 as the raw candidate implied — the page states 4.4+ and gives a named-FIFO fallback for older bash. EOF caveat (all write ends must close) captured as an edge case. +- **Confidence: verified** (documented mechanism) — the original 10/10 capture measurement is noted as field evidence. + +### 3. Feature-detect `date +%3N` by 3-digit output, not by `%N` support +- **Claim:** Detecting GNU vs BSD `date` by "does `%N` work" is wrong because modern + macOS supports `%N`; check that `date +%3N` outputs three digits. +- **Sources checked (real):** + - GNU coreutils — `%N` nanoseconds + numeric field-width form: https://www.gnu.org/software/coreutils/manual/html_node/Options-for-date.html + - FreeBSD/macOS `date` — `%N` added in FreeBSD 14.1, GNU-compatible on modern macOS; no `%3N` width form documented: https://man.freebsd.org/cgi/man.cgi?query=date + - POSIX strftime — an unrecognized conversion specification is **undefined** (BSD passes it through literally): https://pubs.opengroup.org/onlinepubs/9699919799/functions/strftime.html +- **Verification result:** core advice **VERIFIED and strengthened** — modern macOS returns real nanoseconds for `%N`, making it an even worse discriminator than the candidate claimed. Two specifics are **field-observed**: the exact `%3N` → literal `3N` output (consistent with POSIX "undefined", not documented as `3N`), and GNU `%3N` = "first 3 digits" (width mechanism documented; the truncation semantics universally relied on but not spelled out). The page frames `3N` as field-observed. +- **Confidence: verified** (recommendation robust; two specifics noted as field-observed). + +## Existing-layer check + +- Read all `infrastructure/observability/` pages (`logs-metrics-signals`, + `alerting`) and all `infrastructure/containers/` pages (`image-builds`, + `resource-limits-and-probes`), plus `platforms/tools/bsd-vs-gnu-cli`, + `platforms/shells/portable-shell-scripts`, `platforms/environment/timezone-and-locale`. +- **#1 (kubelet metrics):** `logs-metrics-signals` covers *what/how to instrument* + (signal choice, cardinality, golden signals) — a different trigger from "a + specific series is empty despite a healthy scrape." Merging would blur that + page's focus, so **created a new page** and cross-linked both ways + (`related:`). No conflict. +- **#2 (PID 1 tee):** the decisive trigger condition is *container PID 1*, not OS + portability, so it does not belong in `platforms/shells/portable-shell-scripts` + (which already covers process substitution/`set -e`/EXIT traps for the + portability case). **Created a new page** under `infrastructure/containers` + (owns the PID 1 lifecycle artifact) and added a `related:` link both ways to + the shell page. No conflict. +- **#3 (`date %3N`):** directly overlaps the existing `date` guidance in + `bsd-vs-gnu-cli` → **merged** (one Do-table row + one Instead-of row + two + sources + `last_verified` bump). No new page. No conflict with the existing + "Relative date" row (distinct sub-case). + +## Routing decision + +| Insight | Target | Action | +|---------|--------|--------| +| #1 kubelet resource metrics | `infrastructure/observability/kubelet-resource-metrics.md` (new page) | Created; registered in `infrastructure/index.md` observability table; `related:` ↔ `logs-metrics-signals` | +| #2 PID 1 tee log flush | `infrastructure/containers/pid1-entrypoint-log-flush.md` (new page) | Created; registered in `infrastructure/index.md` containers table; `related:` ↔ `portable-shell-scripts`, → `resource-limits-and-probes`, `logs-metrics-signals` | +| #3 `date +%3N` detection | `platforms/tools/bsd-vs-gnu-cli.md` (existing) | Merged one Do-row + one Instead-of row; added GNU date + POSIX strftime sources; `last_verified` → 2026-07-15 | + +No new category was needed — both new pages fit existing categories +(`infrastructure/observability`, `infrastructure/containers`). `log.md` has three +appended entries (2× ingest, 1× revise). diff --git a/log.md b/log.md index bcc206d..ff61b5d 100644 --- a/log.md +++ b/log.md @@ -32,3 +32,6 @@ Append-only. Format: `## [YYYY-MM-DD] >(tee)` loses logs on exit (bash never reaps the process-sub subshell; PID 1 exit kills tee mid-flush); capture `TEE_PID=$!` (bash 4.4+, not 5.1) + EXIT trap `exec 1>&- 2>&-; wait` + `trap 'exit 143' TERM`. Sources verified: Greg's Wiki ProcessSubstitution, Docker stop, pipe(7), bash Exit-Status, PID1 signal writeup. Field-tested 10/10 capture. +## [2026-07-15] revise | platforms/tools/bsd-vs-gnu-cli +1 Do-row + Instead-of row: millisecond timestamp — feature-detect by `date +%3N` 3-digit output, not by whether `%N` works (modern macOS supports `%N` nanoseconds so it no longer discriminates; the `%3N` width form is unrecognized and surfaces literally as `3N`). Sources +GNU date options, +POSIX strftime (unknown conversion undefined). last_verified 2026-07-15. diff --git a/wiki/infrastructure/containers/pid1-entrypoint-log-flush.md b/wiki/infrastructure/containers/pid1-entrypoint-log-flush.md new file mode 100644 index 0000000..9f439a2 --- /dev/null +++ b/wiki/infrastructure/containers/pid1-entrypoint-log-flush.md @@ -0,0 +1,73 @@ +--- +id: infrastructure-containers-pid1-entrypoint-log-flush +domain: infrastructure +category: containers +applies_to: [containers, docker, kubernetes, bash] +confidence: verified +sources: + - https://mywiki.wooledge.org/ProcessSubstitution + - https://docs.docker.com/reference/cli/docker/container/stop/ + - https://www.man7.org/linux/man-pages/man7/pipe.7.html + - https://www.gnu.org/software/bash/manual/html_node/Exit-Status.html + - https://petermalmgren.com/signal-handling-docker/ +last_verified: 2026-07-15 +related: [infrastructure-containers-resource-limits-and-probes, platforms-shells-portable-shell-scripts, infrastructure-observability-logs-metrics-signals] +--- + +# Container PID 1 Entrypoint: Flushing Piped Logs on Exit + +## When this applies + +A bash script runs as PID 1 (the container entrypoint) and duplicates its stdout +to a file with process substitution — `exec > >(tee -a "$LOG")`. Log lines are +lost when the container exits: short jobs lose everything, long jobs lose the tail +(the final summary lines). + +## Do this + +Process substitution `>(tee …)` runs in an **unwaited background subshell** — bash +does not reap it on exit. A container terminates the instant PID 1 exits, so the +`tee` is killed before it flushes its buffer. Make the entrypoint wait for the tee +to drain before it exits. + +1. Capture the tee's PID on the line immediately after starting it (bash **4.4+**, + where `$!` is set to the process-substitution PID): + ```bash + exec > >(tee -a "$LOG") 2>&1 + TEE_PID=$! + ``` + Capture it right away — any other backgrounded job overwrites `$!`. +2. In an EXIT trap, close the write ends first so `tee` sees EOF, then wait for it: + ```bash + trap 'exec 1>&- 2>&-; wait "$TEE_PID"' EXIT + ``` + Order matters — closing fds 1/2 delivers the pipe EOF that makes `tee` finish its + last read and flush; the `wait` then blocks until it has exited. +3. Make container stop run the EXIT trap. On `docker stop`/pod termination, PID 1 + receives SIGTERM then (after the grace period) SIGKILL, and PID 1 **ignores** + SIGTERM unless a handler is registered. Add `trap 'exit 143' TERM` (128+15) so a + stop signal exits cleanly and fires the EXIT trap's flush. + +## Edge cases + +| Case | Then | +|------|------| +| bash < 4.4 | `$!` does not expose the process-substitution PID — start the writer explicitly through a named FIFO (`mkfifo`; `tee -a "$LOG" < "$FIFO" & TEE_PID=$!`) and redirect to the FIFO | +| Another child still holds a copy of fd 1/2 | Closing the fds only in the shell does not deliver EOF — `tee` blocks until every write end is closed; ensure children exit or close their fds first | +| Only the tail is missing (long jobs) vs everything (short jobs) | Same root cause — exit-before-flush; the fix (wait in EXIT trap) covers both | + +## Instead of + +| If you are about to | Do this instead | Why | +|---------------------|-----------------|-----| +| `exec > >(tee …)` with no trap | Capture `TEE_PID=$!` and `wait` it in an EXIT trap after closing the fds | Bash never reaps the process-substitution subshell; PID 1 exit kills it mid-flush | +| Assume logs were captured because the completion line printed to the console | Wait for the tee in the EXIT trap | Console output and the tee's file buffer are independent; PID 1 exit discards the unflushed buffer | +| Rely on `docker stop` alone to end the script | Add `trap 'exit 143' TERM` | PID 1 ignores SIGTERM without a handler, so the EXIT trap (and its flush) never runs before SIGKILL | + +## Sources + +- https://mywiki.wooledge.org/ProcessSubstitution — process substitution runs in an unwaited subshell; bash 4.4+ sets `$!` to its PID so you can `wait` +- https://docs.docker.com/reference/cli/docker/container/stop/ — stop sends SIGTERM, then SIGKILL after the grace period +- https://www.man7.org/linux/man-pages/man7/pipe.7.html — a read sees EOF only once all write-end fds are closed +- https://www.gnu.org/software/bash/manual/html_node/Exit-Status.html — a command killed by signal N exits 128+N (SIGTERM 15 → 143) +- https://petermalmgren.com/signal-handling-docker/ — PID 1 ignores SIGTERM unless a handler is registered; orphans reaped only if PID 1 waits diff --git a/wiki/infrastructure/index.md b/wiki/infrastructure/index.md index 4940602..2aeb8f3 100644 --- a/wiki/infrastructure/index.md +++ b/wiki/infrastructure/index.md @@ -27,6 +27,7 @@ Match your situation to a "load when" line; load only matching pages. |------|-----------| | [image-builds](containers/image-builds.md) | Writing or reviewing a Dockerfile; images rebuild everything on small changes, build slowly, or are too large; choosing an image tagging scheme | | [resource-limits-and-probes](containers/resource-limits-and-probes.md) | Writing or reviewing Kubernetes-style deployment manifests; pods OOMKilled, evicted, or CPU-throttled; a dependency outage triggered a restart storm; traffic hitting pods that are not ready | +| [pid1-entrypoint-log-flush](containers/pid1-entrypoint-log-flush.md) | A container entrypoint bash script pipes stdout via `exec > >(tee …)` and loses log lines on exit (short jobs lose all, long jobs lose the tail); making PID 1 flush piped logs and run its EXIT trap on container stop (SIGTERM) | ## data @@ -46,3 +47,4 @@ Match your situation to a "load when" line; load only matching pages. |------|-----------| | [logs-metrics-signals](observability/logs-metrics-signals.md) | Instrumenting a new or existing service (logs, metrics, correlation ids); an incident revealed you couldn't see what happened; choosing between a log line and a metric; a metric label would carry unbounded values (user ids/UUIDs) | | [alerting](observability/alerting.md) | Creating or reviewing alerts; the team ignores a noisy pager; deciding whether a condition pages, tickets, or stays on a dashboard | +| [kubelet-resource-metrics](observability/kubelet-resource-metrics.md) | Pod CPU/memory (`container_*`) series are empty though kubelet scrape targets are UP; wiring pod-resource dashboards on a lightweight kubelet (OrbStack/kind/k3s); a kube-prometheus-stack resource-metrics target 404s | diff --git a/wiki/infrastructure/observability/kubelet-resource-metrics.md b/wiki/infrastructure/observability/kubelet-resource-metrics.md new file mode 100644 index 0000000..1f36eeb --- /dev/null +++ b/wiki/infrastructure/observability/kubelet-resource-metrics.md @@ -0,0 +1,74 @@ +--- +id: infrastructure-observability-kubelet-resource-metrics +domain: infrastructure +category: observability +applies_to: [kubernetes] +confidence: verified +sources: + - https://github.com/prometheus-community/helm-charts/blob/main/charts/kube-prometheus-stack/values.yaml + - https://kubernetes.io/docs/reference/instrumentation/metrics/ + - https://kubernetes.io/docs/concepts/cluster-administration/system-metrics/ + - https://github.com/kubernetes/kubernetes/pull/86282 + - https://github.com/kubernetes-monitoring/kubernetes-mixin +last_verified: 2026-07-15 +related: [infrastructure-observability-logs-metrics-signals, infrastructure-containers-resource-limits-and-probes] +--- + +# Empty cAdvisor `container_*` Metrics on a Lightweight Kubelet + +## When this applies + +You installed kube-prometheus-stack and pod CPU/memory series +(`container_cpu_usage_seconds_total`, `container_memory_working_set_bytes`) are +empty in Prometheus/Grafana, yet every kubelet scrape target shows UP. Common on +lightweight kubelets (OrbStack, kind, k3s) whose cAdvisor endpoint scrapes fine +but emits no per-container series. + +## Do this + +A scrape target being UP means the endpoint answered — not that it produced the +series you expected. Confirm the series with a PromQL query, not the target list. + +1. Verify the gap at the series level: query `container_memory_working_set_bytes` + directly. If cAdvisor (`/metrics/cadvisor`) returns only `machine_*` and no + `container_*`, the fix is to scrape the kubelet's separate resource-metrics + endpoint instead of relying on cAdvisor. +2. Enable `/metrics/resource` in the kube-prometheus-stack Helm values — and + override the path, because the chart default is stale: + +| Value | Set to | Why | +|-------|--------|-----| +| `kubelet.serviceMonitor.resource` | `true` | Default is `false` ("container metrics are already exposed by cAdvisor" — which is the assumption that fails here) | +| `kubelet.serviceMonitor.resourcePath` | `/metrics/resource` | Chart default `/metrics/resource/v1alpha1` was removed in Kubernetes 1.20 and returns 404 on any modern kubelet — leaving that default makes the new target go down | + +3. Read the resource series with a custom query/dashboard. The `/metrics/resource` + endpoint labels series with `namespace`, `pod`, `container` only — there is **no + `image` label**. The bundled kubernetes-mixin "Compute Resources" dashboards + select cAdvisor series with `job="cadvisor"`, `metrics_path="/metrics/cadvisor"`, + and `image!=""`, so they exclude the resource-endpoint series entirely — build a + custom panel rather than expecting the stock dashboard to light up. + +## Edge cases + +| Case | Then | +|------|------| +| Target UP but `container_*` count is 0 | Scrape success ≠ series creation — switch to `/metrics/resource`; do not trust the target list as a metrics-health signal | +| New resource target shows DOWN (404) | `resourcePath` is still the default `/metrics/resource/v1alpha1`; set it to `/metrics/resource` | +| Root cause of the empty cAdvisor is unknown (e.g. OrbStack — field-observed, not documented upstream) | Apply the fix anyway; `/metrics/resource` is an independent kubelet endpoint and does not depend on why cAdvisor is empty | +| Stock "Compute Resources" dashboard still blank after enabling resource | Its `image!=""`/`job="cadvisor"` selectors also exclude the series — query the raw metric in a custom panel | + +## Instead of + +| If you are about to | Do this instead | Why | +|---------------------|-----------------|-----| +| Conclude metrics are healthy because kubelet targets are UP | Query the actual series in PromQL | A successful scrape does not guarantee the series were generated | +| Fix a blank mixin dashboard by dropping only the `image!=""` filter | Query `/metrics/resource` series in a custom panel | The mixin also pins `job="cadvisor"`/`metrics_path="/metrics/cadvisor"`, so the series stay excluded | +| Trust the chart's default `resourcePath` | Set `resourcePath: /metrics/resource` explicitly | The default points at the v1alpha1 path removed in k8s 1.20 (404) | + +## Sources + +- https://github.com/prometheus-community/helm-charts/blob/main/charts/kube-prometheus-stack/values.yaml — `kubelet.serviceMonitor.resource` (default false) and `resourcePath` (default `/metrics/resource/v1alpha1`) +- https://kubernetes.io/docs/reference/instrumentation/metrics/ — `/metrics/resource` metrics and their `namespace`/`pod`/`container` labels (no `image`) +- https://kubernetes.io/docs/concepts/cluster-administration/system-metrics/ — kubelet `/metrics/resource` endpoint +- https://github.com/kubernetes/kubernetes/pull/86282 — removal of the `/metrics/resource/v1alpha1` path +- https://github.com/kubernetes-monitoring/kubernetes-mixin — Compute Resources dashboards select cAdvisor series with `image!=""` / `job="cadvisor"` diff --git a/wiki/infrastructure/observability/logs-metrics-signals.md b/wiki/infrastructure/observability/logs-metrics-signals.md index 87e9108..9018ec6 100644 --- a/wiki/infrastructure/observability/logs-metrics-signals.md +++ b/wiki/infrastructure/observability/logs-metrics-signals.md @@ -9,7 +9,7 @@ sources: - https://prometheus.io/docs/practices/naming/ - https://cheatsheetseries.owasp.org/cheatsheets/Logging_Cheat_Sheet.html last_verified: 2026-07-10 -related: [infrastructure-observability-alerting] +related: [infrastructure-observability-alerting, infrastructure-observability-kubelet-resource-metrics] --- # Instrumenting a Service with Logs and Metrics diff --git a/wiki/platforms/shells/portable-shell-scripts.md b/wiki/platforms/shells/portable-shell-scripts.md index dcab402..6f5f449 100644 --- a/wiki/platforms/shells/portable-shell-scripts.md +++ b/wiki/platforms/shells/portable-shell-scripts.md @@ -11,7 +11,7 @@ sources: - https://google.github.io/styleguide/shellguide.html - https://www.shellcheck.net/ last_verified: 2026-07-10 -related: [platforms-tools-bsd-vs-gnu-cli, platforms-toolchains-version-management] +related: [platforms-tools-bsd-vs-gnu-cli, platforms-toolchains-version-management, infrastructure-containers-pid1-entrypoint-log-flush] --- # Shell Scripts That Must Run on More Than One Machine or Shell diff --git a/wiki/platforms/tools/bsd-vs-gnu-cli.md b/wiki/platforms/tools/bsd-vs-gnu-cli.md index 21181ef..a64ef06 100644 --- a/wiki/platforms/tools/bsd-vs-gnu-cli.md +++ b/wiki/platforms/tools/bsd-vs-gnu-cli.md @@ -9,7 +9,9 @@ sources: - https://man.freebsd.org/cgi/man.cgi?date(1) - https://man.freebsd.org/cgi/man.cgi?sed(1) - https://man.freebsd.org/cgi/man.cgi?seq(1) -last_verified: 2026-07-10 + - https://www.gnu.org/software/coreutils/manual/html_node/Options-for-date.html + - https://pubs.opengroup.org/onlinepubs/9699919799/functions/strftime.html +last_verified: 2026-07-15 related: [platforms-shells-portable-shell-scripts] --- @@ -28,6 +30,7 @@ flags do not. Apply the portable fix per command: | Command | GNU (Linux) | BSD (macOS) | Portable fix | |---------|-------------|-------------|--------------| | Relative date | `date -d '5 minutes ago'` | `date -v-5M` | Feature-detect with a fallback chain: `date -d '5 minutes ago' 2>/dev/null \|\| date -v-5M`; or compute epoch math: `$(( $(date +%s) - 300 ))` then format | +| Millisecond timestamp | `date +%3N` (first 3 digits of `%N`) | `%N` is supported (nanoseconds; macOS 14.1+ and GNU) but the `%3N` **width** form is not — an unrecognized spec surfaces literally (field-observed as `3N`), giving broken output like `…08.3NZ` | Feature-detect by the OUTPUT, not by `%N`: `[[ "$(date +%3N)" == [0-9][0-9][0-9] ]]` before using `%3N`; else fall back to `gdate +%3N`, or drop the sub-second field | | In-place sed | `sed -i 's/a/b/' f` (bare `-i`) | `-i` takes a backup-extension argument (`sed -i '' 's/a/b/' f`) | `sed -i.bak 's/a/b/' f && rm f.bak` (no space before `.bak` — parses on both), or `perl -pi -e 's/a/b/' f` | | `timeout` | In coreutils | Absent on stock macOS — exits 127 | Install coreutils and call `gtimeout`, or background the command and `kill` it from a sleep watchdog | | `seq -s,` | Separator between numbers only | Separator also emitted after the last number, before the terminator | Join after generating (`seq 1 3 \| paste -sd, -`) and count items by lines, never by separators | @@ -59,6 +62,7 @@ General strategy by situation: | Hardcode GNU flags because the script passed in CI | Feature-detect the flavor or install coreutils on macOS | The same script runs on developers' BSD-userland Macs | | Count list items by counting separators plus one | Count items directly (`wc -l` on newline-separated output) | BSD `seq`/`paste` trailing-separator behavior makes separator counts off-by-one | | Port a failing command by trial-and-error flag swaps | Look the command up in the table above, or read both man pages | The differences are systematic, not random — the fix is per-command and known | +| Detect GNU vs BSD `date` by testing whether `%N` prints | Test whether `date +%3N` prints three digits (`[0-9][0-9][0-9]`) | Modern macOS `date` supports `%N` (nanoseconds), so `%N` no longer discriminates; only the `%3N` width form fails there | ## Sources @@ -66,3 +70,5 @@ General strategy by situation: - https://man.freebsd.org/cgi/man.cgi?date(1) — BSD `date -v` adjustment flag - https://man.freebsd.org/cgi/man.cgi?sed(1) — BSD `sed -i` backup-extension argument - https://man.freebsd.org/cgi/man.cgi?seq(1) — BSD `seq -s` separator emitted after the last number (see man-page example) +- https://www.gnu.org/software/coreutils/manual/html_node/Options-for-date.html — GNU `date` `%N` nanoseconds and the numeric field-width form (`%3N` = first 3 digits) +- https://pubs.opengroup.org/onlinepubs/9699919799/functions/strftime.html — an unrecognized strftime conversion specification is undefined; BSD passes `%3N` through literally (field-observed as `3N`) From 90986ba9b40788a5356c0152e8d68d7573399692 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=EC=B5=9C=EC=98=81=EA=B8=B0?= Date: Wed, 15 Jul 2026 15:21:15 +0900 Subject: [PATCH 2/2] knowledge: add cross-check note to ingest report --- .dev-loop/INGEST_REPORT.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index b3415c1..4db639b 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -4,6 +4,13 @@ Drained 3 queued `★ Insight` candidates. Each was independently researched and verified against primary sources before ingest. Two new pages created, one existing page extended. No auto-merge — this PR is for review. +Cross-Check: each of the 3 candidates was verified by an independent research +subagent against primary sources (chart values.yaml / k8s docs / kubernetes-mixin; +Greg's Wiki + pipe(7) + Docker/bash manuals; GNU + FreeBSD date + POSIX strftime). +All three mechanisms VERIFIED; corrections folded in (mixin also pins +`job=cadvisor` not just `image!=""`; process-sub `$!` is bash 4.4+ not 5.1) and +field-observed specifics (OrbStack root cause, literal `3N`) labeled as such. + ## Verified best-practice ### 1. Empty cAdvisor `container_*` metrics on a lightweight kubelet → scrape `/metrics/resource`