Add metrics for measure delete progress - #181
Conversation
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Pull request overview
Adds Prometheus instrumentation to track progress and performance of long-running delete / garbage-collection operations in pkg/proc, along with documentation and unit tests to validate metric behavior.
Changes:
- Introduces new per-bucket/per-operation delete progress and request metrics (
pkg/metrics/delete_metrics.go). - Instruments delete/untrashify handlers to report batch totals, remaining items, processed/deleted/kept counts, and list/delete latency/size.
- Documents the new metrics in
README.mdand adds unit tests asserting the emitted values.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| README.md | Documents new delete/GC metrics and their labels (bucket, operation, stage). |
| pkg/proc/delete_handler.go | Emits delete/untrashify progress + latency/size metrics during handler execution. |
| pkg/proc/delete_handler_test.go | Adds tests to verify metrics are recorded for success/failure/retry paths. |
| pkg/metrics/delete_metrics.go | Defines new Prometheus gauge/counter/histogram vectors and a small tracker helper. |
| go.mod | Promotes github.com/prometheus/client_model to a direct dependency for test metric inspection. |
Comments suppressed due to low confidence (1)
pkg/proc/delete_handler.go:147
- On a dry-run (
Confirm == false),delete_process_remainingis left atlen(fileList)even though the handler exits immediately. This can make operational dashboards/alerts think a delete is stuck.
Consider resetting delete_process_remaining before returning.
if !msg.Confirm { // Do not delete files if no confirmation flag provided
ylogger.Zero.Info().Str("bucket", bucket).Msg("do not perform actual delete files as no confirmation flag provided")
return nil
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This comment has been minimized.
This comment has been minimized.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
@check-spelling-bot Report🔴 Please reviewSee the 📂 files view, the 📜action log, 👼 SARIF report, or 📝 job summary for details.Unrecognized words (3)dto These words are not needed and should be removedtestutils VecTo accept these unrecognized words as correct and remove the previously acknowledged and now absent words, you could run the following commands... in a clone of the git@github.com:open-gpdb/yproxy.git repository curl -s -S -L 'https://raw.githubusercontent.com/check-spelling/check-spelling/main/apply.pl' |
perl - 'https://github.com/open-gpdb/yproxy/actions/runs/30374199288/attempts/1' &&
git commit -m 'Update check-spelling metadata'OR To have the bot accept them for you, comment in the PR quoting the following line: Available 📚 dictionaries could cover words (expected and unrecognized) not in the 📘 dictionaryThis includes both expected items (269) from .github/actions/spelling/expect.txt and unrecognized words (3)
Consider adding them (in cspell:golang/dict/go.txt
cspell:node/dict/node.txt
cspell:php/dict/php.txt
cspell:python/src/python/python-lib.txt
cspell:dotnet/dict/dotnet.txt
cspell:filetypes/filetypes.txt
cspell:fullstack/dict/fullstack.txt
cspell:java/src/java.txt
cspell:python/src/python/python.txt
cspell:typescript/dict/typescript.txt
cspell:k8s/dict/k8s.txt
cspell:r/src/r.txt
cspell:django/dict/django.txt
cspell:aws/aws.txt
cspell:cpp/src/stdlib-c.txt
cspell:rust/dict/rust.txt
cspell:scala/dict/scala.txt
cspell:cpp/src/stdlib-cpp.txt
cspell:npm/dict/npm.txt
cspell:lua/dict/lua.txtTo stop checking additional dictionaries, add (in check_extra_dictionaries: ""Pattern suggestions ✂️ (11)You could add these patterns to Alternatively, if a pattern suggestion doesn't make sense for this project, add a Notices ℹ️ (2)See the 📂 files view, the 📜action log, 👼 SARIF report, or 📝 job summary for details.
See ℹ️ Event descriptions for more information. If the flagged items are 🤯 false positivesIf items relate to a ...
🚂 If you're seeing this message and your PR is from a branch that doesn't have check-spelling, |
Here we add prometheus metrics to measure deletion progress.
The main metrics are DeleteProcessTotal && DeleteProcessProcessed. They are operation-specific, reported after completed listing operation.
To measure we instrument each operation by specific prometheus object (from promauto space) and increment metrics on each completed operation. The same for the operation latency