Skip to content

feat(ibc): migrate sei-ibc-go metrics to OpenTelemetry with dual emission#3543

Open
amir-deris wants to merge 6 commits into
mainfrom
amir/plt-427-migrate-sei-ibc-go-to-otel
Open

feat(ibc): migrate sei-ibc-go metrics to OpenTelemetry with dual emission#3543
amir-deris wants to merge 6 commits into
mainfrom
amir/plt-427-migrate-sei-ibc-go-to-otel

Conversation

@amir-deris

@amir-deris amir-deris commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds OpenTelemetry instrumentation to sei-ibc-go as part of the broader OTel metrics migration (PLT-427/PLT-428). Follows the same dual-emit pattern established in PLT-330 (x/evm) and PLT-353 (sei-cosmos): legacy telemetry calls are preserved alongside new OTel instruments until dashboards are verified and legacy can be removed.

Changes

5 new metrics.go files — each initializes an OTel meter and registers instruments:

Package Meter Instruments
apps/transfer/keeper ibc_transfer tx_msg_ibc_transfer (gauge), ibc_transfer_packet_receive (gauge), ibc_transfer_send (counter), ibc_transfer_receive (counter)
core/keeper ibc_core tx_msg_ibc_recv_packet, ibc_timeout_packet, tx_msg_ibc_acknowledge_packet (counters)
core/02-client/keeper ibc_client ibc_client_create, ibc_client_update, ibc_client_upgrade, ibc_client_misbehaviour (counters)
core/03-connection/keeper ibc_connection ibc_connection_open_{init,try,ack,confirm} (counters)
core/04-channel/keeper ibc_channel ibc_channel_{open,close}_{init,try,ack,confirm} (counters)

11 source files modified — dual-emit added at existing telemetry call sites:

  • transfer/keeper/relay.gosendTransfer, OnRecvPacket
  • core/keeper/msg_server.goRecvPacket, Timeout, TimeoutOnClose, Acknowledgement
  • core/02-client/keeper/client.goCreateClient, UpdateClient, UpgradeClient, CheckMisbehaviourAndUpdateState
  • core/02-client/keeper/proposal.goClientUpdateProposal
  • core/03-connection/keeper/handshake.goConnOpenInit/Try/Ack/Confirm
  • core/04-channel/keeper/handshake.goChanOpenInit/Try/Ack/Confirm, ChanCloseInit/Confirm

Legacy calls are marked TODO(PLT-428) for removal once OTel instruments are verified in production dashboards.

@cursor

cursor Bot commented Jun 3, 2026

Copy link
Copy Markdown

PR Summary

Low Risk
Observability-only changes alongside preserved legacy metrics; no IBC state machine or token logic altered beyond metric recording in defer blocks.

Overview
Adds OpenTelemetry metrics across sei-ibc-go while keeping existing armon/go-metrics telemetry in place until PLT-428 validation (dual emit).

New metrics.go files register OTel meters/instruments for transfer (send/receive counts and token amount gauges with denom_class), core packet handlers (recv, timeout, ack), client lifecycle (create/update/upgrade/misbehaviour), connection and channel handshake steps. Instrumentation is wired in existing defer blocks next to the legacy counters/gauges, with matching attributes (ports, channels, client type/ID, timeout type, source-chain flag). Transfer send path adds a source string solely to label OTel send metrics.

Reviewed by Cursor Bugbot for commit f416fdd. Bugbot is set up for automated code reviews on this repo. Configure here.

@amir-deris amir-deris changed the title Added new metrics and dual emission for sei-ibc feat(ibc): migrate sei-ibc-go metrics to OpenTelemetry with dual emission Jun 3, 2026
@github-actions

github-actions Bot commented Jun 3, 2026

Copy link
Copy Markdown

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedJun 8, 2026, 11:26 PM

Comment thread sei-ibc-go/modules/apps/transfer/keeper/metrics.go
Comment thread sei-ibc-go/modules/apps/transfer/keeper/relay.go Outdated
Comment thread sei-ibc-go/modules/core/02-client/keeper/client.go
@codecov

codecov Bot commented Jun 3, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.09901% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 58.29%. Comparing base (1a1b8cd) to head (f416fdd).
⚠️ Report is 13 commits behind head on main.

Files with missing lines Patch % Lines
sei-ibc-go/modules/apps/transfer/keeper/metrics.go 50.00% 1 Missing and 1 partial ⚠️
...ei-ibc-go/modules/core/02-client/keeper/metrics.go 50.00% 1 Missing and 1 partial ⚠️
...bc-go/modules/core/03-connection/keeper/metrics.go 50.00% 1 Missing and 1 partial ⚠️
...i-ibc-go/modules/core/04-channel/keeper/metrics.go 50.00% 1 Missing and 1 partial ⚠️
sei-ibc-go/modules/core/keeper/metrics.go 50.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3543      +/-   ##
==========================================
- Coverage   59.12%   58.29%   -0.84%     
==========================================
  Files        2218     2145      -73     
  Lines      183132   174524    -8608     
==========================================
- Hits       108285   101743    -6542     
+ Misses      65082    63730    -1352     
+ Partials     9765     9051     -714     
Flag Coverage Δ
sei-chain-pr 87.79% <90.09%> (?)
sei-db 70.41% <ø> (ø)
sei-db-state-db ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
sei-ibc-go/modules/apps/transfer/keeper/relay.go 91.26% <100.00%> (+0.88%) ⬆️
sei-ibc-go/modules/core/02-client/keeper/client.go 98.64% <100.00%> (+0.19%) ⬆️
...i-ibc-go/modules/core/02-client/keeper/proposal.go 88.88% <100.00%> (+1.13%) ⬆️
...-go/modules/core/03-connection/keeper/handshake.go 86.82% <100.00%> (+0.32%) ⬆️
...ibc-go/modules/core/04-channel/keeper/handshake.go 89.53% <100.00%> (+0.23%) ⬆️
sei-ibc-go/modules/core/keeper/msg_server.go 69.64% <100.00%> (+2.30%) ⬆️
sei-ibc-go/modules/apps/transfer/keeper/metrics.go 50.00% <50.00%> (ø)
...ei-ibc-go/modules/core/02-client/keeper/metrics.go 50.00% <50.00%> (ø)
...bc-go/modules/core/03-connection/keeper/metrics.go 50.00% <50.00%> (ø)
...i-ibc-go/modules/core/04-channel/keeper/metrics.go 50.00% <50.00%> (ø)
... and 1 more

... and 104 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread sei-ibc-go/modules/core/keeper/metrics.go Outdated
Comment thread sei-ibc-go/modules/apps/transfer/keeper/metrics.go
@amir-deris amir-deris requested review from bdchatham and masih June 3, 2026 17:47

@bdchatham bdchatham left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with one Cursor nit

Comment thread sei-ibc-go/modules/apps/transfer/keeper/metrics.go

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit f416fdd. Configure here.

"ibc_transfer_packet_receive",
metric.WithDescription("Total amount of tokens received in IBC packet"),
metric.WithUnit("{token}"),
)),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gauge metrics imply cumulative totals

Medium Severity

The new ibc_transfer_tx_msg and ibc_transfer_packet_receive instruments are Int64Gauge values that record the latest transfer amount per denom_class, but their names and descriptions read like cumulative totals. That diverges from other OTel amount gauges here (last_send_amount, last_withdraw_reward_amount) and can mislead new dashboards built on these series.

Fix in Cursor Fix in Web

Triggered by learned rule: OTel metrics: guard attribute cardinality and use native types

Reviewed by Cursor Bugbot for commit f416fdd. Configure here.

ibcTransferMetrics.ibcTransferSend.Add(ctx.Context(), 1, otelmetric.WithAttributes(
attribute.String(coretypes.LabelDestinationPort, destinationPort),
attribute.String(coretypes.LabelDestinationChannel, destinationChannel),
attribute.String(coretypes.LabelSource, source),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why string label for a boolean value?

}

ibcTransferMetrics.ibcTransferSend.Add(ctx.Context(), 1, otelmetric.WithAttributes(
attribute.String(coretypes.LabelDestinationPort, destinationPort),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Port is numeric? Could you review all attribute types relative to what they measure please?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants