Attribute PR-comment clicks to the Action - #129
Conversation
The review comment links to the webview, and that traffic arrives unattributable. In September 144 people landed on /owner/repo/pull/N pages but only 27 carried a github.com referrer: referrers get stripped, and links get pasted into Slack. So "a teammate clicked the PR comment" and "someone shared the URL" are indistinguishable, and the Action's contribution to webview usage cannot be measured at all. UTM rather than a bare ?src=, because the webview's PostHog client computes utm_* per event from the URL independently of its autocapture settings, and person_profiles:'always' means anonymous visitors get $initial_utm_* stamped on them. That is exactly the population in question. A custom param would sit only inside $current_url and would need web-app code to promote it; this needs no change on the other side and starts a baseline the day it ships. utm_campaign=gh_action is deliberate: source+medium alone cannot separate the Action from a future bot or GitHub App comment, and a campaign value puts the Action's always-on baseline in the same breakdown as a marketing push. Verified against 6 input shapes (dotted repo names, github.io repos, 5-digit PRs, unresolved merge base, missing artifact URL) and rendered through GitHub's own /markdown API to confirm the href survives encoding intact. Owner and repo names cannot contain ? or #, so the query cannot be swallowed into the path. Nothing parses this URL; the sticky-comment action matches its own HTML marker, not the body. Known conflation: action.yml pipes the same body into $GITHUB_STEP_SUMMARY, so clicks from the Actions run page will also attribute as pr_comment. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CodeBoarding reviewStatus: 0 changed components See the full change in CodeBoarding. graph LR
n_action_scripts["action_scripts"]
classDef added fill:#1f883d,stroke:#0b5d23,color:#ffffff;
classDef modified fill:#bf8700,stroke:#7d4e00,color:#ffffff;
classDef deleted fill:#cf222e,stroke:#82071e,color:#ffffff,stroke-dasharray:5 3;
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b4461aa60d
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| # came from here is a github.com referrer, which most clients strip and a link | ||
| # pasted into chat never had. Constant across runs on purpose: a run id here | ||
| # would scatter one pull request's clicks across a new value per re-run. | ||
| WEBVIEW_URL="https://app.codeboarding.org/${GITHUB_REPOSITORY}/pull/${PR_NUMBER}?utm_source=github&utm_medium=pr_comment&utm_campaign=gh_action" |
There was a problem hiding this comment.
The reviewed commit's subject, Attribute PR-comment clicks to the Action, lacks a feat: or fix: prefix. Because this URL change only reaches Action consumers through a release and release-please ignores unparseable commits when deciding whether to propose one, the attribution functionality can remain unshipped indefinitely; use a release-triggering Conventional Commit subject.
AGENTS.md reference: AGENTS.md:L90-L93
Useful? React with 👍 / 👎.
| # came from here is a github.com referrer, which most clients strip and a link | ||
| # pasted into chat never had. Constant across runs on purpose: a run id here | ||
| # would scatter one pull request's clicks across a new value per re-run. | ||
| WEBVIEW_URL="https://app.codeboarding.org/${GITHUB_REPOSITORY}/pull/${PR_NUMBER}?utm_source=github&utm_medium=pr_comment&utm_campaign=gh_action" |
There was a problem hiding this comment.
When a reader follows this link from the Actions job summary, the visit is still labeled utm_medium=pr_comment. The same generated body is copied into $GITHUB_STEP_SUMMARY at action.yml:648, including on manual dispatches or runs where posting the comment fails, so those visits are provably not PR-comment clicks and will corrupt the attribution this change is intended to measure; generate a summary-specific URL or use a medium that covers both placements.
Useful? React with 👍 / 👎.
ivanmilevtues
left a comment
ivanmilevtues
left a comment
There was a problem hiding this comment.
That is great, we can merge it is the soruce thingy!
2 participants
Why
The review comment links to the webview, and that traffic arrives unattributable.
In September 144 people landed on
/owner/repo/pull/Npages but only 27 carried agithub.comreferrer — referrers get stripped, and links get pasted into Slack. So "a teammate clicked the PR comment" and "someone shared the URL" are indistinguishable, and the Action's contribution to webview usage cannot be measured at all.That handoff is growing fast (1 → 21 → 97 → 144 people, June to September), faster than the Action's own volume. It is the part of the product that is working, and it is invisible on every dashboard.
The change
One line in
scripts/action/build-review-comment.sh:UTM rather than a bare
?src=. The webview's PostHog client setsautocapture: falseandcapture_pageview: false, but neither disables campaign-param capture — posthog-js computesutm_*per event from the URL independently. Combined withperson_profiles: 'always'(deliberate, commented as such in that repo), anonymous visitors get$initial_utm_*stamped on them — exactly the population in question. A custom param would sit only inside$current_url, invisible to PostHog's acquisition breakdowns, and would need web-app code to promote it.utm_campaign=gh_actionis deliberate. Source + medium alone cannot separate the Action from a future bot or GitHub App comment, and having a campaign value puts the Action's always-on baseline in the same breakdown as the imminent marketing push — which is the comparison that matters.The web app needs no change. The route already destructures
searchParamsand ignores unknown keys; App Router matches on pathname only; there is nomiddleware.ts. The onerouter.replacepath is gated behind the OAuth callback and would preserve UTMs anyway.Verification
my.repo_name-v2.js),github.iorepos, 5-digit PRs, unresolved merge base, missing artifact URL — all correct/markdownAPI; the href comes back correctly encoded with all three params intact?or#, so the query can never be swallowed into the pathGITHUB_REPOSITORYis always the base repo on the relevant triggers, andguard.shskips fork PRs unless invoked via a trusted/codeboardingsticky-pull-request-commentmatches its own HTML marker, not the bodyshellcheckclean,bash -ncleanPre-existing failure, not from this change:
test_action_statefails locally onanalyze.sh:127${FORCE_FULL,,}— macOS ships bash 3.2 and that syntax needs bash 4+. Reproduced with a bare one-liner, no CodeBoarding code involved. CI runs ubuntu-latest, where it passes. This diff touches neither file.Known conflation
action.yml:642pipes the same body into$GITHUB_STEP_SUMMARY, so clicks from the Actions run page will also attribute aspr_comment. Low volume; splitting them would mean building the body twice. Flagging rather than fixing.🤖 Generated with Claude Code