Skip to content

Fetch .ghe.com org member pages in parallel in CI - #319

Open
Ayomi-gh wants to merge 2 commits into
mainfrom
ayomi-gh/ghe-com-parallel-org-members
Open

Ayomi-gh wants to merge 2 commits into
mainfrom
ayomi-gh/ghe-com-parallel-org-members

Conversation

@Ayomi-gh

Copy link
Copy Markdown
Contributor

What

In GitHub Actions, .ghe.com org members are now fetched over REST with up to 8 threads (one page per request). Production, and github.com orgs in CI, are unchanged. Version bumped to 1.2.11.

Why

.ghe.com orgs aren't served by entitlements-caches, so in CI their members come straight from the REST API one page at a time (#318). When a pending change invalidates the predictive cache, reloading a large org takes about 28 s, and in github/entitlements this happens in around half of no-op runs.

The number of requests is the same as before, just made concurrently.

Testing

  • bundle exec rspec spec/unit: 205 examples, 0 failures, 100% coverage
  • bundle exec rubocop lib spec: no offenses
  • Against the github org on github.com: same 5,618 members, 4.8 s parallel vs 32.0 s sequential

In GitHub Actions, .ghe.com orgs use REST because entitlements-caches
does not serve them. Fetching pages one at a time is slow for large orgs,
so fetch them with up to 8 threads. Production is unchanged. Bump version
to 1.2.11.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI auto review requested due to automatic review settings November 10, 2026 18:11

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Responses without a last pagination link can silently omit members, and exhausted retries can leave other workers issuing requests, prolonging rate-limit failures.

2 open issues
What changed in this PR

Speeds up organization membership loading for .ghe.com organizations in GitHub Actions by fetching REST pages concurrently. Production behavior and github.com CI routing remain unchanged.

Changes:

  • Fetches member pages with up to eight workers using separate Octokit clients.
  • Adds coverage for routing, pagination, worker limits, empty organizations, and request failures.
  • Bumps the plugin version to 1.2.11.
File Description
spec/​unit/​entitlements/​service/​github_spec.rb Tests parallel REST fetching and CI-specific routing.
lib/​version.rb Sets the plugin version to 1.2.11.
lib/​entitlements/​service/​github.rb Adds parallel membership pagination and configurable client creation.
Gemfile.lock Synchronizes the locked plugin version.

Review effort: Max (auto)

Comment thread lib/entitlements/service/github.rb Outdated
Comment on lines +346 to +347
last = client.last_response.rels[:last]
last_page = last ? URI.decode_www_form(URI(last.href).query).to_h.fetch("page").to_i : 1
Comment on lines +353 to +366
Thread.new do
Thread.current.report_on_exception = false
thread_client = new_octokit(auto_paginate: false)
members = []
while (page = pages.pop)
members.concat(Retryable.with_context(:default) do
thread_client.organization_members(org, { role:, page: })
end)
end
members
end
end

first_page + threads.flat_map(&:value)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants