Skip to content

Split capped PR velocity days by repository#20

Open
mahangu wants to merge 6 commits into
Automattic:mainfrom
mahangu:fix/pr-velocity-repo-day-split
Open

Split capped PR velocity days by repository#20
mahangu wants to merge 6 commits into
Automattic:mainfrom
mahangu:fix/pr-velocity-repo-day-split

Conversation

@mahangu

@mahangu mahangu commented Jun 8, 2026

Copy link
Copy Markdown

Summary

  • add a progressive fallback for pr_velocity queries that still exceed GitHub GraphQL Search's 1,000-result node cap after day slicing
  • split capped windows by repository and, when needed, by created-date ranges
  • keep split count reconciliation fail-fast so incomplete child windows are not accepted silently
  • cover repo listing pagination, capped window fanout, created-date bisection, and reconciliation failures

Rationale

Day slicing alone is not always selective enough for high-volume repositories or organizations. A direct day-to-created-date fallback is smaller, but it can still hit the cap when many PRs share the same created and closed day. The repository step keeps leaf queries smaller while the reconciliation checks preserve fail-fast behavior.

Testing

  • poetry run pytest tap_github_search/tests

@mahangu

mahangu commented Jun 9, 2026

Copy link
Copy Markdown
Author

Updated the PR description with the public rationale: day slicing can still exceed GitHub GraphQL Search's 1,000-result node cap, so the repo split keeps leaf queries smaller while reconciliation keeps incomplete splits fail-fast. Also trimmed duplicate created-date split test setup.

- Remove unused org parameter threaded through _iter_processable_queries
  and _iter_repo_queries_for_capped_day; org is derived from the query regex
- Fix misleading error when a capped day query has no org:/repo: qualifier
- Document why CREATED_SEARCH_START_DATE is 2008-01-01
@mahangu mahangu requested a review from joaopamaral June 9, 2026 08:50
@mahangu mahangu self-assigned this Jun 9, 2026
@mahangu mahangu marked this pull request as ready for review June 9, 2026 08:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant