Skip to content

BED-8357 Add enterprise + multi-org collection support#6

Merged
d3vzer0 merged 101 commits into
mainfrom
feature/gh-enterprise2
Jun 9, 2026
Merged

BED-8357 Add enterprise + multi-org collection support#6
d3vzer0 merged 101 commits into
mainfrom
feature/gh-enterprise2

Conversation

@d3vzer0

@d3vzer0 d3vzer0 commented May 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Add GitHub Enterprise ingestion with multi-organization support. Additionally, adds new nodes/edges for Github Workflows.

Changes:

  • Add enterprise resource collector for enterprise orgs, members, teams etc.
  • Add org_login/environment id for lookups to support multi-org collection via enterprise credentials
  • Adds a parser for Github workflows with matching against existing repo/org/environment variables and secrets.
  • Additionally fixes an issue when the GraphQL helper returns no pagination metadata while this is required by the Pydantic model
  • Simplifies doc-generation descriptions for node properties via docstrings instead of dataclass field metadata

d3vzer0 added 30 commits May 4, 2026 12:30
@d3vzer0
Added enterprise resources
ee75652
@d3vzer0
Fixed conditions for collecting external identities and runners
d3d4c02
@d3vzer0
Generic try/except is already handled by openhound for resources and …
e7e2d7b 
…transformers
@d3vzer0
Added enterprise nodes and edges
621b4f2
@d3vzer0
Split org/enterprise resource collection
82577c9
@d3vzer0
Restructured enterprise + es org collection to use dlt resources and …
ff95e73 
…transformers
@d3vzer0
Added BaseUser model and removed _flatten_members method in favor of …
0f4abc5 
…in-resource/transformer parsing of users
@d3vzer0
enterprise_teams should be a transformer, not a resource
d100305
@d3vzer0
Improved and added enterprise_role_users + enterprise_role_teams
76f2b68
@d3vzer0
Improved and added enterprise_external_identities + enterprise_saml_p…
dbafa76 
…rovider
@d3vzer0
Iterate org-specific collection for each org_name in ctx
eb58821
@d3vzer0
Refer to org_name instead of ctx.org_name
d588d94
@d3vzer0
Add org-specific lookup logic
6333a6c
@d3vzer0
Updated org collection to use new per-org clients
e65ef57
@d3vzer0
Updated auth to get per-install token
62001b3
@d3vzer0
Else raise error if enterprise_name org org_name are not specified
43d61a3
@d3vzer0
Remove unused imports
02acef4
@d3vzer0
Split enterprise models
fc06796
@d3vzer0
Split enterprise models
5ae2fc4
@d3vzer0
Replaced dataclass field descriptions with attribute descriptions
2bb0c7e
@d3vzer0
Fixed enterprise lookups/edge generation
712ae10
@d3vzer0
Fixed EnterpriseRoleUser slug/id
263507f
@d3vzer0
Split role and EnterpriseAdmin
34b1670
@d3vzer0
Updated justfile for running collect/preproc/convert
5aecdde
@d3vzer0
Updated lock/deps
7462e25
@d3vzer0
Updated schema with missing nodes
3ea773f
@d3vzer0
Add GH_WorkflowJob and GH_WorkflowStep nodes and related edges
c625e0c
@d3vzer0
Add GH_WorkflowJob and GH_WorkflowStep models
5e51741
@d3vzer0
Dont allow extra data
37716c2
@d3vzer0
Collect workflow content + generate jobs/steps
53aab47
@d3vzer0 d3vzer0 marked this pull request as ready for review June 2, 2026 20:54
@d3vzer0 d3vzer0 changed the title DRAFT: BED-8357 Add enterprise + multi-org collection support BED-8357 Add enterprise + multi-org collection support Jun 2, 2026
ktstrader
ktstrader approved these changes Jun 5, 2026
View reviewed changes

@ktstrader ktstrader left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

externalidentities names, display names, and property_matcher values need to be uppercased but other than that it looks good!

ktstrader and others added 13 commits June 8, 2026 09:44
@ktstrader
fix: lowered the expiry date to leave room for clock drift
5e30a38
@d3vzer0
Name matching should be uppercase
8f7b514
@d3vzer0
Name matching should be uppercase (continued)
c3a9750
@d3vzer0
Add org roles to lookup
a3a463d
@d3vzer0
Add dedicated model for enterprise teams
937e367
@d3vzer0
Add dedicated model for enterprise teams
6fa665b
@d3vzer0
Remove uppercase matching for environmentid
2a86576
@d3vzer0
Add dedicated teams collection via rest since enterprise teams are no…
f5c474b 
…t collected via GraphQL
@d3vzer0
Add additional (optional) properties for Github teams
baac6b9
@d3vzer0
Create a unique, per-org node_id for delegated teams
ee8c095
@d3vzer0
Lookup projected enterprise teams based on org login and slug
eb1674d
@d3vzer0
Add projected_enterprise_teams to lookup
3b0aa0d
@d3vzer0
Conditionally create team->team edge if the team has been assigned to…
93037b8 
… a specific github org
@d3vzer0 d3vzer0 merged commit 44264a3 into main Jun 9, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@ktstrader ktstrader ktstrader approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@d3vzer0 @ktstrader