Skip to content

Fix incorrect workflow_id type for workflow_run table #558

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cbruno10 merged 1 commit into from
Aug 6, 2026
Merged

Fix incorrect workflow_id type for workflow_run table #558

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions docs/tables/github_actions_repository_workflow_run.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,49 @@ where
repository_full_name = 'turbot/steampipe';
```

### List workflow runs between dates
Identify workflow runs between given dates within the 'turbot/steampipe' repository. This can be useful for tracking workflows over time.

```sql+postgres
select
id,
event,
workflow_id,
conclusion,
status,
run_number,
workflow_url,
head_commit,
head_branch,
created_at
from
github_actions_repository_workflow_run
where
repository_full_name = 'turbot/steampipe'
and created_at >= '2026-01-01'
and created_at <= '2026-02-01';
```

```sql+sqlite
select
id,
event,
workflow_id,
conclusion,
status,
run_number,
workflow_url,
head_commit,
head_branch,
created_at
from
github_actions_repository_workflow_run
where
repository_full_name = 'turbot/steampipe'
and created_at >= '2026-01-01'
and created_at <= '2026-02-01';
```

### List failed workflow runs
Identify instances where workflow runs have failed within the 'turbot/steampipe' repository. This can be useful for debugging and identifying problematic workflows.

Expand Down
2 changes: 1 addition & 1 deletion github/table_github_actions_cache.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ func tableGitHubActionsCache() *plugin.Table {
// Other columns
{Name: "ref", Type: proto.ColumnType_STRING, Description: "The git reference of the cache."},
{Name: "version", Type: proto.ColumnType_STRING, Description: "Hash generated from combination of compression tool, runner OS, and path."},
{Name: "last_accessed_at", Type: proto.ColumnType_TIMESTAMP, Transform: transform.FromField("LastAccessedAt").Transform(convertTimestamp), Description: "Time of the most recent cache access."},
{Name: "last_accessed_at", Type: proto.ColumnType_TIMESTAMP, Transform: transform.FromField("LastAccessedAt").NullIfZero().Transform(convertTimestamp), Description: "Time of the most recent cache access."},
{Name: "created_at", Type: proto.ColumnType_TIMESTAMP, Transform: transform.FromField("CreatedAt").Transform(convertTimestamp), Description: "Time when the cache was created."},
{Name: "size_in_bytes", Type: proto.ColumnType_INT, Description: "Size of the cache in bytes."},
}),
Expand Down
4 changes: 2 additions & 2 deletions github/table_github_actions_repository_workflow_job.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,8 @@ func tableGitHubActionsRepositoryWorkflowJob() *plugin.Table {
{Name: "steps", Type: proto.ColumnType_JSON, Description: "The list of step details for the workflow job."},
{Name: "labels", Type: proto.ColumnType_JSON, Description: "The list of labels for the workflow job."},
{Name: "created_at", Type: proto.ColumnType_TIMESTAMP, Transform: transform.FromField("CreatedAt").Transform(convertTimestamp), Description: "Time when the workflow job was created."},
{Name: "started_at", Type: proto.ColumnType_TIMESTAMP, Transform: transform.FromField("StartedAt").Transform(convertTimestamp), Description: "Time when the workflow job was started."},
{Name: "completed_at", Type: proto.ColumnType_TIMESTAMP, Transform: transform.FromField("CompletedAt").Transform(convertTimestamp), Description: "Time when the workflow job was completed."},
{Name: "started_at", Type: proto.ColumnType_TIMESTAMP, Transform: transform.FromField("StartedAt").NullIfZero().Transform(convertTimestamp), Description: "Time when the workflow job was started."},
{Name: "completed_at", Type: proto.ColumnType_TIMESTAMP, Transform: transform.FromField("CompletedAt").NullIfZero().Transform(convertTimestamp), Description: "Time when the workflow job was completed."},
Comment thread
WallyGuzman marked this conversation as resolved.
{Name: "run_attempt", Type: proto.ColumnType_INT, Description: "The attempt number of the workflow run."},
{Name: "runner_id", Type: proto.ColumnType_INT, Description: "The unique identifier of the workflow job runner."},
{Name: "runner_name", Type: proto.ColumnType_STRING, Description: "The name of the workflow job runner."},
Expand Down
87 changes: 56 additions & 31 deletions github/table_github_actions_repository_workflow_run.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ package github

import (
"context"
"strconv"
"time"

"github.com/google/go-github/v55/github"

Expand All @@ -24,8 +24,11 @@ func tableGitHubActionsRepositoryWorkflowRun() *plugin.Table {
{Name: "workflow_id", Require: plugin.Optional},
{Name: "event", Require: plugin.Optional},
{Name: "head_branch", Require: plugin.Optional},
{Name: "head_sha", Require: plugin.Optional},
{Name: "status", Require: plugin.Optional},
{Name: "conclusion", Require: plugin.Optional},
{Name: "actor_login", Require: plugin.Optional},
{Name: "created_at", Require: plugin.Optional, Operators: []string{">", ">=", "<", "<=", "="}},
},
},
Get: &plugin.GetConfig{
Expand All @@ -40,9 +43,9 @@ func tableGitHubActionsRepositoryWorkflowRun() *plugin.Table {
Columns: commonColumns([]*plugin.Column{
// Top columns
{Name: "repository_full_name", Type: proto.ColumnType_STRING, Transform: transform.FromQual("repository_full_name"), Description: "Full name of the repository that specifies the workflow run."},
{Name: "id", Type: proto.ColumnType_INT, Description: "The unque identifier of the workflow run."},
{Name: "id", Type: proto.ColumnType_INT, Description: "The unique identifier of the workflow run."},
{Name: "event", Type: proto.ColumnType_STRING, Description: "The event for which workflow triggered off."},
{Name: "workflow_id", Type: proto.ColumnType_STRING, Description: "The workflow id of the workflow run."},
{Name: "workflow_id", Type: proto.ColumnType_INT, Description: "The workflow id of the workflow run."},
{Name: "node_id", Type: proto.ColumnType_STRING, Description: "The node id of the workflow run."},
{Name: "conclusion", Type: proto.ColumnType_STRING, Description: "The conclusion for workflow run."},
{Name: "status", Type: proto.ColumnType_STRING, Description: "The status of the workflow run."},
Expand All @@ -67,7 +70,7 @@ func tableGitHubActionsRepositoryWorkflowRun() *plugin.Table {
{Name: "pull_requests", Type: proto.ColumnType_JSON, Description: "The pull request details for the workflow run."},
{Name: "repository", Type: proto.ColumnType_JSON, Description: "The repository info for the workflow run."},
{Name: "run_attempt", Type: proto.ColumnType_INT, Description: "The attempt number of the workflow run."},
{Name: "run_started_at", Type: proto.ColumnType_TIMESTAMP, Transform: transform.FromField("RunStartedAt").Transform(convertTimestamp), Description: "Time when the workflow run was started."},
{Name: "run_started_at", Type: proto.ColumnType_TIMESTAMP, Transform: transform.FromField("RunStartedAt").NullIfZero().Transform(convertTimestamp), Description: "Time when the workflow run was started."},
{Name: "updated_at", Type: proto.ColumnType_TIMESTAMP, Transform: transform.FromField("UpdatedAt").Transform(convertTimestamp), Description: "Time when the workflow run was updated."},
{Name: "actor", Type: proto.ColumnType_JSON, Description: "The user whom initiated the first instance of this workflow run."},
{Name: "actor_login", Type: proto.ColumnType_STRING, Description: "The login of the user whom initiated the first instance of the workflow run.", Transform: transform.FromField("Actor.Login")},
Expand All @@ -85,29 +88,60 @@ func tableGitHubRepoWorkflowRunList(ctx context.Context, d *plugin.QueryData, h
opts := &github.ListWorkflowRunsOptions{
ListOptions: github.ListOptions{PerPage: 100},
}
equalQuals := d.EqualsQuals
if equalQuals["event"] != nil {
if equalQuals["event"].GetStringValue() != "" {
opts.Event = equalQuals["event"].GetStringValue()
}
if event := d.EqualsQualString("event"); event != "" {
opts.Event = event
}
if equalQuals["head_branch"] != nil {
if equalQuals["head_branch"].GetStringValue() != "" {
opts.Branch = equalQuals["head_branch"].GetStringValue()
}
if branch := d.EqualsQualString("head_branch"); branch != "" {
opts.Branch = branch
}
if equalQuals["status"] != nil {
if equalQuals["status"].GetStringValue() != "" {
opts.Status = equalQuals["status"].GetStringValue()
}
if headSha := d.EqualsQualString("head_sha"); headSha != "" {
opts.HeadSHA = headSha
}
if status := d.EqualsQualString("status"); status != "" {
opts.Status = status
}
if actorLogin := d.EqualsQualString("actor_login"); actorLogin != "" {
opts.Actor = actorLogin
}

// Status param can take the value from both status and conclusion column
// https://docs.github.com/en/rest/reference/actions#workflow-runs
if equalQuals["conclusion"] != nil {
// https://docs.github.com/en/rest/actions/workflow-runs#list-workflow-runs-for-a-repository
if conclusion := d.EqualsQualString("conclusion"); conclusion != "" {
if opts.Status == "" {
if equalQuals["conclusion"].GetStringValue() != "" {
opts.Status = equalQuals["conclusion"].GetStringValue()
opts.Status = conclusion
}
}

// Convert quals into GitHub search syntax
// https://docs.github.com/en/search-github/getting-started-with-searching-on-github/understanding-the-search-syntax#query-for-dates
if createdAt := d.Quals["created_at"]; createdAt != nil {
var lowerBound, upperBound time.Time
for _, q := range createdAt.Quals {
t := q.Value.GetTimestampValue().AsTime()
// Note: This logic returns boundary rows regardless of operator, but qual recheck will filter those out client-side
// Keep the _latest_ lower bound and _earliest_ upper bound in case of overlapping filters
switch q.Operator {
case "=":
opts.Created = t.Format(time.DateOnly)
case ">", ">=":
if lowerBound.IsZero() || t.After(lowerBound) {
lowerBound = t
}
case "<", "<=":
if upperBound.IsZero() || t.Before(upperBound) {
upperBound = t
}
}
}
if opts.Created == "" {
var lower, upper = lowerBound.Format(time.RFC3339), upperBound.Format(time.RFC3339)
switch {
case !lowerBound.IsZero() && !upperBound.IsZero():
opts.Created = lower + ".." + upper
case !lowerBound.IsZero():
opts.Created = ">=" + lower
case !upperBound.IsZero():
opts.Created = "<=" + upper
}
}
}
Comment on lines +117 to 147

@cbruno10 cbruno10 Jul 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This range building is now order dependent. The quals arrive in query order, so where created_at <= '2026-02-01' and created_at >= '2026-01-01' builds 2026-02-01T00:00:00Z..2026-01-01T00:00:00Z, a reversed range that returns no rows.

Since the filter is pushed down to the API, postgres never gets the rows back to recheck, so they're silently dropped. Three or more created_at quals would also produce a malformed string like A..B..C.

Can you collect the bounds first, then build the string once? Something like:

if createdAt := d.Quals["created_at"]; createdAt != nil {
	var lower, upper string
	for _, q := range createdAt.Quals {
		t := q.Value.GetTimestampValue().AsTime()
		switch q.Operator {
		case "=":
			opts.Created = t.Format(time.DateOnly)
		case ">", ">=":
			lower = t.Format(time.RFC3339)
		case "<", "<=":
			upper = t.Format(time.RFC3339)
		}
	}
	if opts.Created == "" {
		switch {
		case lower != "" && upper != "":
			opts.Created = lower + ".." + upper
		case lower != "":
			opts.Created = ">=" + lower
		case upper != "":
			opts.Created = "<=" + upper
		}
	}
}

One thing worth a code comment either way: .. and >=/<= are inclusive, so strict >/< quals can return boundary rows from the API. That's fine because Steampipe rechecks quals on the returned rows, but it looks lossy at first glance.

@WallyGuzman WallyGuzman Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This range building is now order dependent. The quals arrive in query order, so where created_at <= '2026-02-01' and created_at >= '2026-01-01' builds 2026-02-01T00:00:00Z..2026-01-01T00:00:00Z, a reversed range that returns no rows. Since the filter is pushed down to the API, postgres never gets the rows back to recheck, so they're silently dropped.

I don't think the order of the quals actually makes a difference and changing it does work in the bare API and in this PR:

Shell example with gh api:

$ gh api repos/turbot/steampipe/actions/runs -X GET -f created=2026-02-01T00:00:00Z..2026-01-01T00:00:00Z -F per_page=1   

{
  "total_count": 44,
  "workflow_runs": [
    {
      "id": 21541410334,
      "name": "30 - Admin: Stale Issues and PRs",
  ....
}

$ gh api repos/turbot/steampipe/actions/runs -X GET -f created=2026-01-01T00:00:00Z..2026-02-01T00:00:00Z -F per_page=1 

{
  "total_count": 44,
  "workflow_runs": [
    {
      "id": 21541410334,
      "name": "30 - Admin: Stale Issues and PRs",
  ...
}

SQL example with these changes:

select
  *
from
  github_actions_repository_workflow_run
where
  repository_full_name = 'turbot/steampipe'
  and created_at <= '2026-02-01T00:00:00Z'
  and created_at >= '2026-01-01T00:00:00Z'
limit 1;

+----------------------+----------------------+----------------+----------+-------------+----------------------------+------------+-----------+------------+--->
| login_id             | repository_full_name | id             | event    | workflow_id | node_id                    | conclusion | status    | run_number | ar>
+----------------------+----------------------+----------------+----------+-------------+----------------------------+------------+-----------+------------+--->
| MDQ6VXNlcjM2MjM0MjY= | turbot/steampipe     | 21,541,410,334 | schedule | 172,982,979 | WFR_kwLOE7GVQM8AAAAFA_fWHg | success    | completed | 209        | ht>
|                      |                      |                |          |             |                            |            |           |            |   >
|                      |                      |                |          |             |                            |            |           |            |   >
|                      |                      |                |          |             |                            |            |           |            |   >
+----------------------+----------------------+----------------+----------+-------------+----------------------------+------------+-----------+------------+--->
(END)

Three or more created_at quals would also produce a malformed string like A..B..C. Can you collect the bounds first, then build the string once? Something like...

Maybe a dumb question, but would we ever use three or more clauses on created_at on the same repository_full_name in a single call? I'm not sure the REST API would support more than two created_at conditions [1] solely with something like the suggestion. Unless you are also referring to changing the below code to make multiple ListWorkflowRuns* calls based on the number of created_at values? In that case I think it probably makes more sense to keep all of the values and build multiple strings so that the individual calls can profit from the opts.Created pushdown.

To be clear, I'm not arguing against this suggestion and I'm happy to incorporate it, but I'm more asking about whether this will work as we expect and push down the opts.Created filter to the underlying call. Otherwise, I think we might need to make a series of calls with the appropriate filters.

One thing worth a code comment either way: .. and >=/<= are inclusive, so strict >/< quals can return boundary rows from the API. That's fine because Steampipe rechecks quals on the returned rows, but it looks lossy at first glance.

This is a good point. I hadn't considered the boundary row condition, but I think the approach of adding/subtracting a second (similar to here [2]) might work, if you agree?

[1] https://docs.github.com/en/search-github/getting-started-with-searching-on-github/understanding-the-search-syntax#query-for-dates

[2] https://github.com/turbot/steampipe-plugin-github/blob/main/github/table_github_commit.go#L77

@cbruno10 cbruno10 Jul 29, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right on the reversed range, thanks for testing that. I reproduced it, both orders return the same 44 results, so GitHub normalizes it (undocumented, but it works).

Looking at the 3 quals issue, I had done some testing and found it also fires with just two. The current code assumes the second qual is always the opposite bound of the first, so two same-direction quals break it. For instance, this where clause:

where created_at >= '2026-01-01' and created_at > '2026-01-15'

Builds 2026-01-01T00:00:00Z..2026-01-15T00:00:00Z, which turns the second lower bound into an upper bound. The API returns runs from Jan 1-15, the recheck requires after Jan 15, and everything gets dropped.

The query silently returns zero rows. Same story with two upper bounds, or an = qual followed by a range qual (2026-01-01..2026-02-01T00:00:00Z, which changes the = semantics).

This shape shows up without anyone typing it, as a view or CTE with a baked-in window (created_at >= now() - interval '30 days') queried with the user's own date filter on top. Postgres doesn't collapse redundant range predicates, so both quals reach the plugin.

(Using 3+ quals also has the same issue today)

For the pushdown question, there's no need for multiple API calls. We can collapse all the quals to a single lower and upper bound, build one range string, and then make one call. The API filter just needs to be a superset of what the quals allow, and the qual recheck drops anything extra.

On the boundary rows, I'd skip the +/- 1 second adjustment from the commit table. It has an edge case for created_at > '2026-01-01T00:00:00.5Z', bumping the bound to 00:00:01.5 excludes a row at 00:00:01 that satisfies the qual, and since that happens API-side the recheck never sees the row. Just use the qual value as an inclusive bound for > and < and let the recheck drop the boundary rows. Small cost of a few extra rows fetched at most and adding a short comment could prevent future confusion.

@WallyGuzman WallyGuzman Jul 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right on the reversed range, thanks for testing that. I reproduced it, both orders return the same 44 results, so GitHub normalizes it (undocumented, but it works).
...
(Using 3+ quals also has the same issue today)

First of all, thank you so much for the feedback! Extremely helpful since I'm not that familiar with Postgres.

Second, this makes *complete sense. I'll make these changes, but probably add a bit of logic to make sure we account for (possibly) redundant filters and reduce rows client-side.

For the pushdown question, there's no need for multiple API calls. We can collapse all the quals to a single lower and upper bound, build one range string, and then make one call. The API filter just needs to be a superset of what the quals allow, and the qual recheck drops anything extra.

Got it. Thanks for clarifying!

I think I did not explain what I meant correctly:
What happens in the case of where ((created_at >= '2020-01-01' and created_at <= '2020-02-01') or (created_at >= '2026-01-01' and created_at <= '2026-02-01'))? From trying this locally, it seems like this completely fails at pushing down the opts.Created filters even though it should be possible? I might be missing something here.

On the boundary rows, I'd skip the +/- 1 second adjustment from the commit table. It has an edge case for created_at > '2026-01-01T00:00:00.5Z', bumping the bound to 00:00:01.5 excludes a row at 00:00:01 that satisfies the qual, and since that happens API-side the recheck never sees the row. Just use the qual value as an inclusive bound for > and < and let the recheck drop the boundary rows. Small cost of a few extra rows fetched at most and adding a short comment could prevent future confusion.

You got it.

Expand All @@ -119,16 +153,7 @@ func tableGitHubRepoWorkflowRunList(ctx context.Context, d *plugin.QueryData, h
}
}

var workflowId int64
if equalQuals["workflow_id"] != nil {
if equalQuals["workflow_id"].GetStringValue() != "" {
workflowId_, err := strconv.ParseInt(equalQuals["workflow_id"].GetStringValue(), 10, 64)
if err != nil {
panic(err)
}
workflowId = workflowId_
}
}
workflowId := d.EqualsQuals["workflow_id"].GetInt64Value()

for {
var (
Expand Down