Skip to content

test: add regression coverage for .github/agents/ root-relative import path#25636

Merged
pelikhan merged 2 commits intomainfrom
copilot/fix-agent-imports-path
Apr 10, 2026
Merged

test: add regression coverage for .github/agents/ root-relative import path#25636
pelikhan merged 2 commits intomainfrom
copilot/fix-agent-imports-path

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 10, 2026

The documented .github/agents/planner.md import format (repo-root-relative) was broken in v0.53.3 — the resolver treated it as relative to .github/workflows/, producing "import file not found". The fix for ResolveIncludePath was already present in the codebase, but no test exercised it through the full BFS import processing pipeline (ProcessImportsFromFrontmatterWithSource), leaving the fix without regression coverage.

Changes

  • pkg/parser/import_remote_nested_test.go — adds TestDotGithubAgentsImport with two sub-tests:

    • .github/agents/planner.md — the documented root-relative form
    • /.github/agents/planner.md — the slash-prefixed variant

    Each sub-test verifies the full BFS traversal: path resolves from repo root, file is identified as an agent file, AgentFile is set correctly, and ImportPaths is populated for runtime-import macro generation.

# This now works as documented — no longer requires the ../agents/ workaround
imports:
  - .github/agents/planner.md

Copilot AI changed the title [WIP] Fix agent imports path for workflow files test: add regression coverage for .github/agents/ root-relative import path Apr 10, 2026
Copilot AI requested a review from pelikhan April 10, 2026 12:41
@pelikhan pelikhan marked this pull request as ready for review April 10, 2026 12:43
Copilot AI review requested due to automatic review settings April 10, 2026 12:43
@pelikhan pelikhan merged commit c144ee3 into main Apr 10, 2026
54 checks passed
@pelikhan pelikhan deleted the copilot/fix-agent-imports-path branch April 10, 2026 12:43
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds regression test coverage to ensure .github/agents/... imports are resolved repo-root-relative (and not relative to .github/workflows/) through the full BFS import-processing pipeline.

Changes:

  • Adds TestDotGithubAgentsImport to exercise BFS import traversal for agent-file imports.
  • Covers both documented .github/agents/planner.md and the slash-prefixed /.github/agents/planner.md variant.
  • Asserts AgentFile/ImportPaths are populated for runtime-import macro generation.
Show a summary per file
File Description
pkg/parser/import_remote_nested_test.go Adds regression test for .github/agents/ root-relative import resolution via ProcessImportsFromFrontmatterWithSource.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 1/1 changed files
  • Comments generated: 1

Comment on lines +934 to +937
// The agent file should be detected
assert.NotEmpty(t, result.AgentFile, "AgentFile should be set")
assert.Contains(t, result.ImportPaths, result.AgentFile,
"ImportPaths should contain the agent import path")
Copy link

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the slash-prefixed subtest, the assertions are too weak to act as a regression test for canonicalization: assert.NotEmpty(result.AgentFile) would still pass if the resolver started returning an absolute path or a leading-slash path (which would break runtime $GITHUB_WORKSPACE usage). Consider asserting the exact expected canonical value (e.g. .github/agents/planner.md) and also validating AgentImportSpec preserves the original /.github/... import string, plus that ImportPaths contains the canonical path.

Suggested change
// The agent file should be detected
assert.NotEmpty(t, result.AgentFile, "AgentFile should be set")
assert.Contains(t, result.ImportPaths, result.AgentFile,
"ImportPaths should contain the agent import path")
// The agent file should be detected and canonicalized to the repo-root-relative path
assert.Equal(t, ".github/agents/planner.md", result.AgentFile,
"AgentFile should be canonicalized to the repo-root-relative path")
// Preserve the original import spec while storing the canonical import path for runtime use
assert.Equal(t, "/.github/agents/planner.md", result.AgentImportSpec,
"AgentImportSpec should preserve the original slash-prefixed import")
assert.Contains(t, result.ImportPaths, ".github/agents/planner.md",
"ImportPaths should contain the canonical agent import path")

Copilot uses AI. Check for mistakes.
@github-actions github-actions bot mentioned this pull request Apr 10, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel Report

Test Quality Score: 85/100

Excellent test quality

Metric Value
New/modified tests analyzed 1 (TestDotGithubAgentsImport with 2 sub-tests)
✅ Design tests (behavioral contracts) 1 (100%)
⚠️ Implementation tests (low value) 0 (0%)
Tests with error/edge cases 1 (100%) — slash-prefixed path variant
Duplicate test clusters 0
Test inflation detected N/A (test-only regression PR; no production code changed)
🚨 Coding-guideline violations None

Test Classification Details

Test File Classification Issues Detected
TestDotGithubAgentsImport / root-relative .github/agents/ path resolves as agent file pkg/parser/import_remote_nested_test.go:875 ✅ Design None
TestDotGithubAgentsImport / slash-prefixed /.github/agents/ path also resolves as agent file pkg/parser/import_remote_nested_test.go:904 ✅ Design Minor: assert.NotEmpty is slightly weaker than asserting an exact expected value

Quality Analysis

TestDotGithubAgentsImport (pkg/parser/import_remote_nested_test.go:875)

Design invariant enforced: The test enforces that .github/agents/planner.md (and its slash-prefixed variant /.github/agents/planner.md) import paths resolve from the repository root through ProcessImportsFromFrontmatterWithSource(), correctly setting AgentFile and populating ImportPaths. This directly encodes the user-facing contract described in the imports documentation.

Value if deleted: High — deleting this test would allow the exact regression (.github/agents/ imports silently failing with "import file not found") to re-enter the codebase undetected. This is precisely the scenario described in the PR.

Contract vs. implementation: Design test. The assertions verify observable outputs (AgentFile, ImportPaths) that downstream workflow compilation depends on, not internal function call counts or data structure layout.

Positive signals:

  • ✅ Build tag //go:build !integration present on line 1
  • ✅ No mock libraries — exercises real filesystem and real BFS import traversal
  • ✅ All assertions carry descriptive messages (e.g., "'.github/agents/planner.md' import should resolve successfully")
  • ✅ Two sub-tests cover two path formats: documented form and slash-prefixed variant
  • ✅ Uses require.* appropriately for setup preconditions, assert.* for behavioral verifications
  • ✅ The t.TempDir() + real file writes approach ensures the test is resilient and environment-independent

Minor observation (sub-test 2): assert.NotEmpty(t, result.AgentFile, "AgentFile should be set") is slightly weaker than asserting a specific expected value (as sub-test 1 does with assert.Equal(t, ".github/agents/planner.md", ...)). Since the slash-prefixed form may normalize to the non-prefixed form, the exact normalized value would strengthen this assertion. This is a suggestion only, not a blocker.

Missing coverage (score impact): No explicit negative/error paths are tested (e.g., import path that does not exist at all, malformed path outside the repo root). Adding a case where the agent file is absent would complete the contract. This is the primary reason the score is 85 rather than 100.


Score Calculation

Component Points Earned Max
Behavioral Coverage (100% design tests) 40 40
Error/Edge Case Coverage (slash-prefix variant; no explicit error path) 15 30
Low Duplication (0 duplicate clusters) 20 20
Proportional Growth (test-only PR, inflation check N/A) 10 10
Total 85 100

Language Support

Tests analyzed:

  • 🐹 Go (*_test.go): 1 test function (2 sub-tests) — unit (//go:build !integration)
  • 🟨 JavaScript (*.test.cjs, *.test.js): 0 tests

Verdict

Check passed. 0% of new tests are implementation tests (threshold: 30%). No coding-guideline violations detected. The test is a well-written regression test that enforces a concrete behavioral contract with real component interaction, descriptive assertions, and coverage of two path variants.

Optional improvement: Consider adding a third sub-test for the negative case — e.g., importing a .github/agents/ path whose file does not exist — to verify the resolver returns an appropriate error rather than silently succeeding with an empty AgentFile.


📖 Understanding Test Classifications

Design Tests (High Value) verify what the system does:

  • Assert on observable outputs, return values, or state changes
  • Cover error paths and boundary conditions
  • Would catch a behavioral regression if deleted
  • Remain valid even after internal refactoring

Implementation Tests (Low Value) verify how the system does it:

  • Assert on internal function calls (mocking internals)
  • Only test the happy path with typical inputs
  • Break during legitimate refactoring even when behavior is correct
  • Give false assurance: they pass even when the system is wrong

Goal: Shift toward tests that describe the system's behavioral contract — the promises it makes to its users and collaborators.

🧪 Test quality analysis by Test Quality Sentinel · ● 432.4K ·

Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 85/100. Test quality is excellent — 0% of new tests are implementation tests (threshold: 30%). TestDotGithubAgentsImport is a well-structured regression test that enforces a concrete behavioral contract using real component interactions, all assertions carry descriptive messages, and the //go:build !integration build tag is present. No coding-guideline violations detected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Agent imports: .github/agents/ path does not resolve — must be under .github/workflows/

3 participants