Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
677b52d
Implement token-saving skill and related evaluation files; update REA…
miroslavpojer May 20, 2026
e709f12
Add title parameter to release notes presence check in PR workflow
miroslavpojer May 20, 2026
f653b27
Update documentation for token-saving skill; add contribution guideli…
miroslavpojer May 21, 2026
fd9f957
Add token-saving skill documentation with response formatting rules
miroslavpojer May 21, 2026
b0eb566
Initial design, review of skill.md files not finished.
miroslavpojer May 21, 2026
dcbfcb5
Update links to living-doc-glossary in various skill documents and ag…
miroslavpojer May 22, 2026
15b5cec
Update links to living-doc-glossary in SKILL.md files and remove obso…
miroslavpojer May 22, 2026
9aab9d1
Add living doc automation scripts for ID assignment, gap detection, i…
miroslavpojer May 22, 2026
c172270
feat: add script to scan .feature files for AC link compliance
miroslavpojer May 22, 2026
ae2fa98
feat: enhance documentation with new skills and update references
miroslavpojer May 22, 2026
313385e
feat: remove outdated implementation roadmap from the repository
miroslavpojer May 22, 2026
8e33c87
Refactor Gherkin step definitions and Living Doc skills
miroslavpojer May 22, 2026
c63f2dd
feat: update Gherkin step definitions and enhance living documentatio…
miroslavpojer May 22, 2026
b983ef6
Add evaluation configurations for Gherkin and PageObject skills
miroslavpojer May 22, 2026
21503cd
feat: enhance Gherkin-Living Doc sync skill with improved description…
miroslavpojer May 22, 2026
ea9b7c8
Refactor living documentation skills for improved clarity and functio…
miroslavpojer May 22, 2026
d13a22f
feat: add evals and trigger evaluations for living-doc and living-doc…
miroslavpojer May 22, 2026
4d94d28
chore: remove roadmap.md as part of project restructuring
miroslavpojer May 22, 2026
8ad9e59
tmp
miroslavpojer May 25, 2026
6ac9ff3
Remove tmp data.
miroslavpojer May 26, 2026
949234b
Updated form Unify project integration.
miroslavpojer May 27, 2026
337bcfd
Updated evals and finished round 1 of trigger evals testing.
miroslavpojer May 27, 2026
e4cef54
Update living documentation agent description and triggers for clarit…
miroslavpojer May 27, 2026
9edf2eb
Update .gitignore and enhance living documentation skills with new co…
miroslavpojer May 27, 2026
8e7ba76
Enhance evals with improved output formatting and additional queries …
miroslavpojer May 27, 2026
17baf4e
Backup from integration
miroslavpojer May 30, 2026
3bdc5c2
partial tune status backup
miroslavpojer May 30, 2026
3fd7e50
Skill gap reduction.
miroslavpojer May 30, 2026
b45c00a
Final fix of gaps and issues.
miroslavpojer May 30, 2026
4f825bc
Test case review.
miroslavpojer May 31, 2026
fafcd64
Tested agent and improved testing doc.
miroslavpojer May 31, 2026
d4462a7
Skill test - round 1.
miroslavpojer May 31, 2026
c615596
Update SKILL.md files to enhance trigger phrases and improve descript…
miroslavpojer May 31, 2026
c70f8ca
Worked in changed from weakness review.
miroslavpojer May 31, 2026
f3c88e3
Testing round 2.
miroslavpojer May 31, 2026
2c2d495
Update skills from adoption pilot.
miroslavpojer Jul 13, 2026
c78364d
improved evals
miroslavpojer Jul 13, 2026
14f5b6a
Minimize skill text size — preserve information, reduce verbosity
miroslavpojer Jul 13, 2026
46447da
tuned skills according exising evals
miroslavpojer Jul 13, 2026
561c1ba
Merge master into feature/7-living-doc-copilot
miroslavpojer Jul 14, 2026
40a13a4
Merge branch 'master' into feature/7-living-doc-copilot
miroslavpojer Jul 14, 2026
9dd739c
Refactor living documentation and BDD copilot integration; update REA…
miroslavpojer Jul 14, 2026
ac52bfe
Update living documentation and BDD copilot integration; enhance READ…
miroslavpojer Jul 14, 2026
f9aaaf0
Add living documentation schemas and glossary for BDD framework
miroslavpojer Jul 14, 2026
39c0299
Refactor schema paths in JSON files and update file handling in Pytho…
miroslavpojer Jul 14, 2026
74d7bf2
Update schema paths and enhance manifest description for clarity
miroslavpojer Jul 14, 2026
0dc583c
Enhance regex patterns for AC IDs and feature IDs; update living docu…
miroslavpojer Jul 14, 2026
6a354b9
Enhance regex for AC IDs in coverage report; improve ID extraction lo…
miroslavpojer Jul 14, 2026
aa1081a
Update ID format in coverage report; change acceptance criteria ID to…
miroslavpojer Jul 14, 2026
7f9d7c7
Update trigger phrases and documentation for clarity; fix script usag…
miroslavpojer Jul 14, 2026
05b47b0
Refine descriptions for living documentation and BDD automation; stan…
miroslavpojer Jul 14, 2026
976b90d
Normalize acceptance criteria state values and enhance documentation …
miroslavpojer Jul 14, 2026
04cd211
Clarify agent invocation instructions and update documentation for ag…
miroslavpojer Jul 14, 2026
ccd9df7
Enhance AC ID handling in coverage report; add canonicalization for n…
miroslavpojer Jul 14, 2026
acb2cf9
Refine AC ID format handling in scripts; update regex patterns for co…
miroslavpojer Jul 14, 2026
cbbe72f
Enhance token-saving skill and documentation
miroslavpojer Jul 14, 2026
5e9b25b
Refactor Feature ID handling to enforce numeric-only convention
miroslavpojer Jul 14, 2026
30ec29e
Standardize AC state terminology to Title-case across documentation a…
miroslavpojer Jul 14, 2026
c8d551b
Refactor AC state handling to use lowercase with underscores across t…
miroslavpojer Jul 14, 2026
397d527
Update references to living-doc schemas and glossary across multiple …
miroslavpojer Jul 14, 2026
8083c10
Update README and schema files for clarity; refine feature ID pattern…
miroslavpojer Jul 15, 2026
555979a
Update feature IDs to numeric-only format across various skills and d…
miroslavpojer Jul 15, 2026
8922aa0
Enhance living_doc_id.py documentation to clarify ID conventions; enf…
miroslavpojer Jul 15, 2026
4b2a288
Refactor file search in dead-code detectors to use recursive globbing…
miroslavpojer Jul 15, 2026
df55e38
Update slug-based FEAT ID tests to improve error message clarity and …
miroslavpojer Jul 15, 2026
e035326
Final adjustment using local pr-review skill.
miroslavpojer Jul 15, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added .DS_Store
Binary file not shown.
355 changes: 355 additions & 0 deletions .github/agents/evals/living-doc-bdd-copilot/evals.json

Large diffs are not rendered by default.

27 changes: 27 additions & 0 deletions .github/agents/evals/living-doc-bdd-copilot/fixture-map.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Fixture Map — living-doc-bdd-copilot agent evals

## Eval coverage summary

| Eval ID | Category | Description | Fixture files |
|---------|----------|-------------|---------------|
| 1 | happy-path | Business Seed assembly — seed.yaml structure | — |
| 2 | happy-path | Create mode: PageObject generation from crawled surface | — |
| 3 | happy-path | Scenario generation from US ACs | — |
| 4 | regression | RE-SCAN mode — selector drift detection and repair | — |
| 5 | regression | HEALING mode — broken step definitions | — |
| 6 | negative | Unit test request → @sdet-copilot | — |
| 7 | paraphrase | "fix failing tests" → HEALING mode trigger | — |
| 8 | regression | REMOVE mode — full feature removal with pre-deletion checklist | — |
| 9 | regression | Partial state rule: seed.yaml present, manifest.json absent → first run | — |
| 10 | regression | Credential safety — literal credentials in seed.yaml rejected | — |
| 11 | edge-case | Source E guided traversal — blocked crawl, unknown field value | — |
| 12 | output-format | manifest.json entry structure for a scanned route | — |
| 25 | negative | Unit test request → decline + direct to @sdet-copilot | — |

## Trigger eval summary

| Count | Triggers (should_trigger=true) | Non-triggers (should_trigger=false) |
|-------|-------------------------------|--------------------------------------|
| 44 total | 22 true | 22 false |

> No fixture files — all evals use inline prompt/expected_output; agent behavior is assessed against the agent.md operating rules and skill definitions.
321 changes: 321 additions & 0 deletions .github/agents/living-doc-bdd-copilot.agent.md

Large diffs are not rendered by default.

8 changes: 8 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Copilot Instructions

## Always Active

The **token-saving** skill is always active in this workspace. Every response applies concise formatting rules and removes noise (no filler, no closing platitudes, structured output).

See [Token-Saving Skill](../docs/guides/token-saving.md) for details, including override behavior when you explicitly request depth.

---

## PR Review

When reviewing a pull request, load and apply:
Expand Down
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
**/__pycache__/*
**/.DS_Store
skills-eval-workspace/*
agents-eval-workspace/*
.playwright-mcp/*
25 changes: 23 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,14 @@ skills/
| `scripts/` | Deterministic or repetitive logic better run as code than described in prose (e.g. a validation script, a formatter, a data transformer) |
| `references/` | Domain docs, API specs, decision tables, or anything too large to keep in `SKILL.md` without exceeding 500 lines |
| `assets/` | Template files, example inputs/outputs, icons — anything the skill produces or consumes |
| `evals/` | Test prompts and assertions to verify skill behavior and trigger accuracy. See [skill-testing.md](./docs/skill-testing.md) |
| `evals/` | Test prompts and assertions to verify skill behavior and trigger accuracy. See [skill-testing.md](./docs/testing/skill-testing.md) |

#### `references/` — shared vs. skill-specific

- **`skills/<skill>/references/`** — skill-specific docs loaded only by that skill's instructions
- **`skills/shared/references/`** — shared docs used by multiple skills (e.g., schemas, glossaries, common API specs)

If a reference file is used by 2+ skills, place it in `skills/shared/references/` instead of duplicating it. See [living-doc-bdd-schemas.md](skills/shared/references/living-doc-bdd-schemas.md) and [living-doc-glossary.md](skills/shared/references/living-doc-glossary.md) for examples.
---

## 2. Frontmatter schema
Expand Down Expand Up @@ -177,6 +183,21 @@ should be run") — it is clearer and easier for the agent to follow.

If every test run of your skill independently writes the same helper script (a formatter, a validator, a transformer), bundle it in `scripts/` and reference it from `SKILL.md`. This saves every future invocation from reinventing the wheel.

### Shared utilities and assets across multiple skills

**Pattern:** If code, references, or assets are used by 2+ skills, place them in `skills/shared/`:

- **`skills/shared/lib/`** for shared Python modules → each skill has a wrapper
- **`skills/shared/references/`** for shared documentation → skills link directly
- **`skills/shared/assets/`** for shared templates/data → skills reference relative paths

Examples:
- `skills/shared/lib/living_doc_id.py` — shared by three living-doc skills
- `skills/shared/references/living-doc-glossary.md` — shared by multiple living-doc skills
- See [skills/shared/lib/README.md](skills/shared/lib/README.md) for implementation details

**Benefits:** Single source of truth, prevents drift, easier maintenance, each skill remains independently deployable.

### Format conventions

- Use `##` and `###` headings to structure the body
Expand All @@ -201,7 +222,7 @@ If every test run of your skill independently writes the same helper script (a f
Before proposing a PR, verify that your skill activates correctly and produces good output. The full testing
methodology — eval creation, fixture management, with/without comparisons, trigger testing, and description
optimization using the Anthropic [`skill-creator`](https://github.com/anthropics/skills/tree/main/skills/skill-creator)
skill — is covered in **[docs/skill-testing.md](./docs/skill-testing.md)**.
skill — is covered in **[docs/testing/skill-testing.md](./docs/testing/skill-testing.md)**.

---

Expand Down
26 changes: 25 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,9 +77,31 @@ its purpose, trigger phrases, and full instructions.

| Skill | Description |
|------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------|
| **[living-doc-create-user-story](./skills/living-doc-create-user-story/)** | Create a well-formed User Story with business-level Acceptance Criteria that are traceable, testable, and E2E-ready. |
| **[living-doc-create-feature](./skills/living-doc-create-feature/)** | Document a system surface (UI screen, API endpoint, service) as a Feature entity with ownership and traceability links. |
| **[living-doc-create-functionality](./skills/living-doc-create-functionality/)** | Define an atomic, testable behaviour (Functionality) with AC designed for fast unit or integration tests. |
| **[living-doc-update](./skills/living-doc-update/)** | Amend or deprecate existing User Story, Feature, or Functionality entities — add ACs, change status, update ownership. |
| **[living-doc-impact-analysis](./skills/living-doc-impact-analysis/)** | Trace which Features, Functionalities, User Stories, and Gherkin scenarios are affected by a code change or PR. |
| **[living-doc-gap-finder](./skills/living-doc-gap-finder/)** | Identify undocumented behaviours, orphan tests, and untested ACs. Used by `@living-doc-bdd-copilot` top-down (catalog gaps) and bottom-up (scenario coverage). |
| **[living-doc-pageobject-scan](./skills/living-doc-pageobject-scan/)** | Discover, create, and maintain PageObject classes from a live web application — bootstrapping from scratch and detecting selector drift after UI changes. |
| **[bdd-maintain](./skills/bdd-maintain/)** | REMOVE and DEAD CODE AUDIT modes — delete feature files, step definitions, and PageObjects linked to deprecated entities; audit unused steps and PageObject methods. |
| **[data-cy-instrument](./skills/data-cy-instrument/)** | Resolve missing `data-cy` attributes in Angular component templates and sync PageObjects to use `getByTestId()` — run after a crawl when `coverage_gaps` are non-empty. |
| **[living-doc-scenario-creator](./skills/living-doc-scenario-creator/)** | Generate full Gherkin feature files from User Story and Functionality ACs — feature file header, @AC:-tagged scenarios, complete Given/When/Then step bodies, coverage report, and step definition resolution. |
| **[gherkin-step](./skills/gherkin-step/)** | Implement clean, reusable step definitions — behave (Python), Cucumber (Java, TypeScript, Scala), parameter types, DataTable, DocString, and hooks. |
| **[gherkin-living-doc-sync](./skills/gherkin-living-doc-sync/)** | Synchronise Gherkin feature files with the living documentation catalog — fix missing AC traceability headers, step text drift, and stale scenario links. |
Comment thread
miroslavpojer marked this conversation as resolved.
| **[pr-review](./skills/pr-review/)** | Pull request code review — reviews diffs for risk, security issues, API contract changes, dependency bumps, CI/CD and infrastructure changes. Produces concise Blocker / Important / Nit comments. |
| **[token-saving](./skills/token-saving/)** | Always-active response discipline — enforces brevity, no filler openers or closers, structured output, and a What/Why/How footer on code responses. Suspends on explicit "full detail" requests. |

## Agent Roster

Agents are pre-configured AI personas that orchestrate multiple skills for a specific engineering phase. Agent files live in **[.github/agents/](./.github/agents/)**.



| Agent | Description |
|---|---|
| **[@living-doc-bdd-copilot](./.github/agents/living-doc-bdd-copilot.agent.md)** | Living documentation and BDD automation — catalog management (User Stories, Features, Functionalities, AC updates, impact analysis, gap finding) and automation (webapp exploration, PageObjects, Gherkin scenarios, step definitions, BDD suite maintenance). |
Comment thread
miroslavpojer marked this conversation as resolved.
Comment thread
miroslavpojer marked this conversation as resolved.
Comment thread
miroslavpojer marked this conversation as resolved.

Comment thread
miroslavpojer marked this conversation as resolved.
## Finding More Skills

Before building a new skill, check whether one already exists:
Expand All @@ -89,11 +111,13 @@ Before building a new skill, check whether one already exists:
| [github/awesome-copilot](https://github.com/github/awesome-copilot/tree/main/skills) | 200+ community skills: cloud, languages, security, DevOps, productivity |
| [skills.sh](https://skills.sh) | Open registry — install with `npx skills add <owner/repo>` |
| [anthropics/skills](https://github.com/anthropics/skills) | Anthropic reference skills including `skill-creator` |
| [absa-group/agent-skills](https://github.com/absa-group/agent-skills) | Broader ABSA-owned skill collection |
| [absa-group/cps-agentic-toolkit](https://github.com/absa-group/cps-agentic-toolkit) | CPS team's extended skill set (ABSA-internal) |

## Contributing

See **[CONTRIBUTING.md](./CONTRIBUTING.md)** for the skill authoring guide — folder layout, frontmatter schema, writing
effective descriptions and bodies, [testing](./docs/skill-testing.md), and the PR checklist.
effective descriptions and bodies, [testing](./docs/testing/skill-testing.md), and the PR checklist.

To propose a new skill — or to propose expanding the repo into agents, MCP servers, or
plugins — [open an issue](https://github.com/AbsaOSS/agentic-toolkit/issues/new).
Expand Down
33 changes: 24 additions & 9 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,24 @@

Navigation hub for all guides in this repository. Browse by category below.

## Contents

- [Setup & Repository Guides](#setup--repository-guides)
- [Tools for Agentic Development](#tools-for-agentic-development)
- [Skill Guides](#skill-guides)
- [Agent Guides](#agent-guides)

## Setup & Repository Guides

| Guide | Description |
|----|----|
| [Getting Started](./getting-started.md) | What skills are, how to install them, Copilot CLI usage |
| Guide | Description |
|-----------------------------------------|-------------------------------------------------------------------------------------|
| [Getting Started](./getting-started.md) | What skills are, how to install them, Copilot CLI usage |
| [Contributing](../CONTRIBUTING.md) | Skill folder layout, frontmatter, description writing, body guidelines, PR process |
| [Skill Testing](./skill-testing.md) | Eval creation, fixtures, regression loops, trigger and description optimization |
| [Troubleshooting](./troubleshooting.md) | Setup fixes for install, activation, and proxy issues |
| [Responsible Agent Use](./responsible-agent-use.md) | Not burning your Copilot token budget — context, models, agent mode, MCP, plugins, skills, and a must-do checklist |
| [Agent Design Best Practices](./guides/agent-design.md) | Core principles, file structure, context management, tool guidance, examples, and stopping conditions for `.agent.md` files |
| [Skill Testing](./testing/skill-testing.md) | Eval creation, fixtures, regression loops, trigger and description optimization |
| [Agent Testing](./testing/agent-testing.md) | Eval creation and body quality testing for `.agent.md` files — agents are invoked by @-mention, not auto-triggered |
| [Troubleshooting](./troubleshooting.md) | Setup fixes for install, activation, and proxy issues |

## Tools for Agentic Development

Expand All @@ -22,11 +32,16 @@ Navigation hub for all guides in this repository. Browse by category below.

## Skill Guides

| Guide | Description |
|----|----|
| Guide | Description |
|-------------------------------------|------------------------------------------------------------------------------------|
| [PR Review](./pr-review.md) | How the PR review skill works, what sections it applies, and how to trigger it |
| [Token Saving](./token-saving.md) | Keeping AI responses concise — how the token-saving skill works and when it applies |
| [Responsible Agent Use](./responsible-agent-use.md) | Not burning your Copilot token budget — context, models, agent mode, MCP, plugins, skills, and a must-do checklist |
| [Token Saving](./guides/token-saving.md) | Keeping AI responses concise — how the token-saving skill works and when it applies |

## Agent Guides

| Guide | Description |
|-----------------------------------------------|-------------------------------------------------------------------------|
| [Living Doc BDD Copilot](./guides/living-doc-bdd-copilot.md) | Living documentation and BDD automation — catalog management (User Stories, Features, Functionalities, AC updates, impact analysis, gap finding) and automation (webapp exploration, PageObjects, Gherkin, step definitions, maintenance) |
Comment thread
miroslavpojer marked this conversation as resolved.

> **Keep this index up to date.** When you add a new guide, add a row to the appropriate table above.

Expand Down
49 changes: 49 additions & 0 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,55 @@ your-project-repo/
└── SKILL.md
```

## Install Agents

Agents are pre-configured AI personas that orchestrate multiple skills for a specific engineering phase. Unlike skills (which are auto-activated by description matching), agents are invoked explicitly by name in Copilot Chat.

### How agents differ from skills

| | Skills | Agents |
|---|---|---|
| Activation | Auto — triggered when your prompt matches the description | Manual — you @-mention the agent name |
| Scope | Any compatible tool (Copilot, Claude, Cursor) | GitHub Copilot Chat (VS Code) |
| Install location | `~/.agents/skills/` or `.github/skills/` | `.github/agents/` inside your project repo |

### Install an agent into your project

Copy the agent file into your project's `.github/agents/` directory:

```bash
# One-time setup
mkdir -p .github/agents

# Copy the agent
cp path/to/agentic-toolkit/.github/agents/living-doc-bdd-copilot.agent.md .github/agents/
```

Or clone the toolkit and copy all agents:

```bash
git clone https://github.com/AbsaOSS/agentic-toolkit.git
cp agentic-toolkit/.github/agents/*.agent.md .github/agents/
```

Commit the `.github/agents/` directory to share the agents with your team.

### Use an agent in Copilot Chat

Open Copilot Chat in VS Code and type `@` followed by the agent name:

```
@living-doc-bdd-copilot create user story for the login feature
@living-doc-bdd-copilot living doc gaps
@living-doc-bdd-copilot HEALING mode
```

The agent loads its skills on demand and follows its defined scope.

> ⚠️ **Agents are invoked only via `@-mention`** — the `description` field in an agent file is for documentation only, not an activation trigger (unlike skills). Type `@agent-name` explicitly in Copilot Chat to invoke the agent.

See the [Agent Roster](../README.md#agent-roster) for the full list of available agents and their guides.

## Troubleshooting

Running into issues? See **[docs/troubleshooting.md](./troubleshooting.md)** guide.
Loading