[FEATURE] Skills testing framework

[skyopsai]

Feature Type

It's important to test skills to ensure that they are validated and triggered properly in different scenarios

Description

I'm considering an LLM native testing framework like Gherkin or to implement something that is suitable for this use case.

Use Case

New skill added -> different depth and checklists are validated:

• skill practices (less than 500 lines, refrences, python only scripts etc)
• prompt or skill injection guardrails
• CI triggered

Optionally Periodic validations

Problem it solves:
ensures new added skills are high quality and consistent throughout the agent kit.

Willingness to Contribute

• I am willing to submit a PR for this feature
• I can help test this feature
• I can provide domain expertise/guidance

[skyopsai]

match the README.md with the marketplace.json and the directory structure

[SirAppSec]

It would be extremely useful to have benchmarks between versioning/variation for a set test
With metrics to measure the effectiveness of the following:

• how good it is at triggering
• how good it is at solving the problems
• how bad it is at expected scenarios
• how hood it is at picking the correct tool for the job

Good to have:

• specifically which references was accessed

• exact commands that was running and log output

• task achieved? Measurable/comparable tasks (can be yes/no preferably split to atomic parts and obtain completion%)