Add TestRails action#6380
Conversation
suhaibmujahid
left a comment
suhaibmujahid
left a comment
There was a problem hiding this comment.
Nice! Thank you! Please see my comments, and we can discuss details.
| return f"{_base_url()}/index.php?/api/v2/{endpoint.lstrip('/')}" | ||
|
|
||
|
|
||
| def _api_request( |
There was a problem hiding this comment.
Let us move the part to communicate with TestRail api into its own lib (i.e., libs/testrail-client) like we do for Phabricator (libs/phabricator-client). We could follow that as a reference design.
We keep here the business logic only.
| return ActionResult.ok( | ||
| { | ||
| "suite_id": suite_id, | ||
| "suite_url": _suite_url(suite_id), |
There was a problem hiding this comment.
Using url instead of suite_url would avoid know issues, see #6330
| "suite_url": _suite_url(suite_id), | |
| "url": _suite_url(suite_id), |
| ACTION_TYPE = "testrail.submit_test_cases" | ||
|
|
||
|
|
||
| def record_test_plan( |
There was a problem hiding this comment.
Is this working? I expect not.
Could you please structure it like libs/hackbot-runtime/hackbot_runtime/actions/phabricator.py?
|
|
||
| def record_test_plan( | ||
| recorder: ActionsRecorder, | ||
| test_plan: dict[str, Any], |
There was a problem hiding this comment.
Please use pydantic to annotate the inputs like in actions/phabricator.py. That will be provided to the agent to know ho to use the tool.
| recorder: ActionsRecorder, | ||
| test_plan: dict[str, Any], | ||
| *, | ||
| reasoning: str = "Upload the generated test cases to TestRail.", |
There was a problem hiding this comment.
No point in having a default value here. If we have them test case by test case, then having the agent providing reasoning would make sense, but for the test plan itself, it is kind of useless, and we could drop it.
|
|
||
| class SubmitTestCasesHandler: | ||
| async def apply(self, params: dict[str, Any], ctx: ApplyContext) -> ActionResult: | ||
| feature = str(params.get("feature") or "").strip() |
There was a problem hiding this comment.
This is a test case handler, but it's currently handling test plans too. :p
I still think it'd be better to allow adding cases one by one. We could have a separate method to create a suite, and another to create a section; each taking a ref. Test cases would then reference that ref, making it available when the cases get applied, so we can retrieve their IDs once they're created and use them to create the cases.
2 participants
This pull request introduces support for recording and applying TestRail actions for generated test plans. It adds a new action type for submitting generated test cases to TestRail, implements the handler to make the necessary API calls, and ensures these actions are properly registered and tested. Additionally, the test plan generator agent is updated to automatically record and apply these actions, and new tests are included to verify the integration.