[CRAFTING] Add mxcli tooling for editing and inspecting e2e test projects by leonardomendix · Pull Request #2240 · mendix/web-widgets

leonardomendix · 2026-06-01T13:57:05Z

Pull request type

Test related change (New E2E test, test automation, etc.)

Description

This PR introduces automation/mxcli/ — a pair of scripts and Claude Code configuration that let AI agents (and developers) read and modify widget .mpr test projects using mxcli (https://github.com/mendixlabs/mxcli), a
CLI tool for Mendix projects. It also adds three Claude Code slash commands that chain these tools into a complete e2e development workflow.

What is mxcli?

mxcli reads and writes Mendix .mpr binary project files using MDL (Mendix Definition Language), a SQL-like syntax. This allows an AI agent to inspect page structure, discover widget names, create new pages, and validate changes — without opening Studio Pro.

github-actions · 2026-06-01T14:48:16Z

AI Code Review

⚠️ Approved with suggestions — low-severity items only, safe to merge

What was reviewed

File	Change
`.claude/commands/mendix/e2e-spec.md`	New slash command: guidance for writing Playwright e2e specs
`.claude/commands/mendix/e2e-workflow.md`	New slash command: orchestrates the full e2e development workflow
`.claude/commands/mendix/edit-test-project.md`	New slash command: guides editing `.mpr` test projects via mxcli
`.claude/settings.json`	Adds permissions allowlist for Playwright MCP and mxcli read-only commands
`automation/mxcli/mx-testproject.sh`	New shell wrapper: dispatches subcommands to mxcli for a given widget's test project
`automation/mxcli/setup.sh`	New installer: downloads the mxcli binary for the current platform
`automation/tools/.gitignore`	Gitignores the downloaded `mxcli` and `mxcli.exe` binaries

Skipped (out of scope): dist/, pnpm-lock.yaml

CI checks were not retrievable in this session — verify CI is green before merging.

Findings

⚠️ Low — `setup.sh` downloads over HTTPS without checksum verification

File: automation/mxcli/setup.sh line 35
Note: curl -fsSL downloads the binary and immediately marks it executable with no integrity check (no SHA-256 or GPG verification). If the GitHub release is tampered with or the download is intercepted, a malicious binary would be silently installed and later auto-approved to run by the settings.json allowlist. Consider adding a checksum step:

EXPECTED_SHA256="<known-sha-for-each-platform-build>"
echo "${EXPECTED_SHA256}  ${BINARY}" | sha256sum --check

Alternatively, pin to a commit SHA in the URL rather than a semver tag, or use GitHub's release asset attestation when available.

⚠️ Low — Wildcard `Bash(bash automation/mxcli/mx-testproject.sh*)` allows write subcommands

File: .claude/settings.json line 12
Note: The exec, shell, and diff subcommands of mx-testproject.sh can mutate the .mpr file (e.g. exec with a CREATE PAGE statement). The current allowlist blanket-approves all subcommands. Read-only operations (inspect, list-pages, search, lint, report, callers, refs) are safe to auto-approve, but exec, shell, and diff (which can apply writes) should remain prompted. Consider splitting the allowlist:

"Bash(bash automation/mxcli/mx-testproject.sh inspect *)",
"Bash(bash automation/mxcli/mx-testproject.sh list-pages *)",
"Bash(bash automation/mxcli/mx-testproject.sh search *)",
"Bash(bash automation/mxcli/mx-testproject.sh lint *)",
"Bash(bash automation/mxcli/mx-testproject.sh report *)",
"Bash(bash automation/mxcli/mx-testproject.sh callers *)",
"Bash(bash automation/mxcli/mx-testproject.sh refs *)"

Leave exec, shell, diff, and check unpermissioned so the user is always prompted before any write operation.

⚠️ Low — `detect_module` silently returns empty string; downstream commands will fail with a confusing error

File: automation/mxcli/mx-testproject.sh line 87
Note: detect_module pipes through || true, so if the regex finds nothing it returns an empty string. The caller checks for this in list-pages, but the inspect block (lines 96–111) does not guard against MODULE being empty — it just silently skips the per-module sections without telling the user. Add a check:

if [[ -z "$MODULE" ]]; then
    echo "Warning: could not auto-detect a non-System/non-Atlas module. Run 'exec <widget> \"SHOW MODULES\"' to inspect manually." >&2
fi

Positives

The edit-test-project.md command prominently warns against CREATE OR MODIFY PAGE ... {} on existing pages with a concrete restore path — exactly the kind of guard that prevents silent data loss in alpha tooling.
The safety flow (validate → diff → apply) is documented before any write examples, making the destructive path harder to reach by accident.
Mutable binaries (mxcli, mxcli.exe) are correctly gitignored rather than committed.
The setup.sh script correctly handles the re-entrant case (--force flag) and prints the installed version as a self-test.
Slash command files consistently cross-reference each other and the canonical docs/requirements/e2e-test-guidelines.md instead of duplicating rules — good single-source-of-truth discipline.

r0b1n · 2026-06-02T10:58:26Z

+
+- Specs live at: `packages/pluggableWidgets/<widget>/e2e/*.spec.js`
+- Playwright strict mode is ON
+- Full rules: `docs/requirements/e2e-test-guidelines.md`


Already mentioned above 🤔

The instructions are different. One is a guardrail to avoid duplication when this file is edited by an agent, and the other defines the context for the command.

r0b1n · 2026-06-02T10:59:55Z

+
+```bash
+# Full overview: modules, pages, entities, navigation menu labels
+bash automation/mxcli/mx-testproject.sh inspect <widget>


What is here?

It's a helper for inspecting MPR file and describing pages, modules, widgets, etc.

r0b1n · 2026-06-02T11:01:20Z

+
+```bash
+# Full CI run (Docker):
+pnpm --filter @mendix/<widget> run e2e


And here, they are likely different?

Yeah, in a certain way. This is the way to pass some flags to skip rebuilding and redownloading test project, and then you have the test project files locally to be able to commit to the testProjects repository.

r0b1n · 2026-06-02T11:03:28Z

+## Checklist
+
+- [ ] Import from `@mendix/run-e2e/fixtures` — NOT `@playwright/test`
+- [ ] No manual `afterEach` logout — fixture handles it
+- [ ] No `waitForLoadState("networkidle")` — prohibited, see guidelines
+- [ ] No locator matches multiple elements without `.first()` or a compound selector
+- [ ] No assertions on Atlas design classes
+- [ ] All tests pass in CI (`pnpm run e2e`)


[Part of it] is duplication from guidelines.

It's a checklist based on the guidelines to make the command more precise.

r0b1n · 2026-06-02T11:05:24Z

+## Prerequisites
+
+- `automation/tools/mxcli` must exist. If missing, run: `bash automation/mxcli/setup.sh`
+- The `.mpr` test project must be **closed in Studio Pro** before any mxcli edits


How can agent verify this?

Maybe checking the lock file in the test project folder?

r0b1n · 2026-06-02T11:13:22Z

+For **brand-new empty pages** created by mxcli, direct URL works:
+
+```js
+await page.goto("/p/my-new-page");
+```
+


Is is passed in some way or auto generated, how to know the url?

It's auto-generated. It knows when inspecting the mpr file using mxcli.

r0b1n · 2026-06-02T11:14:29Z

+
+## Warnings
+
+- `CREATE OR MODIFY PAGE ... {}` wipes all widget content — never use on existing pages


It is mentioned before in the document

Indeed, I'll fix that.

r0b1n · 2026-06-02T11:14:41Z

+## Warnings
+
+- `CREATE OR MODIFY PAGE ... {}` wipes all widget content — never use on existing pages
+- Do NOT run mxcli while Studio Pro has the project open


How to know it is?

Maybe the lock file, I'll try to see if that's possible.

r0b1n · 2026-06-02T11:15:48Z

+await page.goto("/p/my-new-page");
+```
+
+After edits: Studio Pro shows "Project was modified externally" — click **Reload**.


Not sure to which section it belongs. It mentions project should not be modified when project is open in it.

Good one, I think I'll remove this part.

test: add new e2e workflow cc command using mxcli

25f43ec

leonardomendix requested a review from a team as a code owner June 1, 2026 13:57

github-actions Bot added the automation label Jun 1, 2026

This comment has been minimized.

Sign in to view

chore: remove e2e guidelines duplications

0785631

r0b1n reviewed Jun 2, 2026

View reviewed changes

chore: process feedback comments

7e9f901

leonardomendix mentioned this pull request Jun 2, 2026

Add bubble chart e2e #2242

Open


		## Warnings

		- `CREATE OR MODIFY PAGE ... {}` wipes all widget content — never use on existing pages

Conversation

leonardomendix commented Jun 1, 2026

Pull request type

Description

Uh oh!

This comment has been minimized.

github-actions Bot commented Jun 1, 2026

AI Code Review

What was reviewed

Findings

⚠️ Low — setup.sh downloads over HTTPS without checksum verification

⚠️ Low — Wildcard Bash(bash automation/mxcli/mx-testproject.sh*) allows write subcommands

⚠️ Low — detect_module silently returns empty string; downstream commands will fail with a confusing error

Positives

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

⚠️ Low — `setup.sh` downloads over HTTPS without checksum verification

⚠️ Low — Wildcard `Bash(bash automation/mxcli/mx-testproject.sh*)` allows write subcommands

⚠️ Low — `detect_module` silently returns empty string; downstream commands will fail with a confusing error