Add gameplay tests to some starters - #1031
Merged
Merged
Conversation
Two gameplay tests covering the core mechanics: jumping off a platform with Space (and landing back where it took off), and collecting the row of coins by dropping down to the ground and running over them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Two gameplay tests covering the core mechanics: jumping off the ground with Space, and walking forward into a coin to collect it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Two gameplay tests covering the core mechanics: accelerating drives the car forward along its heading, and driving into a traffic cone knocks it out of the way. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Two gameplay tests covering the core mechanics: firing a single shell out of the cannon with F, and turning the turret onto a target and blowing it away. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Two gameplay tests covering the core mechanics: walking and strafing with WASD relative to where the player looks, and aiming at a target with the mouse and shooting it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
… batch Reports what was tested and why, what is missing in the gameplay test harness, and what was surprising while writing these tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
The gameplay test runner writes gameplay-test-results.json and gameplay-test-screenshots/ next to the game it runs: these are run outputs and must never be committed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Contributor
The latest Linux build of GDevelop is downloaded from the S3 bucket its own CI publishes to, and used to open every example game that has gameplay tests in CLI mode (--run-command RUN_ALL_TESTS). On `main`, every game with gameplay tests is tested, split across several containers (the `all-gameplay-tests-parallelism` pipeline parameter). On a branch or Pull Request, only the games it modifies are tested. A pipeline can also be triggered with `run-all-gameplay-tests` to check everything from a branch. Results are reported as JUnit, so CircleCI shows each gameplay test with the assertion that failed, and splits the next runs by recorded timings. The per game results, GDevelop output and failure screenshots are stored as artifacts. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Jumping off a platform with Space (with the expected height derived from the behavior's own jump speed and gravity), and running and jumping to the ledge holding the coins to collect them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Moving in the four directions with the arrow keys, checked against the distance the Top-Down Movement behavior is configured to cover, and being stopped by a wall instead of going through it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Moving in the four directions with the arrow keys, checked against the distance the Top-Down Movement behavior is configured to cover, and being stopped by a wall instead of going through it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Flapping upwards with Space against the gravity that pulls the bird down, and ending the run by touching a hazard, which restarts the scene. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Earning money by clicking the main clicker (and not by clicking next to it), and buying the passive upgrade, which costs money, gets more expensive, and then earns on its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Renames the report to cover every starting-* game, and adds what the top-down, flappy bird, clicker and pixel variants taught: deriving the expected values from the behavior configuration rather than from a measurement, object variables being readable but not writable, and restarting the scene between measurements. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
The ship firing on its own with bullets flying right and the arrow keys moving it, and an enemy put in the line of fire losing health hit after hit until it is destroyed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
The player running to the right on its own and jumping with Space as high as its behavior is configured to, and running into the first hazard of the level, which ends the run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Turning toward the mouse and firing while the button is held, with the bullets flying where the player aims, and an enemy losing health until it is destroyed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
The player firing at the nearest enemy without any input until it is destroyed, and being touched by an enemy ending the run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Adds what the shoot'em up, endless runner, twin stick shooter and vampire survivor taught: preferring what the level already does over arranging it, expressing "did we reach it" on distances rather than on coordinates crossing, and the repeated workaround for the missing scene restart signal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Driving the car forward with the pedal along its heading, with the steering only turning it once it moves (the events scale the torque with the speed), and pushing a physics bush out of the way. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
The ball falling and coming to rest on top of the floor, and being dragged with the mouse then falling again once released. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
The ball falling and coming to rest on top of the floor, and being dragged with the mouse then falling again once released. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Adds the most important finding so far: a stepUntil condition that keeps its own state silently passes without stepping a frame, which is an easy way to write a test that proves nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Attacking with X striking the enemy in front (health, hurt animation and knockback), and the movement behavior being deactivated for the duration of the attack animation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Driving the car forward with the pedal, with the steering only turning it once it moves, and pushing a physics bush out of the way. The checkpoint and lap logic is not covered: it depends on a custom point of the checkpoint arrows, which a test cannot read. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Clicking sending the player walking there with the pathfinding behavior until it reports it reached its destination, and making a detour around an impassable obstacle instead of walking through it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Clicking sending the player walking there with the pathfinding behavior until it reports it reached its destination, and making a detour around an impassable obstacle instead of walking through it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
…int-and-click starters Adds the missing access to an object's custom points (which leaves the lap logic of the racing starter untested), and why asserting on the shape of a path beats reconstructing collision boxes from the snapshot. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Firing a bullet in the direction the player faces (checked on the bullet's own angle, since the bullets are rotated to their direction), and destroying a target by shooting it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Adds the missing way to ask what is at a position (which forces a test to click candidate cells until the game accepts one), and how much sharper the assertions get when a game states an exact rule such as a grid. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Drawing the top card by dragging the face down card off the deck (it turns face up, leaves the deck and is replaced by a new face down card), and dropping a card back onto the deck to return it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Selecting a unit with a drag box and ordering it to walk somewhere while the others stay put, and selecting every unit at once to send the whole group. Selection itself is not observable, so it is checked through the move orders it enables. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Adds the invisibility of selection state (free conditions of extensions and object effects are not in the snapshot), and why the size of a dragged object matters when arranging where to drop it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Pressing the action key next to an NPC opening the dialog layer and taking control away from the player, and clicking yes in the dialog making that NPC leave. The yes button is reached through the parts of the dialog custom object, converted from the object's own coordinates to the scene. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Walking and strafing with WASD relative to where the player looks, and jumping as high as the character behavior is configured to. Note: this starter reproduces the known 'Jolt is not defined' boot race on every run, so whichever of the two tests runs first currently fails for that reason. Both pass when they are not the first to run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
…ters Adds the biggest surprise of this batch: the parts of a custom object report their position inside the parent, not in the scene, so clicking one at its reported centerX/centerY silently misses. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
…arters Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
… starters Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
The cone test of starting-3d-driving timed out on CI at 30.1s, and the two next-slowest tests were close behind. They now stop as soon as what they check has happened, use shorter run-ups, and measure over shorter windows with thresholds to match. The slowest test is now 18s instead of 30s. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Built GDevelop master (ba74a65, "Wait for the game to be fully booted before running a gameplay test") locally and ran starting-first-person four times: 8/8 tests passed, where the build without the fix failed its first test on every run. No test needed to change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
A gameplay test gets a fixed 30s of wall clock and cannot ask for more, while the per-frame render cost on a CI container was measured to double between two runs of the same game (68 frames in 30.3s, where it had stepped 73 frames in 14.0s) with the run's overall median unchanged. A timeout is therefore a statement about the machine, not about the game, so such a run is given a second chance. A failed assertion is never retried. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
Built master 4cc37b4 locally and re-ran four games against the previous commit. The cap bounds the interval between renders, and a single 3D render already costs more than that interval, so no render is ever skipped there: starting-3d-driving still spends 367ms of each 371ms frame outside stepping. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
GDevelop master 4cc37b4 converts the parts of a custom object to the parent's coordinate space and gives them the parent's layer. The test was adding the parent's origin itself, which now double-counts: the click landed off the "Yes" button and the NPC never left. Also record the render cap measured on CI: 661s -> 390s over the 76 tests with identical frame counts, 3-7.5x on 2D and 1.2-2.9x on 3D. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2jc7PVAvmMmirAQude2v1
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.