Upload Snapshots to Sentry using SnapshotPreview and SwiftSnapshotTesting libraries

[cameroncooke]

This branch exists just to test the unreleased now merged changes to SnapshowPreviews on the here: EmergeTools/SnapshotPreviews#252

See workflow run which is using SnapshotPreviews with CI export functionality:
https://github.com/EmergeTools/hackernews/actions/runs/23857485626/job/69554642234?pr=780

Resulting Sentry Snapshots URL:
https://sentry.sentry.io/preprod/snapshots/177173/

[emerge-tools]

📸 Snapshot Test

2 modified, 70 unchanged

Name	Added	Removed	Modified	Renamed	Unchanged	Errored	Approval
HackerNews com.emergetools.hackernews.snapshots	0	0	2	0	70	0	⏳ Needs approval

---

🛸 Powered by Emerge Tools

[sentry]

📲 Install Builds

iOS

🔗 App Name	App ID	Version	Configuration
HackerNews	com.emergetools.hackernews	3.10 (1)	Release
HackerNews	com.emergetools.hackernews	3.10 (1)	AdHoc

⚙️ launchpad-test-ios Build Distribution Settings

[sentry]

Sentry Snapshot Testing

Name	Added	Removed	Modified	Renamed	Unchanged	Status
com.emergetools.hackernews com.emergetools.hackernews	0	0	0	0	18	✅ Unchanged

⚙️ hackernews-ios Snapshot Settings

[NicoHinderling]

intentional?

[cameroncooke]

@NicoHinderling Yes, because Swift Snapshot Testing does diffing locally, we would only want to enable recording on CI workflows. We use an environmental variable in the GitHub workflow example TEST_RUNNER_SNAPSHOT_TESTING_RECORD which causes the library to record. This allows the user to run tests and catch regressions locally before pushing via the Pointfree library's own diff algo and on CI it will just be used to export images. It's also worth noting enabling recording causes XCTest failures which we ignore on CI but will run as real regression tests locally.

[NicoHinderling]

I think lets actually uncomment and keep this in here! I think it's a great example :)

[cameroncooke]

@NicoHinderling There is actually one issue at the moment the filenames don't include the simulator device info or runtime so iPad images would overwrite the iOS ones. What do we think the best strategy is for users on this?

• Should we use nested folders does that name space on Sentry?
• Should we just rename the files adding a prefix or suffix post generation?
• Should the library automatically name images based on device/runtime?

Some things that are not clear to me is how we disambiguate on the Sentry end and what the expected exp. is for multiple platform/device snapshots.

[NicoHinderling]

getsentry/sentry-cli#3269

i actually just merged a PR related to this yesterday to add proper subdirectory support within the snapshots images-to-upload folder...

so I think two things:

1. In our example, we should upload the ipad and iphone images as nested subdirectories, that way the image_file_name value will be unique for all images
2. the deviceType is exactly the kind of metadata that the snapshotPreviews library should expose that we can then pass into the JSON sidecar metadata thanks to your PR :)

As a reminder, the setup currently allows you to include whatever extra fields you want in the metadata upload, so could you make sure for this example we include declaredDevice/simulatorDeviceName in this upload?

[cameroncooke]

Cool so we already upload simulatorDeviceName:

{
  "baseFileName" : "Comment_View_Indentation_Preview_Indentation_0",
  "display_name" : "Indentation 0",
  "group" : "Comment View Indentation Preview",
  "image_file_name" : "Comment_View_Indentation_Preview_Indentation_0",
  "orientation" : "portrait",
  "previewDisplayName" : "Indentation 0",
  "previewId" : "0",
  "previewIndex" : 0,
  "simulatorDeviceName" : "iPad Air 11-inch (M4)",
  "simulatorModelIdentifier" : "iPad16,9",
  "testName" : "-[HackerNewsSnapshotTest portrait-Comment View Indentation Preview-0-5]",
  "typeDisplayName" : "Comment View Indentation Preview",
  "typeName" : "HackerNews.CommentViewIndentation_Preview"
}

[cameroncooke]

So just tested with nested folders with duplicate file names and it works well. Only thing I'm not sure about is how the user disambiguates snapshot variants in the Sentry UI.

Ideas:

1. We just leave it up to users to add a disambiguator i.e. suffix to the display_name field, though for Snapshots we still have to have an opinionated format as we generate the value dynamically.
2. We support nested groups maybe we can parse / in the group name and show some kind of nested ui
3. We add/use a field from json metadata that we then use in the UI to disambiguate in addition to display_name and group.

Be good to get your thoughts on this @NicoHinderling

[sentry]

Bug: The iPad snapshot generation step lacks continue-on-error: true, making it less resilient to transient failures compared to similar steps in other workflows.
_{Severity: LOW}

Suggested Fix

Add continue-on-error: true to the "Generate snapshot images (iPad)" step. This will align its behavior with other snapshot testing jobs and prevent the entire workflow from failing due to non-critical, transient test issues.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: .github/workflows/ios_sentry_upload_snapshots.yml#L92-L103

Potential issue: The "Generate snapshot images (iPad)" step in the
`ios_sentry_upload_snapshots.yml` workflow does not set `continue-on-error: true`. This
is inconsistent with similar snapshot testing steps in other workflows like
`ios_sentry_upload_swift_snapshots.yml` and `ios_emerge_upload_snapshots.yml`, which do
use this flag. Without it, the entire workflow could fail due to transient issues with
the iPad simulator or the test itself, reducing the overall reliability of the CI
pipeline. While a race condition with simulator boot-up is mitigated by a preceding
step, this lack of error handling introduces unnecessary brittleness.