Pair snapshots via Verify's received map, with a name based fallback

README.md

@@ -63,6 +63,73 @@ OPTIONS:
 > dotnet verify reject
 ```
 
+## How snapshots are paired
+
+Accepting a snapshot means moving a `.received.` file over the `.verified.` file it belongs to. Those two names are not always the same, so the tool has to work out which verified file each received file maps to. For example, a multi targeted project puts the runtime and version on the received name only:
+
+```
+MyTests.MyTest.DotNet11_0.received.txt  ->  MyTests.MyTest.verified.txt
+```
+
+Each received file is resolved in this order:
+
+```mermaid
+flowchart TD
+    Start["A .received. file"] --> Recorded{"Did Verify record<br>the pairing?"}
+    Recorded -->|yes| UseRecord["Use the recorded<br>.verified. path"]
+
+    subgraph Fallback["Fallback, when there is no record"]
+        SameName{"A .verified. file<br>with same name?"} -->|yes| UseSame["Use it"]
+        SameName -->|no| Reduces{"Does name reduce<br>to a .verified. file<br>beside it?"}
+        Reduces -->|yes| UseReduced["Use it, shown as rerouted"]
+        Reduces -->|no| Derived["Use received-derived name, which can be wrong for a new snapshot"]
+    end
+
+    Recorded -->|no| SameName
+```
+
+### Recorded pairings
+
+From [Verify 31.27.0](https://github.com/VerifyTests/Verify/issues/1809), whenever a received file is left on disk, Verify records the verified file it belongs to. This tool reads those records, so the pairing is exact rather than guessed.
+
+A record is a two line text file, the received path then the verified path, both absolute:
+
+```
+C:\code\MyProject\Tests\MyTests.MyTest.DotNet11_0.received.txt
+C:\code\MyProject\Tests\MyTests.MyTest.verified.txt
+```
+
+Records go in the intermediate (`obj`) directory of the test project rather than beside the snapshot, so they neither clutter the directory holding the code and snapshots nor get picked up by the `*.received.*` glob used to find snapshots:
+
+```
+{IntermediateDirectory}/VerifyReceived/{hash}.txt
+```
+
+ * `{IntermediateDirectory}` is the project's `IntermediateOutputPath`, captured at build time by Verify's MSBuild props and emitted into the test assembly as a `Verify.IntermediateDirectory` metadata attribute. It is per configuration and per target framework, so eg `obj/Debug/net10.0/`. A project that does not consume those props has no directory to write to, and so records nothing.
+ * `{hash}` is an FNV-1a hash of the received path, as 16 hex characters. Deriving the name from the path means re running a test overwrites its record rather than accumulating one per run.
+
+Since a record is only written when a received file is left on disk, none are written by a passing test, by [AutoVerify](https://github.com/VerifyTests/Verify/blob/main/docs/autoverify.md), which accepts in process, or on a [build server](https://github.com/VerifyTests/Verify/blob/main/docs/build-server.md), where nothing consumes them and the recorded paths do not apply off the agent.
+
+Records outlive the received files they describe, for example once a snapshot has been accepted or its test deleted. Stale records are ignored rather than acted on, since a record is only used when the received file it names still exists. They are cleared whenever `obj` is.
+
+To find the records, the tool scans down from the working directory for `VerifyReceived` directories, skipping `.git` and `node_modules`. So the working directory has to contain the `obj` directory, which is the case when running from a project or repository root. Pointing `-w` at a snapshot subdirectory alone means the records are not seen.
+
+### Fallback
+
+Where no record exists, the tool falls back to matching each received file against the verified files that sit next to it. This applies to:
+
+ * snapshots produced by a Verify older than 31.27.0
+ * an `obj` directory that is not under the working directory, as above, or that has been removed since the test run
+
+The fallback handles the common cases, including multi targeting, `UniqueFor*`, and a trailing ignored parameter. It cannot cover everything though:
+
+ * A brand new snapshot has no verified file to match against, so a runtime suffix cannot be removed. Accepting it produces a verified file that Verify will not read back.
+ * A leading or middle ignored parameter cannot be reconstructed, since the verified name is not a truncation of the received name.
+
+When a received file is paired with a differently named verified file, `review` shows it as `(rerouted)`.
+
+See [Verify's file naming docs](https://github.com/VerifyTests/Verify/blob/main/docs/naming.md) for how the names are built.
+
 ## Building
 
 ```

src/Directory.Packages.props

@@ -14,6 +14,7 @@
     <PackageVersion Include="Spectre.Console.Testing" Version="0.57.2" />
     <PackageVersion Include="Spectre.IO.Testing" Version="0.23.0" />
     <PackageVersion Include="Spectre.Verify.Extensions" Version="28.16.0" />
+    <PackageVersion Include="Verify.ExceptionParsing" Version="31.27.0" />
     <PackageVersion Include="Verify.XunitV3" Version="31.27.0" />
     <PackageVersion Include="xunit.v3" Version="3.2.2" />
     <PackageVersion Include="xunit.runner.visualstudio" Version="3.1.5" />

src/Verify.Terminal.IntegrationTests/GlobalUsings.cs

@@ -0,0 +1,11 @@
+global using System.Reflection;
+global using Shouldly;
+global using Spectre.IO;
+global using Verify.Terminal;
+global using VerifyTests;
+global using VerifyXunit;
+global using Xunit;
+
+// Verify keeps global static naming/uniqueness state and these tests hit the real filesystem, so run
+// them serially to keep scenarios isolated.
+[assembly: CollectionBehavior(DisableTestParallelization = true)]

src/Verify.Terminal.IntegrationTests/Harness.cs

@@ -0,0 +1,99 @@
+namespace Verify.Terminal.IntegrationTests;
+
+// An isolated temp directory that real Verify writes into and the real SnapshotFinder scans.
+public sealed class Harness : IDisposable
+{
+    private readonly string _directory;
+
+    public Harness(string name)
+    {
+        // Verify writes no received maps on a build server, so force it off to keep these scenarios
+        // deterministic locally and on CI. The assembly disables test parallelization, so this is safe.
+        DiffEngine.BuildServerDetector.Detected = false;
+
+        _directory = System.IO.Path.Combine(
+            System.IO.Path.GetTempPath(),
+            "verify-terminal-it",
+            $"{name}-{Guid.NewGuid():N}");
+        System.IO.Directory.CreateDirectory(_directory);
+        Directory = new(_directory);
+    }
+
+    public DirectoryPath Directory { get; }
+
+    // Verify writes maps to this test project's obj directory, but a real run scans a root that
+    // contains obj. So copy this scenario's maps under the harness directory to match that layout.
+    // Returns how many were copied, so a test can assert the map path is actually set up rather than
+    // silently falling back.
+    public int PublishMaps()
+    {
+        var copied = 0;
+        var source = System.IO.Path.Combine(
+            AttributeReader.GetIntermediateDirectory(typeof(Harness).Assembly),
+            "VerifyReceived");
+        if (!System.IO.Directory.Exists(source))
+        {
+            return copied;
+        }
+
+        var target = System.IO.Path.Combine(_directory, "obj", "VerifyReceived");
+        System.IO.Directory.CreateDirectory(target);
+
+        foreach (var file in System.IO.Directory.GetFiles(source))
+        {
+            var lines = System.IO.File.ReadAllLines(file);
+            if (lines.Length > 0 &&
+                lines[0].StartsWith(_directory, StringComparison.OrdinalIgnoreCase))
+            {
+                System.IO.File.Copy(file, System.IO.Path.Combine(target, System.IO.Path.GetFileName(file)), true);
+                copied++;
+            }
+        }
+
+        return copied;
+    }
+
+    public VerifySettings CreateSettings()
+    {
+        var settings = new VerifySettings();
+        settings.UseDirectory(_directory);
+        // No diff tool, and allow the same prefix to be verified twice (generate then re-verify).
+        settings.DisableDiff();
+        settings.DisableRequireUniquePrefix();
+        return settings;
+    }
+
+    public void SeedVerified(string fileName, string content) =>
+        System.IO.File.WriteAllText(System.IO.Path.Combine(_directory, fileName), content);
+
+    public IReadOnlyList<string> ReceivedFileNames() =>
+        System.IO.Directory
+            .GetFiles(_directory, "*.received.*")
+            .Select(System.IO.Path.GetFileName)
+            .ToList();
+
+    // Runs the real SnapshotFinder (real globber, real filesystem) over the temp directory.
+    public Snapshot FindSingle()
+    {
+        var environment = new Spectre.IO.Environment();
+        var fileSystem = new FileSystem();
+        var globber = new Globber(fileSystem, environment);
+        var finder = new SnapshotFinder(globber, environment);
+        return finder.Find(Directory).Single();
+    }
+
+    public bool Accept(Snapshot snapshot) =>
+        new SnapshotManager(new FileSystem()).Accept(snapshot);
+
+    public void Dispose()
+    {
+        try
+        {
+            System.IO.Directory.Delete(_directory, recursive: true);
+        }
+        catch
+        {
+            // Best effort cleanup of the temp directory.
+        }
+    }
+}

src/Verify.Terminal.IntegrationTests/IntegrationTestBase.cs

@@ -0,0 +1,185 @@
+namespace Verify.Terminal.IntegrationTests;
+
+// Base for the naming integration tests. Drives real Verify to produce received files, then asserts
+// how the real SnapshotFinder pairs them with verified files.
+//
+// The purpose is to pin the assumptions Verify.Terminal makes about Verify's received/verified naming,
+// so a future Verify version that changes naming breaks these tests rather than silently misbehaving.
+public abstract class IntegrationTestBase
+{
+    // The value verified in every scenario. After a successful accept the verified file holds this,
+    // so a re-verify with the same value passes.
+    protected const string Value = "the-received-value";
+
+    // Runs Verify expecting failure (new or changed snapshot) and returns the verified file name Verify
+    // itself reports it wants. That name is the ground truth for the verified naming.
+    protected static async Task<string> ProduceReceived(VerifySettings settings)
+    {
+        var exception = await Record.ExceptionAsync(async () => await Verifier.Verify(Value, settings));
+        exception.ShouldNotBeNull("Verify was expected to fail and produce a received file.");
+        return ParseVerifiedFileName(exception.Message);
+    }
+
+    // Runs Verify and reports whether it passed (no exception).
+    protected static async Task<bool> Verifies(VerifySettings settings)
+    {
+        var exception = await Record.ExceptionAsync(async () => await Verifier.Verify(Value, settings));
+        return exception is null;
+    }
+
+    protected static string AssemblyConfiguration() =>
+        typeof(IntegrationTestBase).Assembly
+            .GetCustomAttribute<AssemblyConfigurationAttribute>()!
+            .Configuration;
+
+    // A scenario where the correct verified file exists, so the finder should pair the received file
+    // with it (rerouting when the names differ) and an accept round-trips cleanly.
+    // Run against both, since the map is how Verify behaves now, while the fallback still applies to
+    // snapshots from an older Verify, or when obj is not scanned. Both have to reach the same file.
+    // Looped rather than a [Theory], since a test method parameter would be appended to the snapshot
+    // name by Verify and change the very names under assertion.
+    protected async Task AssertExistingVerifiedIsDetected(
+        string method,
+        Action<VerifySettings> configure,
+        string expectedVerified)
+    {
+        await Run(withMap: true);
+        await Run(withMap: false);
+
+        return;
+
+        async Task Run(bool withMap)
+        {
+            using var harness = new Harness(method);
+
+            VerifySettings Settings()
+            {
+                var settings = harness.CreateSettings();
+                settings.UseTypeName("N");
+                settings.UseMethodName(method);
+                configure(settings);
+                return settings;
+            }
+
+            var because = withMap ? "with map" : "without map";
+
+            var correctVerified = await ProduceReceived(Settings());
+
+            // Verify's own verified name matches what Verify.Terminal assumes.
+            correctVerified.ShouldBe(expectedVerified, because);
+
+            // In a multi-targeted project the received file always ends with the runtime and version.
+            var received = harness.ReceivedFileNames().ShouldHaveSingleItem();
+            received.ShouldEndWith($".{Namer.RuntimeAndVersion}.received.txt", customMessage: because);
+
+            // Make the correct verified file exist, then let the finder pair against it.
+            harness.SeedVerified(correctVerified, "old-verified");
+            if (withMap)
+            {
+                // Both paths reach the same file here, so assert the map really was published,
+                // otherwise this would silently be testing the fallback twice.
+                harness.PublishMaps().ShouldBeGreaterThan(0);
+            }
+
+            var snapshot = harness.FindSingle();
+
+            System.IO.Path.GetFileName(snapshot.Verified.FullPath).ShouldBe(correctVerified, because);
+            var literal = received.Replace(".received.", ".verified.");
+            snapshot.IsRerouted.ShouldBe(correctVerified != literal, because);
+
+            harness.Accept(snapshot).ShouldBeTrue(because);
+
+            // The received value now lives at the correct verified name, so Verify passes.
+            (await Verifies(Settings())).ShouldBeTrue(because);
+        }
+    }
+
+    // A brand new snapshot with no verified file on disk. With nothing to pair against, the finder
+    // falls back to the received-derived name. Whether that is correct depends on whether the correct
+    // verified name equals the received-derived name.
+    protected async Task AssertNewSnapshot(
+        string method,
+        Action<VerifySettings> configure,
+        string expectedVerified,
+        bool expectRoundTrips)
+    {
+        using var harness = new Harness(method);
+
+        VerifySettings Settings()
+        {
+            var settings = harness.CreateSettings();
+            settings.UseTypeName("N");
+            settings.UseMethodName(method);
+            configure(settings);
+            return settings;
+        }
+
+        var correctVerified = await ProduceReceived(Settings());
+        correctVerified.ShouldBe(expectedVerified);
+
+        var received = harness.ReceivedFileNames().ShouldHaveSingleItem();
+        received.ShouldEndWith($".{Namer.RuntimeAndVersion}.received.txt");
+
+        // No verified file exists, so the finder can only fall back to the received-derived name.
+        var snapshot = harness.FindSingle();
+        var literal = received.Replace(".received.", ".verified.");
+        System.IO.Path.GetFileName(snapshot.Verified.FullPath).ShouldBe(literal);
+        snapshot.IsRerouted.ShouldBeFalse();
+
+        harness.Accept(snapshot).ShouldBeTrue();
+
+        (await Verifies(Settings())).ShouldBe(expectRoundTrips);
+    }
+
+    // A brand new snapshot, but with Verify's received map available. The map names the verified file,
+    // so the finder places it correctly instead of falling back to the received-derived name.
+    protected async Task AssertNewSnapshotWithMap(
+        string method,
+        Action<VerifySettings> configure,
+        string expectedVerified)
+    {
+        using var harness = new Harness(method);
+
+        VerifySettings Settings()
+        {
+            var settings = harness.CreateSettings();
+            settings.UseTypeName("N");
+            settings.UseMethodName(method);
+            configure(settings);
+            return settings;
+        }
+
+        var correctVerified = await ProduceReceived(Settings());
+        correctVerified.ShouldBe(expectedVerified);
+
+        var received = harness.ReceivedFileNames().ShouldHaveSingleItem();
+        received.ShouldEndWith($".{Namer.RuntimeAndVersion}.received.txt");
+
+        harness.PublishMaps().ShouldBeGreaterThan(0);
+        var snapshot = harness.FindSingle();
+
+        System.IO.Path.GetFileName(snapshot.Verified.FullPath).ShouldBe(correctVerified);
+        snapshot.IsRerouted.ShouldBe(correctVerified != received.Replace(".received.", ".verified."));
+
+        harness.Accept(snapshot).ShouldBeTrue();
+
+        // The accept landed where Verify expects, so the next run passes.
+        (await Verifies(Settings())).ShouldBeTrue();
+    }
+
+    private static string ParseVerifiedFileName(string message)
+    {
+        const string marker = "Verified:";
+        foreach (var line in message.Split('\n'))
+        {
+            var index = line.IndexOf(marker, StringComparison.Ordinal);
+            if (index >= 0)
+            {
+                var path = line[(index + marker.Length)..].Trim();
+                return System.IO.Path.GetFileName(path);
+            }
+        }
+
+        throw new InvalidOperationException($"No 'Verified:' line found in Verify exception message:\n{message}");
+    }
+}