fix: detect circular references by reference, not by Equals

[endel]

The circular-reference guard added in 3.0.0 fires on graphs that aren't circular.

Stack<object>.Contains() compares with EqualityComparer<object>.Default — Equals(), not reference identity. So any type that overrides Equals to compare a subset of its serialized state trips the guard as soon as an ancestor and a descendant agree on that subset. Types with full structural equality (a plain positional record, or no override at all) are immune, since a finite tree can never equal its own subtree — but partial equality is a common pattern:

class ChatMsg
{
	public string SenderId; public string Text; public ChatMsg Quoted;
	public override bool Equals(object o) { return o is ChatMsg && ((ChatMsg)o).SenderId == this.SenderId; }
	public override int GetHashCode() { return this.SenderId == null ? 0 : this.SenderId.GetHashCode(); }
}

// a user quoting their own earlier message
var value = new ChatMsg { SenderId = "u1", Text = "b", Quoted = new ChatMsg { SenderId = "u1", Text = "a" } };
Json.Serialize(value, stream, SerializationOptions.SuppressTypeInformation);

• 2.4.x → {"SenderId":"u1","Text":"b","Quoted":{"SenderId":"u1","Text":"a","Quoted":null}}
• 3.0.0 → SerializationException: Circular reference detected for type 'ChatMsg'. Path: 'Quoted'.

Two distinct objects, no cycle. MsgPack.Serialize behaves the same. An expression tree with a repeated operator and equality by Op, or entity types with a duplicated id in a nesting relationship, fail the same way.

The fix

SerializationContext.IsInHierarchy(object) walks the hierarchy comparing with ReferenceEquals, and the three call sites (ArraySerializer, ObjectSerializer, DictionarySerializer) use it instead of Hierarchy.Contains(value). Only the same instance nested inside itself can recurse forever. It also behaves correctly for boxed value types, where each boxing produces a distinct object.

Tests

Three tests added to ResilienceTests: equal-but-distinct nested objects over MsgPack and JSON, plus a shared-instance (DAG) control. The first two fail on master with the exception above and pass with the fix.

I also checked the fix doesn't weaken real detection — self-referencing objects, indirect two-object cycles (a.Child = b; b.Child = a), and a list containing itself all still throw, over both JSON and MsgPack. The existing CircularReference* and DeepNesting* tests are untouched and still pass.

Full suite: 98 passed, 0 failed (95 before, plus the 3 new).

Note on the first commit

The .NET test suite doesn't build on master — GameDevWare.Serialization.csproj still globs Assets/Plugins/GameDevWare.Serialization/**/*.cs, which moved to Packages/com.gamedevware.serialization/Runtime in 3.0.0, so the library project compiles zero sources. The first commit repoints it, which is what makes the regression test runnable. It's separated out so you can drop or replace it if you handle that differently — the fix itself is entirely in the second commit.

Found while validating the 3.0.0 update for the Colyseus Unity SDK (colyseus/colyseus-unity-sdk#265). Thanks for the release — the rest of it checked out cleanly on our side: MessagePack output is byte-identical to 2.4.x for all three DateTimeKinds, and date parsing is unchanged across every format we tested.

[deniszykov]

I already did almost the same thing using IEqualityComparer<object>, but thanks anyway.