Skip to content

Preserve opaque JSON reference properties in C# code-model deserialization - #12026

Merged
Jorge Rangel (jorgerangel-msft) merged 27 commits into
mainfrom
copilot/fix-csharp-json-id-misinterpretation
Sep 23, 2026
Merged

Jorge Rangel (jorgerangel-msft) merged 27 commits into
mainfrom
copilot/fix-csharp-json-id-misinterpretation

Conversation

Copilot AI commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

The reference pre-scan treats user-owned $id properties as code-model metadata, causing repeated JSON Schema IDs to collide with each other or generated references.

  • Opaque payloads: Exclude unknown and union example value subtrees from indexing. Continue scanning their types and decorator-held graph definitions.
  • Reserved wire names: Treat only string-valued dictionary $id entries as metadata, preserving object-valued $id example properties alongside $ref and $values.

For example, these payload properties remain user data—not reference instructions:

{
  "kind": "unknown",
  "type": { "kind": "unknown" },
  "value": {
    "$id": "https://example.com/person.schema.json",
    "$ref": "#/definitions/person",
    "$values": []
  }
}

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
1 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
@microsoft-github-policy-service microsoft-github-policy-service Bot added the emitter:client:csharp Issue for the C# client emitter: @typespec/http-client-csharp label Sep 21, 2026
Copilot AI changed the title [WIP] Fix C# code-model reference pre-scan for $id in JSON payloads Preserve opaque JSON reference properties in C# code-model deserialization Sep 21, 2026
@pkg-pr-new

pkg-pr-new Bot commented Sep 21, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@typespec/http-client-csharp@12026

commit: aeeccfc

Copilot AI and others added 2 commits September 21, 2026 21:08
…data

Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Comment thread packages/http-client-csharp/emitter/src/lib/raw-json.ts Outdated
Comment thread packages/http-client-csharp/emitter/src/code-model-writer.ts Outdated
Comment thread packages/http-client-csharp/readme.md Outdated
Comment thread packages/http-client-csharp/emitter/src/lib/raw-json.ts Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved compatibility, graph-identity, and resource-bound problems can produce rejected or incorrect code models.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 2 High severity · 2 Medium severity

Open (4)
What changed in this PR

Introduces a versioned C# code-model format that preserves opaque JSON properties while retaining object-reference support and legacy input compatibility.

Changes:

  • Adds v2 envelope, key escaping, and raw JSON handling.
  • Updates deserialization and cyclic-reference processing.
  • Adds documentation, fixtures, and regression tests.
File Description
packages/​http-client-csharp/​readme.md Documents the v2 format and compatibility behavior.
packages/​http-client-csharp/​generator/​Microsoft.TypeSpec.Generator.Input/​test/​TypeSpecInputExampleConverterTests.cs Tests opaque example preservation and reference handling.
packages/​http-client-csharp/​generator/​Microsoft.TypeSpec.Generator.Input/​test/​TypeSpecInputConverterTests.cs Tests v2 decoding, references, cycles, and limits.
packages/​http-client-csharp/​generator/​Microsoft.TypeSpec.Generator.Input/​test/​TestData/​TypeSpecInputConverterTests/​LoadsVersionedEmitterFixture/​tspCodeModel.json Provides an emitter-compatible v2 fixture.
packages/​http-client-csharp/​generator/​Microsoft.TypeSpec.Generator.Input/​src/​InputTypes/​Serialization/​Utf8JsonReaderExtensions.cs Decodes escaped dictionary and decorator data.
packages/​http-client-csharp/​generator/​Microsoft.TypeSpec.Generator.Input/​src/​InputTypes/​Serialization/​TypeSpecSerialization.cs Detects and loads versioned envelopes.
packages/​http-client-csharp/​generator/​Microsoft.TypeSpec.Generator.Input/​src/​InputTypes/​Serialization/​TypeSpecReferenceHandler.cs Separates metadata indexing from opaque legacy values.
packages/​http-client-csharp/​generator/​Microsoft.TypeSpec.Generator.Input/​src/​InputTypes/​Serialization/​TypeSpecJsonConverter.cs Expands and encodes referenced raw JSON graphs.
packages/​http-client-csharp/​generator/​Microsoft.TypeSpec.Generator.Input/​src/​InputTypes/​Serialization/​InputDecoratorInfoConverter.cs Tracks graph-encoded decorator arguments.
packages/​http-client-csharp/​generator/​Microsoft.TypeSpec.Generator.Input/​src/​InputTypes/​InputDecoratorInfo.cs Exposes encoded-argument metadata.
packages/​http-client-csharp/​emitter/​test/​Unit/​code-model-writer.test.ts Tests v2 serialization and opaque data preservation.
packages/​http-client-csharp/​emitter/​src/​lib/​raw-json.ts Adds opaque-property markers.
packages/​http-client-csharp/​emitter/​src/​lib/​example-converter.ts Marks union and unknown example values as raw JSON.
packages/​http-client-csharp/​emitter/​src/​code-model-writer.ts Emits v2 envelopes and escapes data keys.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread packages/http-client-csharp/emitter/src/code-model-writer.ts Outdated
Comment thread packages/http-client-csharp/emitter/src/code-model-writer.ts Outdated
Comment thread packages/http-client-csharp/emitter/src/lib/raw-json.ts Outdated
…ate test projects

Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Copilot AI and others added 4 commits September 23, 2026 15:40
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot it looks like we have a stackoverflow that needs to be addressed. In addition, we should always ensure we run the full test suite before pushing changes:

Versioning.RenamedFrom.V1 -> D:\a\_work\1\s\packages\http-client-csharp\generator\artifacts\bin\Versioning.RenamedFrom.V1\Debug\net8.0\Versioning.RenamedFrom.V1.dll
The active test run was aborted. Reason: Test host process crashed : Stack overflow.
   at System.Text.Json.Serialization.JsonConverter`1[[System.__Canon, System.Private.CoreLib, Version=10.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ReadCore(System.Text.Json.Utf8JsonReader ByRef, System.__Canon ByRef, System.Text.Json.JsonSerializerOptions, System.Text.Json.ReadStack ByRef)
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(System.Object, IntPtr*)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(System.Object, System.Reflection.BindingFlags)
   at System.Reflection.RuntimeMethodInfo.Invoke(System.Object, System.Reflection.BindingFlags, System.Reflection.Binder, System.Object[], System.Globalization.CultureInfo)
   at System.Reflection.MethodBase.Invoke(System.Object, System.Object[])
   at NUnit.Framework.Internal.Reflect.InvokeMethod(System.Reflection.MethodInfo, System.Object, System.Object[])
   at NUnit.Framework.Internal.MethodWrapper.Invoke(System.Object, System.Object[])
   at NUnit.Framework.Internal.Commands.TestMethodCommand.InvokeTestMethod(NUnit.Framework.Internal.TestExecutionContext)
   at NUnit.Framework.Internal.Commands.TestMethodCommand.RunTestMethod(NUnit.Framework.Internal.TestExecutionContext)
   at NUnit.Framework.Internal.Commands.TestMethodCommand.Execute(NUnit.Framework.Internal.TestExecutionContext)
   at NUnit.Framework.Internal.Execution.SimpleWorkItem+<>c__DisplayClass4_0.<PerformWork>b__0()
   at NUnit.Framework.Internal.ContextUtils+<>c__DisplayClass1_0`1[[System.__Canon, System.Private.CoreLib, Version=10.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].<DoIsolated>b__0(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at NUnit.Framework.Internal.ContextUtils.DoIsolated(System.Threading.ContextCallback, System.Object)
   at NUnit.Framework.Internal.ContextUtils.DoIsolated[[System.__Canon, System.Private.CoreLib, Version=10.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.Func`1<System.__Canon>)
   at NUnit.Framework.Internal.Execution.SimpleWorkItem.PerformWork()
   at NUnit.Framework.Internal.Execution.WorkItem.RunOnCurrentThread()
   at NUnit.Framework.Internal.Execution.WorkItem.Execute()
   at NUnit.Framework.Internal.Execution.ParallelWorkItemDispatcher.Dispatch(NUnit.Framework.Internal.Execution.WorkItem, NUnit.Framework.Internal.Execution.ParallelExecutionStrategy)
   at NUnit.Framework.Internal.Execution.ParallelWorkItemDispatcher.Dispatch(NUnit.Framework.Internal.Execution.WorkItem)
   at NUnit.Framework.Internal.Execution.CompositeWorkItem.RunChildren()
   at NUnit.Framework.Internal.Execution.CompositeWorkItem.PerformWork()
   at NUnit.Framework.Internal.Execution.WorkItem.RunOnCurrentThread()
   at NUnit.Framework.Internal.Execution.WorkItem.Execute()
   at NUnit.Framework.Internal.Execution.TestWorker.TestWorkerThreadProc()
   at System.Threading.Thread.StartCallback()

Copilot AI and others added 2 commits September 23, 2026 16:45
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>

Copilot AI commented Sep 23, 2026

Copy link
Copy Markdown
Contributor Author

Copilot it looks like we have a stackoverflow that needs to be addressed. In addition, we should always ensure we run the full test suite before pushing changes:

Addressed in a066c67c with regression coverage in aeeccfcc.

The stack overflow was caused by very deep nested reference expansion recursing through JsonSerializer.Deserialize before the existing JSON MaxDepth guard was reached on Windows. Reference resolution is now capped independently below the platform stack limit, so excessive/cyclic reference chains fail with the existing JsonException message instead of crashing the test host. I added ReferenceResolutionStopsBeforePlatformStackOverflow to cover a chain that is still within the JSON document-depth limit but now fails cleanly.

Validation:

  • dotnet test ./generator/Microsoft.TypeSpec.Generator.Input/test/Microsoft.TypeSpec.Generator.Input.Tests.csproj — passed, 273 tests.
  • dotnet build ./generator/Microsoft.TypeSpec.Generator.Input/src/Microsoft.TypeSpec.Generator.Input.csproj — passed.
  • dotnet test ./generator/TestProjects/Spector.Tests/TestProjects.Spector.Tests.csproj --filter "FullyQualifiedName~Versioning.RenamedFrom" — 3 tests discovered/skipped locally because node_modules is not present in this sandbox; no crash while building/loading Versioning.RenamedFrom.V1.
  • Attempted full dotnet test ./generator; it progressed through building/loading the generated projects including Versioning.RenamedFrom.V1, but then hung locally in the generator test run and had to be stopped.
  • Attempted canonical npm run test; blocked locally because emitter dependencies are not installed (vitest: not found).
  • Changed-file dotnet format --verify-no-changes — passed. Broader format checks hit pre-existing unrelated formatting issues.
  • Secret scan — passed.
  • Final parallel validation — no review comments; CodeQL reported 0 alerts (C# analysis skipped due database size limits).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Non-string $id validation remains property-order dependent and can leak InvalidOperationException.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)

Merged via the queue into main with commit a09867d Sep 23, 2026
31 checks passed
@jorgerangel-msft
Jorge Rangel (jorgerangel-msft) deleted the copilot/fix-csharp-json-id-misinterpretation branch September 23, 2026 21:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

emitter:client:csharp Issue for the C# client emitter: @typespec/http-client-csharp

Projects

None yet

Development

Successfully merging this pull request may close these issues.

C# code-model reference pre-scan misinterprets $id in opaque JSON payloads

4 participants