[WotCon] Materialize WoT Thing Descriptions into the server address space - #4154
Merged
marcschier merged 74 commits intoAug 5, 2026
Merged
Conversation
…o marcschier/wot-14-materialization
…to marcschier/wot-14-materialization # Conflicts: # src/Opc.Ua.Types/Utils/FileSystem/LocalFileSystem.cs # src/Opc.Ua.Types/Utils/FileSystem/VirtualFileSystem.cs
Adds the materialization runtime that turns registered WoT Thing Descriptions into live OPC UA nodes: a dependency graph over the registered documents, a coordinator that drives resolution and projection, and a binding runtime that attaches protocol channels to the projected variables. Projection runs through the NodeManager lifecycle rather than mutating a running address space in place, so a refresh either commits a complete new generation or leaves the previous one untouched. The resolver, converter, contributor and target variable seams are interfaces so a host can substitute its own document sources. Also adds the registry NodeManager that exposes the registry itself in the address space, and the hosting extensions that register the whole runtime with dependency injection. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
Bring the PR14 materialization, dependency graph, binding runtime, registry projection lifecycle, and NodeManager coverage across from the stacked integration branch. Keep the tests compiling for both net10.0 and net48 by naming ImmutableDictionary.Empty on the binding plan models and updating the memory binding test seam to the current codec resolver contract. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
…arcschier/wot-16-samples # Conflicts: # src/Opc.Ua.Types/Utils/FileSystem/VirtualFileSystem.cs
Adds three runnable samples: two flat OPC UA tag servers and an aggregation server that materializes them into a DI/Machinery/Pumps model through the WoT registry, plus an aggregation client that drives the result. The AggregationServer also ships a memory protocol binding, which doubles as the worked example for the binding contributor guide. Documents the WoT Connectivity surface end to end: a new protocol bindings guide, the expanded WoT Connectivity overview covering the registry, materialization and runtime projection, and the shadow-reload and shared byte-store sections for the runtime NodeSet and xRegistry guides. Registers the sample and test projects in the solution so they are built and run by CI. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
Brings across the binder registry, codec, planner and target-mapping tests, the custom-binder sample test, the combined-model preservation tests and the aggregation document tests, closing the last coverage gap against the integration branch. The custom-binder sample needed updating: WotEndpointPolicy now fails closed on unknown URI schemes, so a binding that introduces its own scheme has to opt it in. That is exactly the situation every custom binding author hits, so the sample now shows it and the contributor guide documents the policy, the failure it produces and the checklist item. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
TransferServicesExtensions.BindToTransferService already ships but had no tests. Adds argument validation, required-method-children and routing coverage. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
…16-samples # Conflicts: # UA.slnx
…alization # Conflicts: # src/Opc.Ua.Types/Utils/FileSystem/VirtualFileSystem.cs
marcschier
force-pushed
the
marcschier/wot-14-materialization
branch
from
August 1, 2026 11:45
6f3e667 to
cd84294
Compare
…alization # Conflicts: # src/Opc.Ua.Types/Utils/FileSystem/VirtualFileSystem.cs
…16-samples # Conflicts: # src/Opc.Ua.Types/Utils/FileSystem/VirtualFileSystem.cs
marcschier
force-pushed
the
marcschier/wot-14-materialization
branch
from
August 1, 2026 11:47
cd84294 to
777e0a8
Compare
…alization # Conflicts: # tests/Opc.Ua.WotCon.Tests/Registry/WotBlobResourceStoreTests.cs
marcschier
changed the base branch from
marcschier/wot-13-registry
to
marcschier/wot-15-client
August 1, 2026 11:55
cristipogacean
approved these changes
Aug 3, 2026
Dry-run refreshes computed and reported a candidate result but still executed the retirement path and advanced the coordinator generation. That let a preview deactivate binders, remove host projections, drop tracked closures, and shift later optimistic-concurrency checks. Compute the candidate generation without committing it, apply expected-generation checks to the committed refresh generation, and make retirement reconciliation report what would be retired without touching binders, host registrations, or tracked closure state during dry runs. Add regression coverage for retirement previews and repeated dry-run generation checks before a real refresh. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
…arcschier/wot-14-materialization
…n' into marcschier/wot-16-samples
…arcschier/wot-14-materialization
…n' into marcschier/wot-16-samples
…arcschier/wot-14-materialization
…n' into marcschier/wot-16-samples
The WoT registry node manager can be asked to delete its address space after the externally-owned materialization coordinator has been disposed. That disposed the coordinator semaphore before DeleteAddressSpaceAsync called RemoveAllAsync, so shutdown reported ObjectDisposedException from the node manager cleanup path. Keep coordinator cleanup available after Dispose when live projections remain, block new refreshes, and release the semaphore once the deferred cleanup has drained. Also order the direct integration-test teardown through the server fixture so the node manager cleanup runs before test-owned services are released, and add a regression test for shutting down with a live materialized projection after coordinator disposal. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
…n' into marcschier/wot-16-samples
During ordered server disposal, the NodeManager lifecycle has already entered shutdown before WotRegistryNodeManager deletes its address space. Removing materialized WoT projections by routing back through the lifecycle is therefore the wrong operation: the lifecycle will tear down every remaining dynamic NodeManager as part of its shutdown completion. Expose the lifecycle shutdown state and make the WoT projection host treat removal as a no-op once that state is active. Normal refresh/removal paths still unregister projections through the lifecycle, while full server teardown avoids re-entering a component that is already shutting down. The shutdown regression now exercises synchronous server disposal with active projections. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
…n' into marcschier/wot-16-samples
…arcschier/wot-14-materialization # Conflicts: # src/Opc.Ua.Server/NodeManager/Lifecycle/NodeManagerLifecycle.cs
…n' into marcschier/wot-16-samples
…arcschier/wot-14-materialization
…n' into marcschier/wot-16-samples
Master replaced the ambient AsyncLocal re-entrancy guard with an explicit IOperationContext parameter on the lifecycle add, reload and remove operations, so the WoT projection host no longer compiled: its cancellation token was binding to the new caller-context parameter. The host passes null, which states that these calls are not made on behalf of an executing OPC UA request. The shadow and immediate reload paths are unchanged because those overloads deliberately take no caller context; they are the reload modes that are safe to invoke from a request callback. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
…n' into marcschier/wot-16-samples
Merging master added an IOperationContext parameter to the NodeManager lifecycle add operation, so these tests no longer bound their argument to the intended overload. They pass null because the registration is created by the test fixture rather than on behalf of an executing request. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
…arcschier/wot-14-materialization
…n' into marcschier/wot-16-samples
## Summary - add optional WoT Connectivity asset TD mirroring into the WoT xRegistry - add DI opt-in via AddWotRegistryBridge and direct WotConnectivityServerOptions.RegistryBridge fallback - document default-off, best-effort failure policy and usage - add NUnit/Moq coverage for create, update, delete, disabled and failure paths ## Validation - dotnet build src\\Opc.Ua.WotCon.Server\\Opc.Ua.WotCon.Server.csproj -c Release -v:m (all TFMs, 0 warnings/errors) - dotnet test tests\\Opc.Ua.WotCon.Tests\\Opc.Ua.WotCon.Tests.csproj -c Release -p:CustomTestTarget=net10.0 (0 failed / 983 passed) --------- Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
…arcschier/wot-14-materialization
…n' into marcschier/wot-16-samples
…marcschier/wot-16-samples
…ion (#4156) Adds the runnable WoT Connectivity samples and completes the documentation. This is **PR 16 of a stack** splitting #4093, and the last one. ## Samples Three projects under `samples/WotCon/`: * **FlatTagServer** — a plain OPC UA server exposing flat tags, used as an aggregation source. * **AggregationServer** — registers Thing Descriptions with the WoT registry and materializes two flat sources into a runtime-loaded DI/Machinery/Pumps Pump model. It also ships a small in-memory protocol binding, which is the worked example the binding contributor guide walks through. * **AggregationClient** — drives the aggregated model: commands, `Refresh`, monitoring, and generation replacement. ## Documentation * New `docs/WotBindings.md` — the bindings that ship today (planner/executor architecture, bundled vs separate packages, operation coverage, target mapping, lazy channels, generation lifetime) plus the contributor guide for adding your own, with diagnostics, tests, packaging, TFM, trimming and NativeAOT guidance. * Expanded `docs/WoTConnectivity.md` covering the registry, dependency-closure materialization and runtime NodeSet projection. * `docs/RuntimeNodeSets.md` gains the shadow-reload section and `docs/XRegistry.md` the shared byte-store section. Both document APIs introduced earlier in the stack; they are collected here because this is the documentation PR. * `docs/README.md` links the new pages. ## Solution registration Registers the three samples and `Opc.Ua.WotCon.Samples.Tests` in `UA.slnx` so CI actually builds and runs them. ## Stack position Sits on top of **#4154** (materialization) and additionally merges **#4142** (registry client) and **#4144** (protocol executors), because the samples exercise the whole stack — `AggregationServer` needs the OPC UA executor and the client sample needs the registry client. Its diff therefore shows their content until they land. It also carries `WotRegistryProjectionLiveTests.cs`, which #4154 deliberately left out: those tests need both the materialization runtime *and* the registry client, so this is the first branch where they compile. ## Validation * All three samples build clean on every target framework. * `Opc.Ua.WotCon.Tests`: **830 passed**, 0 failed (769 from #4154 plus 61 from the live projection tests). * `Opc.Ua.WotCon.Samples.Tests`: 6 of 7 pass locally; the remaining one fails with `CryptographicException: The system cannot find the path specified`, a known certificate-store problem on this machine that was previously reproduced against a clean baseline worktree and is not related to this change. CI will confirm. * 0 warnings, 0 errors. One fix was needed while bringing the sample across: its memory binding called `ResolveCodec` with the pre-`out WotPayloadDescriptor` signature. The API gained the payload out-parameter and a `bool` result during review of the binding PRs, so the sample now follows the same `if (!ResolveCodec(...)) return Unsupported(...)` pattern as the shipped planners. Integration PR: #4093.
marcschier
added a commit
that referenced
this pull request
Aug 5, 2026
…ation, Server and xRegistry work it builds on (#4128) ## Summary Adds OPC UA WoT Connectivity 1.1 to the stack, together with the Types, source generation, Server and xRegistry work it is built on. This branch is the merge point for the fourteen-PR stack that replaced integration PR #4093; every constituent PR was reviewed and approved separately and is listed below. 414 files changed, +108,936 / -3,199. Roughly 46k added lines of product code, 57k of tests, plus samples, tools and documentation. ## Types - Complete `ExtensionObject` raw-body decoding for binary, XML and JSON bodies by resolving the concrete type through the message context's encodeable factory, and simplify the `Variant` structure helpers by delegating to it (#4128). - Add `IAtomicFileReplace` as an optional `IFileSystem` capability so existing external implementations keep working, with atomic publish for `LocalFileSystem` via `File.Replace`/`File.Move` and for `VirtualFileSystem` by re-keying the in-memory entry (#4128). - Add lossless conversion between WoT documents and NodeSet2 (`Opc.Ua.Wot.WotNodeSetConverter`). A byte-exact `uav:nodeSet` envelope is preserved when requested, the structured `uav:nodes` projection is used when the readable vocabulary is incomplete, and NodeSet2 is otherwise synthesized from readable WoT terms. Unmapped JSON members survive a round trip as pointer-addressed residue (#4131). ## Source generation - Improve the NodeSet to ModelDesign conversion and the generated node state (#4132). - Generate OPC UA models directly from WoT Thing Description files (#4134). ## Server - Add live NodeManager shadow and immediate reload. A reload materializes a new generation beside the active one and switches atomically; the superseded generation is retired gracefully so existing MonitoredItems keep being served until they drain, or immediately when the caller asks for it. Includes the request admission and drain machinery that lets an orderly shutdown wait for admitted requests instead of tearing down underneath them (#4147). - Make subscription transfer between sessions transactional, so a failed transfer leaves neither session holding a partially moved subscription (#4150). ## xRegistry - Allow clients to use explicit registry roots rather than assuming a well-known location (#4130). ## WoT Connectivity - Add the WoT Connectivity 1.1 information model: a registry-first revision layered on the abstract xRegistry base model, incorporating the complete published OPC 10100-1 v1.02 surface as deprecated nodes in the same namespace so existing 1.02 clients keep working (#4139). - Add the protocol binding abstractions and planners that compile WoT forms into executable plans (#4140). - Add the HTTP, Modbus, OPC UA and MQTT binding executors (#4144). - Add the registry and its stores, including document versioning, validation, dependency resolution and refresh (#4146). - Add the registry client (#4142). - Materialize WoT Thing Descriptions into the server address space, deriving types from Thing Models and instances from Thing Descriptions (#4154). - Restore the WoT asset registry bridge so the deprecated 1.02 asset surface is backed by the registry (#4171). - Add the WoT Connectivity samples and complete the documentation (#4156), and restore the remaining coverage and docs (#4172). ## New projects `src/Opc.Ua.WotCon.Bindings`, `src/Opc.Ua.WotCon.Bindings.Mqtt`, `tests/Opc.Ua.WotCon.Bindings.Tests`, `tests/Opc.Ua.WotCon.Samples.Tests`, and the `samples/WotCon` AggregationServer, AggregationClient and FlatTagServer samples. ## Constituent pull requests Merged into this branch top-down, each independently approved: | PR | Title | | --- | --- | | #4128 | [Types] Complete ExtensionObject decoding and add an atomic file replace capability | | #4131 | [Types] Add lossless conversion between WoT documents and NodeSet2 | | #4132 | [SourceGeneration] Improve NodeSet to ModelDesign conversion and generated node state | | #4134 | [SourceGeneration] Generate OPC UA models from WoT Thing Description files | | #4147 | [Server] Add live NodeManager shadow and immediate reload | | #4150 | [Server] Make subscription transfer between sessions transactional | | #4130 | [XRegistry] Allow clients to use explicit registry roots | | #4139 | [WotCon] Add the WoT Connectivity 1.1 information model | | #4140 | [WotCon] Add the WoT protocol binding abstractions and planners | | #4144 | [WotCon] Add the HTTP, Modbus, OPC UA and MQTT binding executors | | #4146 | [WotCon] Add the WoT Connectivity registry and its stores | | #4142 | [WotCon] Add the WoT Connectivity registry client | | #4154 | [WotCon] Materialize WoT Thing Descriptions into the server address space | | #4156 | [WotCon] Add the WoT Connectivity samples and complete the documentation | | #4171 | Restore WoT asset registry bridge | | #4172 | Restore WoT coverage and docs | Supersedes #4093. ## Follow-up The WoT Connectivity and WoT Binding drafts have moved to 1.1-draft2 since this work was authored. Aligning with that revision - the removed group vocabulary, the new projection/View construct, the remaining model vocabulary terms and the two new portable-identity validation rules - is tracked separately and will follow in its own pull request. ## Validation Built for every target framework with zero warnings and zero errors, and validated at the tip of the stack with `Opc.Ua.WotCon.Tests` at 999 passed / 0 failed and `Opc.Ua.Server.Tests` at 4024 passed / 0 failed. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR 14 of the stacked series splitting the large WoT Connectivity integration PR #4093.
This PR adds the server-side materialization runtime that turns WoT Thing Description resources from the xRegistry snapshot into OPC UA address-space projections. The coordinator builds the dependency closure for each resource, plans binding metadata, activates binding channels, and publishes the resulting projection through the registry NodeManager so refresh, retire, and reload operations remain observable and deterministic.
Projection intentionally goes through the NodeManager lifecycle reload API instead of mutating live nodes directly. That keeps complex type reload, runtime NodeSet publication, shadow reload, immediate reload, and retirement on the same lifecycle path as other dynamic server address-space changes.
This branch has a triple dependency. It is based on #4146 (registry), and additionally needs #4147 (lifecycle reload API) and #4128 (2-argument TryGetStructure). Both #4147 and #4128 are merged into this branch, so this PR's diff will show their content until those dependencies land.
Validation performed: