diff --git a/README.md b/README.md index 0f81fa5d..2812baee 100644 --- a/README.md +++ b/README.md @@ -47,9 +47,7 @@ Untold Engine is built for developers and teams who: Creator & Lead Developer: https://www.haroldserrano.com -[![untoldengine-image](/docs/images/engine-highlight-5.png)](https://vimeo.com/1176995991?fl=ip&fe=ec) - -Click to Play +![archviz-demo](docs/images/gifs/archviz-demo.gif) --- @@ -69,18 +67,16 @@ workflow, product experience, and customer-specific behavior. ## Watch It in Action — Apple Vision Pro Demos - - - - - - - - - - - -
+| Demo | Description | +| --- | --- | +| [CoolSaber](https://github.com/untoldengine/UntoldArcade/tree/main/CoolSaber) | PSVR2-driven lightsaber duels, blades clashing over SharePlay | +| [CoolWater](https://github.com/untoldengine/UntoldArcade/tree/main/CoolWater) | Real-time animated water with reflection, refraction, and ripples | +| [CoolCloth](https://github.com/untoldengine/UntoldArcade/tree/main/CoolCloth) | GPU cloth simulation (XPBD) you can punch a ball through | +| [Archviz Viewer](https://untoldengine.github.io/UntoldEngine/LearningPaths/ArchvizToVisionPro/) | Blender-authored architectural scene walked through in mixed reality | +| [Bedroom Digital Twin](https://untoldengine.github.io/UntoldEngine/LearningPaths/BedroomDigitalTwin/) | Tap-to-inspect digital twin bedroom with live mock status data | +| [City Streaming](https://untoldengine.github.io/UntoldEngine/LearningPaths/CityStreamingOnVisionPro/) | City-scale scene streamed in tile by tile with LOD/HLOD | + +Full source for every demo above lives in the [UntoldArcade](https://github.com/untoldengine/UntoldArcade) repo. ## Requirements @@ -89,6 +85,8 @@ workflow, product experience, and customer-specific behavior. - Apple Silicon Mac - Supported platforms: macOS, iOS, and visionOS +![coolsaber-demo](docs/images/gifs/coolsaber-demo.gif) + ## Try the Engine Right Now The best first step is to run the Starter Demo. It is intentionally small and @@ -124,8 +122,6 @@ The demos live under `Sources/Demos`. They are runnable when working inside the engine repository, but are not exposed as products to apps that add Untold Engine as a package dependency. -![untoldengine-image-2](/docs/images/engine-highlight-6.png) - --- ## Getting Started diff --git a/Tools/UntoldEngineCLI/Sources/UntoldEngineCLI/Resources/manifest.json b/Tools/UntoldEngineCLI/Sources/UntoldEngineCLI/Resources/manifest.json index fd7577a6..46965868 100644 --- a/Tools/UntoldEngineCLI/Sources/UntoldEngineCLI/Resources/manifest.json +++ b/Tools/UntoldEngineCLI/Sources/UntoldEngineCLI/Resources/manifest.json @@ -18,12 +18,20 @@ "size": "253.4 MB" }, { - "id": "starter-tiled-scene", - "name": "Tiled Scene Starter Assets", - "description": "A tile-streamed city scene for the Geometry Streaming System.", + "id": "starter-streamed-city", + "name": "Streamed City Starter Assets", + "description": "A low-poly city scene for the tile-based Geometry Streaming System.", "version": "1.0.0", - "downloadURL": "https://d8pyi1c08k1w.cloudfront.net/TiledScenePack.zip", - "size": "34.1 MB" + "downloadURL": "https://d8pyi1c08k1w.cloudfront.net/LowPolyCity.zip", + "size": "58.9 MB" + }, + { + "id": "starter-digital-twin", + "name": "Digital Twin Starter Assets", + "description": "Bedroom digital-twin scene with light portal and selectable devices.", + "version": "1.0.0", + "downloadURL": "https://d8pyi1c08k1w.cloudfront.net/DigitalTwinPack.zip", + "size": "424.3 MB" } ] } diff --git a/docs/API/GettingStarted.md b/docs/API/GettingStarted.md index ea1de0f8..57fc8cb1 100644 --- a/docs/API/GettingStarted.md +++ b/docs/API/GettingStarted.md @@ -193,7 +193,7 @@ Let's download a Stream Starter Scene ```bash -untoldengine assets install starter-tiled-scene +untoldengine assets install starter-streamed-city ``` @@ -205,10 +205,10 @@ To load a Tiled-Stream scene, you need to use `setEntityStreamScene` and pass th let sceneRoot = createEntity() -setEntityName(entityId: sceneRoot, name: "City") +setEntityName(entityId: sceneRoot, name: "LowPolyCity") // Local manifest -setEntityStreamScene(entityId: sceneRoot, manifest: "City", withExtension: "json") { success in +setEntityStreamScene(entityId: sceneRoot, manifest: "LowPolyCity", withExtension: "json") { success in setSceneReady(success) } ``` diff --git a/docs/LearningPaths/ArchvizToVisionPro.md b/docs/LearningPaths/ArchvizToVisionPro.md index 5f1eabcd..06189c39 100644 --- a/docs/LearningPaths/ArchvizToVisionPro.md +++ b/docs/LearningPaths/ArchvizToVisionPro.md @@ -31,6 +31,8 @@ The goal is a simple but complete archviz viewer: This path uses the starter archviz asset so you can focus on the engine workflow first. After this is working, you can replace the starter asset with your own Blender export. +Want to run it right away instead of building it step by step? The finished project, **ArchvizViewer**, is in the [UntoldArcade](https://github.com/untoldengine/UntoldArcade) demo collection: [github.com/untoldengine/UntoldArcade/tree/main/ArchvizViewer](https://github.com/untoldengine/UntoldArcade/tree/main/ArchvizViewer). Clone it, open the Xcode project, and run. + ## Prerequisites You need: diff --git a/docs/LearningPaths/BedroomDigitalTwin.md b/docs/LearningPaths/BedroomDigitalTwin.md new file mode 100644 index 00000000..dc3192c4 --- /dev/null +++ b/docs/LearningPaths/BedroomDigitalTwin.md @@ -0,0 +1,424 @@ +# Bedroom Digital Twin + +This learning path turns a structured bedroom model into an interactive Vision Pro digital twin. + +The purpose is to show how Untold Engine can give different runtime behavior to different parts of the same room: + +- Window geometry can act as a light portal. +- Window geometry can use passthrough ghost rendering. +- Context geometry can be rendered normally, hidden, or shown as wireframe. +- Named objects can remain selectable and show mock digital-twin state. +- The whole room can be moved and rotated with spatial gestures. + +By the end of the path, you will have a Vision Pro scene where the bedroom is not just rendered as a model. It behaves like a structured spatial twin. + +## What You Will Build + +The example scene is a bedroom with: + +- floor, ceiling, and walls +- a window with glass +- selectable furniture and devices +- mock status data for selected objects +- Vision Pro spatial manipulation + +The main engine systems are: + +- `setEntityMeshAsync` for loading the room +- `loadSceneAuthored` for Blender-authored color management (see the note in [Replace The Starter Asset With Your Own Bedroom Scene](#replace-the-starter-asset-with-your-own-bedroom-scene) about lights and cameras) +- `SceneChannel` for geometry categories +- a light portal for window lighting +- passthrough ghost render mode for mixed reality +- XR spatial input for selection and room manipulation + +This path uses the starter digital-twin asset pack so you can focus on the engine workflow first. After this is working, you can replace the starter asset with your own Blender export. + +Want to run it right away instead of building it step by step? The finished project, **BedroomTwin**, is in the [UntoldArcade](https://github.com/untoldengine/UntoldArcade) demo collection: [github.com/untoldengine/UntoldArcade/tree/main/BedroomTwin](https://github.com/untoldengine/UntoldArcade/tree/main/BedroomTwin). Clone it, open the Xcode project, and run. + +## Create The Vision Pro Project + +Create a standalone visionOS project: + +```bash +cd ~/Projects +untoldengine create BedroomTwin --platform visionos +open BedroomTwin/BedroomTwin.xcodeproj +``` + +Open: + +```text +Sources/BedroomTwin/GameScene.swift +``` + +## Install The Starter Digital Twin Asset + +From the generated project folder, install the starter digital-twin asset pack: + +```bash +cd ~/Projects/BedroomTwin +untoldengine assets install starter-digital-twin +``` + +The CLI finds the project's `GameData` folder and merges the asset files into it. + +The important file for this path is the `DigitalTwin.untold` model, installed under: + +```text +Sources/BedroomTwin/GameData/Models/DigitalTwin/ + DigitalTwin.untold + Textures/ + ... +``` + +## Room Naming Reference + +The starter pack's `DigitalTwin.untold` model is already structured for this path: + +- Ordinary names for the room shell and static furniture (`Floor`, `Ceiling`, `Walls`, `Bed`, `Chair`, and similar) — these fall into the default `.contextGeometry` channel. +- A `WIN_` prefix on the window and its glass: `WIN_Window`, `WIN_Window_Glass`. +- An `NM_` prefix on the objects that stay selectable and carry mock digital-twin state. + +| Name | Role | +| --- | --- | +| `NM_Table_Lamp` | Light fixture — on/off, brightness | +| `NM_CeilingLamp_Left` / `NM_CeilingLamp_Right` | Light fixtures — on/off, brightness | +| `NM_Door` | Smart lock — open/closed, locked/unlocked | +| `NM_Curtains` | Smart blinds — position, automation | +| `NM_Laptop` | General smart device | + +This naming gives the engine enough structure to treat the room shell, the window, and operational objects differently. If you want to build this scene from your own source model instead of the starter pack, see [Replace The Starter Asset With Your Own Bedroom Scene](#replace-the-starter-asset-with-your-own-bedroom-scene). + +## Define Room Channels + +In your game scene code, define a custom scene channel for the window: + +```swift +extension SceneChannel { + static let windowGeometry = SceneChannel.userCustom(index: 0) +} +``` + +Then register the `WIN_` prefix before loading the room: + +```swift +registerSceneChannelPrefix("WIN_", channels: .windowGeometry) +``` + +When the exported entities are registered, objects named `WIN_Window` and `WIN_Window_Glass` will be assigned to `.windowGeometry`. + +Objects with the `NM_` prefix already use the engine's selectable-object convention. They default to `.selectableGeometry` and `.preserveIdentity`, which keeps them pickable and prevents them from being merged into context batches. + +## Configure Channel Behavior + +Set up the room behavior before or immediately after loading the asset: + +```swift +setSceneChannel(.contextGeometry, .pickParticipation(false)) +setSceneChannel(.contextGeometry, .renderMode(.normal)) + +setSceneChannel( + .windowGeometry, + .renderMode(.passthroughGhost(opacity: 0.0)) +) + +setSceneChannel( + .windowGeometry, + .lightPortal(.enabled( + intensity: 0.5, + range: 4.0, + useRealWorldTint: true, + maxActivePortals: 4, + activationDistance: 10.0 + )) +) +``` + +This setup does three things: + +- Context geometry such as walls, floor, and ceiling is not pickable. +- Window geometry becomes a passthrough ghost surface in mixed reality. +- Window geometry also acts as a light-portal source. + +For a more diagnostic look, switch context geometry to wireframe: + +```swift +setSceneChannel(.contextGeometry, .renderMode(.wireframe)) +``` + +For a cutaway view, hide context geometry: + +```swift +setSceneChannel(.contextGeometry, .renderMode(.hidden)) +``` + +Selectable `NM_` objects remain visible and pickable while the room shell changes render modes. + +## Load The Bedroom + +In `init()`, configure the engine systems, register the prefix, configure channels, then load the asset: + +```swift +configureEngineSystems() + +registerSceneChannelPrefix("WIN_", channels: .windowGeometry) + +setSceneChannel(.contextGeometry, .pickParticipation(false)) +setSceneChannel(.contextGeometry, .renderMode(.normal)) + +setSceneChannel( + .windowGeometry, + .renderMode(.passthroughGhost(opacity: 0.0)) +) + +setSceneChannel( + .windowGeometry, + .lightPortal(.enabled( + intensity: 0.5, + range: 4.0, + useRealWorldTint: true, + maxActivePortals: 4, + activationDistance: 10.0 + )) +) + +let room = createEntity() +setEntityName(entityId: room, name: "DigitalTwin") + +setEntityMeshAsync(entityId: room, filename: "DigitalTwin", withExtension: "untold") { success in + guard success else { + setSceneReady(false) + return + } + + loadSceneAuthored(filename: "DigitalTwin", withExtension: "untold") + setSceneReady(true) +} +``` + +`setEntityMeshAsync` looks for the asset by name in `GameData`, so the filename is `"DigitalTwin"` and the extension is `"untold"`. + +## Configure XR Input And Rendering + +Use the same Vision Pro setup pattern as the archviz path: + +```swift +private func configureEngineSystems() { + gameMode = true + + registerXREvents() + setInput(.xr(.pickingBackend(.octreeGPUPreferred))) + setInput(.xr(.twoHandRotateAxisMode(.dynamicSnapped))) + setInput(.xr(.sceneReady(true))) + + setRendering(.postProcessing(.enabled)) + setRendering(.antiAliasing(.msaa)) + setPostFX(.ssao(.enabled(false))) + + setRendering(.environment(.lightingMode(.realWorldEstimate))) + setRendering(.environment(.realWorldLightingContribution(1.0))) + + TextureStreamingSystem.shared.apply(.superdetailed) +} +``` + +The real-world lighting calls are useful when `useRealWorldTint` is enabled for the light portal. + +## Add Mock Digital-Twin State + +For the first version, keep the data local and simple. The tutorial is about the engine workflow, not live IoT infrastructure. + +```swift +struct TwinObjectInfo { + let title: String + let status: String + let detail: String +} + +let twinInfoByName: [String: TwinObjectInfo] = [ + "NM_Table_Lamp": TwinObjectInfo( + title: "Table Lamp", + status: "On", + detail: "Brightness: 80%" + ), + "NM_CeilingLamp_Left": TwinObjectInfo( + title: "Ceiling Lamp (Left)", + status: "On", + detail: "Brightness: 60%" + ), + "NM_CeilingLamp_Right": TwinObjectInfo( + title: "Ceiling Lamp (Right)", + status: "Off", + detail: "Brightness: 0%" + ), + "NM_Door": TwinObjectInfo( + title: "Bedroom Door", + status: "Closed", + detail: "Lock: Engaged" + ), + "NM_Curtains": TwinObjectInfo( + title: "Curtains", + status: "Online", + detail: "Position: 30% | Automation: Enabled" + ), + "NM_Laptop": TwinObjectInfo( + title: "Laptop", + status: "Sleep", + detail: "Battery: 62%" + ) +] +``` + +In a real app, this data could come from a building system, smart-home API, database, or web service. In this learning path, mocked state keeps the focus on selection and runtime behavior. + +## Select Room Objects + +A multi-material object exports as one entity per material — `NM_Table_Lamp` becomes `NM_Table_Lamp_mat0`, `NM_Table_Lamp_mat1`, and so on, each a top-level entity with no parent wrapper. Only single-material objects (`NM_Curtains`, `NM_Laptop` in this pack) export with their bare name intact. Strip a trailing `_mat` before looking up `twinInfoByName` so picking any submesh of a multi-material object still resolves: + +```swift +func twinLookupName(for entityId: EntityID) -> String { + var name = getEntityName(entityId: entityId) + if let range = name.range(of: #"_mat\d+$"#, options: .regularExpression) { + name.removeSubrange(range) + } + return name +} +``` + +Use spatial tap state to select `NM_` objects: + +```swift +private var selectedTwinObject: TwinObjectInfo? + +func handleInput() { + guard gameMode, isSceneReady() else { return } + + var state = getXRSpatialInputState() + + if state.spatialTapActive, let picked = state.pickedEntityId { + selectedTwinObject = twinInfoByName[twinLookupName(for: picked)] + + if let info = selectedTwinObject { + Logger.log(message: "\(info.title): \(info.status) - \(info.detail)") + } + } + + SpatialManipulationSystem.shared.processAnchoredSceneManipulationLifecycle( + from: state, + dragSensitivity: 10.0, + rotateSensitivity: 1.0 + ) +} +``` + +Because `.contextGeometry` picking is disabled, taps should ignore the room shell and return selectable `NM_` objects instead. + +In a full app, `selectedTwinObject` would drive a SwiftUI overlay, inspector panel, or floating label. For this path, logging the selected object is enough to confirm that the selection model works. + +## Add Runtime View Modes + +A digital twin viewer often needs more than one view of the same model. Use scene-channel render modes to expose simple modes: + +```swift +enum BedroomViewMode { + case normal + case wireframeShell + case hiddenShell + case passthroughWindow +} + +func applyViewMode(_ mode: BedroomViewMode) { + switch mode { + case .normal: + setSceneChannel(.contextGeometry, .renderMode(.normal)) + setSceneChannel(.windowGeometry, .renderMode(.normal)) + + case .wireframeShell: + setSceneChannel(.contextGeometry, .renderMode(.wireframe)) + setSceneChannel(.windowGeometry, .renderMode(.normal)) + + case .hiddenShell: + setSceneChannel(.contextGeometry, .renderMode(.hidden)) + setSceneChannel(.windowGeometry, .renderMode(.normal)) + + case .passthroughWindow: + setSceneChannel(.contextGeometry, .renderMode(.normal)) + setSceneChannel(.windowGeometry, .renderMode(.passthroughGhost(opacity: 0.0))) + } +} +``` + +This is the core digital-twin idea in the tutorial: the model is structured, so each category can have a different runtime behavior. + +## Verify Light Portals + +Use diagnostics when setting up the scene: + +```swift +let candidates = discoverSceneLightPortalCandidates() +let discovery = getLightPortalDiscoveryDiagnostics() +print(candidates) +print(discovery) +``` + +If no candidates are found, check: + +- window objects use the `WIN_` prefix +- `registerSceneChannelPrefix("WIN_", channels: .windowGeometry)` runs before loading +- window geometry has renderable bounds +- the channel has `.lightPortal(.enabled(...))` + +## What This Learning Path Demonstrates + +The bedroom scene behaves as a digital twin because the exported model has runtime structure: + +- The shell is context geometry. +- The window has a project-specific channel. +- Named room objects stay selectable. +- The light portal uses the window channel. +- Passthrough ghost rendering uses the same window channel. +- Spatial input lets the user inspect and manipulate the room. +- Mock metadata gives selections meaning beyond rendering. + +This is the foundation you would extend with live device data, building-management APIs, object-specific controls, annotations, or persistence. + +## Replace The Starter Asset With Your Own Bedroom Scene + +Once the starter pack works, the next step is exporting your own bedroom (or similar room) model. + +This path's starter pack was built from a stock archviz bedroom asset whose objects were not pre-named for the engine's conventions. Getting from a similar stock asset to something like `DigitalTwin.untold` means: + +1. Rename context geometry normally — the room shell and static furniture keep ordinary names. +2. Add a `WIN_` prefix to window/glass geometry only. Curtains, blinds, and other window dressing are not glazing — leave them as context geometry or a selectable twin object instead, since the light portal expects a transparent surface. +3. Add an `NM_` prefix to whatever should stay selectable and carry mock state — lamps, a door, blinds, small devices. + +Three cleanup steps mattered when preparing this pack's source model, and are worth checking on any similar stock asset: + +- **Drop baking-helper geometry.** The source file had a `Light_Blocking_Volume` mesh — an EEVEE baking helper sized to enclose the room, not meant to render at runtime. Exclude anything like it, or mark it hidden/non-pickable context geometry. +- **Parent light-fixture glow meshes to their fixture.** Small emissive "bulb" meshes are sometimes separate top-level objects rather than children of the fixture they belong to. Parent them before renaming, the same way glass should be parented under its window frame — otherwise selecting the fixture only picks up the shell, not its glow. +- **Check for Blender light and camera objects.** Stock EEVEE archviz assets sometimes bake all lighting into emissive materials and ship with zero `LIGHT`/`CAMERA` datablocks. `loadSceneAuthored` will still import color management, but there's nothing for it to bring in on the lighting/camera side. If you want real light contribution — especially relevant once `useRealWorldTint` is enabled for the light portal — add actual Blender light objects before exporting. + +Export the result to `.untold` with the Blender add-on or CLI exporter, place it under the project's `GameData` folder, and update the filename passed to `setEntityMeshAsync` and `loadSceneAuthored`. + +For example, if your exported model is `MyBedroom.untold`: + +```swift +setEntityMeshAsync(entityId: room, filename: "MyBedroom", withExtension: "untold") { success in + guard success else { + setSceneReady(false) + return + } + + loadSceneAuthored(filename: "MyBedroom", withExtension: "untold") + setSceneReady(true) +} +``` + +## Where To Go Next + +- [Scene Channels And Passthrough Rendering](../Tutorials/SceneChannelsTutorial.md) +- [Light Portals](../Tutorials/LightPortalsTutorial.md) +- [Spatial Input And Manipulation](../Tutorials/SpatialInputTutorial.md) +- [XR App Basics](../Tutorials/XRTutorial.md) +- [Blender Add-On Workflow](../Tutorials/BlenderAddonTutorial.md) +- [Export Assets With The CLI](../Tutorials/CLIExporterTutorial.md) diff --git a/docs/LearningPaths/CityStreamingOnVisionPro.md b/docs/LearningPaths/CityStreamingOnVisionPro.md new file mode 100644 index 00000000..affb6835 --- /dev/null +++ b/docs/LearningPaths/CityStreamingOnVisionPro.md @@ -0,0 +1,407 @@ +# City Streaming On Vision Pro + +This learning path turns a large city scene into a Vision Pro app that streams nearby geometry in and out of memory as the user moves through the city. + +The purpose is to show how Untold Engine handles scenes that are too large to load as one always-resident model: + +- The starter streamed-city asset ships as a tiled manifest. +- The app loads `LowPolyCity.json` with `setEntityStreamScene`. +- Tile geometry streams based on camera distance. +- LOD and HLOD representations cover farther parts of the city. +- Static batching updates automatically as streamed geometry becomes resident. +- Spatial debug overlays and profiler stats show what the streaming system is doing. + +By the end of the path, you will have a city-scale Vision Pro scene that loads incrementally instead of forcing the entire city into GPU memory at startup. + +## What You Will Build + +The example scene is a simplified city district with: + +- city blocks +- roads and sidewalks +- several buildings +- optional named landmarks +- a tiled streaming manifest +- Vision Pro spatial manipulation +- streaming debug overlays +- profiler output for residency and memory behavior + +This path is the large-scene counterpart to the single-asset archviz path. Use `setEntityMeshAsync` for smaller assets that should stay resident. Use `setEntityStreamScene` when the scene is large enough that only nearby geometry should be loaded. + +Want to run it right away instead of building it step by step? The finished project, **CityStreaming**, is in the [UntoldArcade](https://github.com/untoldengine/UntoldArcade) demo collection: [github.com/untoldengine/UntoldArcade/tree/main/CityStreaming](https://github.com/untoldengine/UntoldArcade/tree/main/CityStreaming). Clone it, open the Xcode project, and run. + +## Create The Vision Pro Project + +Create a standalone visionOS project: + +```bash +cd ~/Projects +untoldengine create CityStreaming --platform visionos +open CityStreaming/CityStreaming.xcodeproj +``` + +Open: + +```text +Sources/CityStreaming/GameScene.swift +``` + +## Install The Starter Streamed City Asset + +From the generated project folder, install the starter streamed-city asset pack: + +```bash +cd ~/Projects/CityStreaming +untoldengine assets install starter-streamed-city +``` + +The CLI finds the project's `GameData` folder and merges the asset files into it. + +The important files for this path are the `LowPolyCity.json` manifest and its `LowPolyCity/` tile folder, installed under: + +```text +Sources/CityStreaming/GameData/StreamModels/ + LowPolyCity.json + LowPolyCity/ +``` + +The manifest name passed to the engine is `"LowPolyCity"` and the extension is `"json"`: + +```swift +setEntityStreamScene(entityId: cityRoot, manifest: "LowPolyCity", withExtension: "json") { success in + setSceneReady(success) +} +``` + +## Load The City Manifest + +In `init()`, configure the engine and load the streamed scene under a stable root entity: + +```swift +configureEngineSystems() +configureCityStreaming() + +let cityRoot = createEntity() +setEntityName(entityId: cityRoot, name: "LowPolyCity") + +setEntityStreamScene(entityId: cityRoot, manifest: "LowPolyCity", withExtension: "json") { success in + setSceneReady(success) +} +``` + +A helper keeps the streaming setup isolated: + +```swift +private func configureCityStreaming() { + setGeometryStreaming(.enabled(true)) +} +``` + +The engine already uses sensible default values for tile/mesh/LOD/HLOD concurrency, query radius, and candidate selection, so enabling streaming is enough to see tiles load and unload as you move through the scene. See [Tune Streaming Behavior](#tune-streaming-behavior) below once this is working and you want to adjust it. + +`setEntityStreamScene` registers lightweight tile stub entities first. The engine then loads and unloads tile geometry as the camera moves. + +Do not call `generateBatches()` for streamed tiles. The streaming path updates batching incrementally as tile, LOD, and HLOD representations become resident. + +## Configure XR Input And Rendering + +Use the standard Vision Pro setup: + +```swift +private func configureEngineSystems() { + gameMode = true + + registerXREvents() + setInput(.xr(.pickingBackend(.octreeGPUPreferred))) + setInput(.xr(.twoHandRotateAxisMode(.dynamicSnapped))) + setInput(.xr(.sceneReady(true))) + + setRendering(.postProcessing(.enabled)) + setRendering(.antiAliasing(.msaa)) + setPostFX(.ssao(.enabled(false))) + +} +``` + +## Move Through The City + +For a Vision Pro city viewer, combine two locomotion modes: tap-to-teleport for large jumps across the city, and pinch-drag/two-hand rotate for fine positioning: + +```swift +private let maxTeleportDistanceMeters: Float = 40.0 + +func handleInput() { + guard gameMode, isSceneReady() else { return } + + let state = getXRSpatialInputState() + + // Tap a walkable surface to teleport there. + if state.spatialTapActive, + let hitPos = state.pickedEntityWorldPosition, + let normal = state.pickedEntityWorldNormal, + isWalkable(normal), + isWithinTeleportRange(hitPos) + { + translateSceneBy(delta: simd_float3(-hitPos.x, 0, -hitPos.z)) + } + + // Pinch + drag to move the scene root; two-hand pinch to rotate it. + SpatialManipulationSystem.shared.processAnchoredSceneManipulationLifecycle( + from: state, + dragSensitivity: 10.0, + rotateSensitivity: 1.0 + ) +} + +private func isWalkable(_ normal: simd_float3) -> Bool { + let slope = acos(simd_clamp(simd_dot(simd_normalize(normal), simd_float3(0, 1, 0)), -1, 1)) + return slope <= (.pi / 6) // ~30°, rejects building facades/walls +} + +private func isWithinTeleportRange(_ hitPos: simd_float3) -> Bool { + simd_length(simd_float3(hitPos.x, 0, hitPos.z)) <= maxTeleportDistanceMeters +} +``` + +Teleport and drag/rotate can be used together: tap a spot on the road or sidewalk to jump there instantly, or pinch-hold and move your hand to pan and rotate the city for finer adjustments. As the camera moves relative to the city — whether from a teleport jump or a drag — the streaming system updates tile residency. + +`isWalkable` rejects hits whose surface normal points more than ~30° away from world up, so taps on building facades or steep walls are ignored. `isWithinTeleportRange` caps how far a single tap can jump; keep `maxTeleportDistanceMeters` at or below your manifest's `queryRadius` (see [Tune Streaming Behavior](#tune-streaming-behavior)) so a teleport destination is guaranteed to already be inside the streaming system's load range. + +For a desktop-style debug build, you can also use free-fly camera movement as shown in the Large Scene Streaming Demo. + +## Tune Streaming Behavior + +Once the default streamed scene is working, adjust concurrency inside `configureCityStreaming()`: + +```swift +setGeometryStreaming(.tileConcurrency(2)) +setGeometryStreaming(.meshConcurrency(3)) +setGeometryStreaming(.lodConcurrency(4)) +setGeometryStreaming(.hlodConcurrency(4)) +``` + +Configure candidate selection: + +```swift +setGeometryStreaming(.queryRadius(120.0)) +setGeometryStreaming(.frustumGate(.enabled( + meshPadding: 6.0, + tilePadding: 8.0 +))) +setGeometryStreaming(.velocityLookAhead(time: 0.5, minSpeed: 1.5)) +setGeometryStreaming(.candidateSorting(importance: true, occlusion: true)) +``` + +These calls do not load content by themselves. They configure how the streaming system behaves after a manifest-backed scene is registered. + +Use a `queryRadius` large enough to cover the farthest `unload_radius` in the manifest. If the query radius is too small, out-of-range tiles may not be discovered for unloading. + +## Add Spatial Debug Overlays + +Tile bounds are the first overlay to enable when validating a manifest: + +```swift +setSpatialDebug(.tileBounds(enabled: true, maxTileNodeCount: 500)) +``` + +Use octree residency coloring when diagnosing streaming behavior: + +```swift +setSpatialDebug(.octreeLeafBounds(.enabled( + maxLeafNodeCount: 0, + occupiedOnly: true, + colorMode: .residency +))) +``` + +Use LOD and texture-tier overlays when tuning representation quality: + +```swift +setSpatialDebug(.lodLevels(true)) +setSpatialDebug(.textureStreamingTiers(true)) +``` + +These overlays help answer practical questions: + +- Are tile bounds where you expected them? +- Are nearby tiles resident? +- Are distant tiles using LOD or HLOD representations? +- Are textures dropping to lower tiers under memory pressure? + +Disable overlays before judging final visual quality. + +## Inspect Streaming Stats + +Log high-level streaming state during development: + +```swift +let stats = GeometryStreamingSystem.shared.getStats() +Logger.log(message: stats.description) +``` + +For a fuller snapshot: + +```swift +GeometryStreamingSystem.shared.printStats() +``` + +Useful fields: + +| Field | Meaning | +| --- | --- | +| `loadedCount` | Streaming entities and tiles currently resident. | +| `loadingCount` | Entities and tiles currently loading. | +| `unloadedCount` | Entities and tiles not resident. | +| `activeLoads` | Async loads currently in flight. | +| `loadCandidates` | Nearby candidates eligible this update. | +| `pendingLoadBacklog` | Candidates waiting because slots are full. | + +If `pendingLoadBacklog` stays high, the scene may be slot-starved. Lower tile size, tune manifest radii, or adjust concurrency after testing on device. + +## Enable Profiler Output + +The engine profiler gives better long-running visibility than ad hoc logs: + +```swift +setEngine(.metrics(.enabled)) +setEngineStatsLogging( + enabled: true, + profile: .verbose, + intervalSeconds: 1.0 +) +``` + +Watch these lines: + +- `Streaming` for residency counts, active loads, candidates, backlog, and upload gate cost. +- `TileReps` for full / LOD / HLOD representation residency and overlap. +- `TileRenderCost` for draw and triangle cost by representation tier. +- `Memory` for mesh and texture budget pressure. + +If the profiler shows frequent memory pressure, reduce tile size, reduce texture size, lower concurrency, or use stronger LOD/HLOD coverage. + +## Remote City Streaming + +Once local streaming works, the same API can load a remote manifest: + +```swift +let cityRoot = createEntity() +setEntityName(entityId: cityRoot, name: "RemoteCity") + +if let url = URL(string: "https://cdn.example.com/City/City.json") { + setEntityStreamScene(entityId: cityRoot, url: url) { success in + setSceneReady(success) + } +} +``` + +Remote manifests are downloaded and cached locally. Tile, LOD, HLOD, and texture URLs are resolved relative to the manifest URL and fetched on demand. + +Use local manifests first. Move to remote streaming only after the manifest, tile bounds, LODs, and memory behavior are correct. + +## Switching Cities Or Sessions + +If your app switches from one streamed scene to another, clear parsed tile memory before starting the new streaming session: + +```swift +GeometryStreamingSystem.shared.forceUnloadAllParsedTiles() +destroyEntity(entityId: oldCityRoot) +``` + +Then create a new root and call `setEntityStreamScene` for the next manifest. + +Do not call `forceUnloadAllParsedTiles()` during normal camera movement. Distance-based unload handles ordinary traversal. + +## What This Learning Path Demonstrates + +The city scene works because the engine treats the city as a streamed world instead of one giant mesh: + +- `LowPolyCity.json` describes spatial tiles. +- Tile stubs register quickly. +- Nearby tiles load asynchronously. +- Farther tiles use LOD or HLOD representations. +- Tiles unload when they leave range. +- Texture quality can adapt with distance and memory pressure. +- Batching updates as streamed geometry becomes resident. +- Profiler and spatial debug overlays show the runtime behavior. + +This is the foundation for larger spatial visualization apps: city models, campuses, infrastructure, industrial sites, and large real-world scans. + +## Replace The Starter Asset With Your Own City Scene + +Once the starter streamed-city asset works, the next step is to export your own tiled city scene. + +Start with a city model that is organized into meaningful areas. The scene does not need to be huge for the first pass. A few blocks are enough to validate the workflow. + +Example source structure: + +```text +Road_Main +Road_Second +Sidewalk_North +Sidewalk_South +Block_A +Block_B +Block_C +Building_A_01 +Building_A_02 +Building_B_01 +Building_C_01 +``` + +Use `NM_` only for landmarks or objects that must remain individually selectable: + +```text +NM_CityHall +NM_TransitStation +NM_Tower +NM_PublicArt_01 +``` + +For most city geometry, avoid thousands of tiny independent objects. Merge or organize small details where possible before export. The tiled pipeline is designed for large spatial regions, not for treating every bolt, sign, or curb segment as a unique gameplay object. + +Export the city with the [Blender Add-On Workflow](../Tutorials/BlenderAddonTutorial.md) which allows you to set the partitioning, streaming radius, unload radius, etc., as shown below. + +![Tile Runtime Preview](../images/PreviewRuntimeLOD.png) + +The output should look like this, with the tile folder named after the manifest: + +```text +City.json +City/ +``` + +`City.json` is the manifest. It describes the tile bounds, streaming radii, optional per-tile LODs, optional HLODs, and tile payload files. + +`City/` contains the tile assets that the engine loads on demand. + +Important manifest concepts: + +| Field | Purpose | +| --- | --- | +| `streaming_radius` | Distance where a tile's full representation should become visible. | +| `unload_radius` | Distance where the tile can be unloaded. | +| `prefetch_radius` | Distance where background parsing can begin before the tile is needed. | +| `lod_levels` | Intermediate per-tile representations. | +| `hlod_levels` | Coarse far-distance representations. | + +Place `City.json` and the `City/` tile folder under the project's `GameData/StreamModels`, then update the filename passed to `setEntityStreamScene`: + +```swift +setEntityStreamScene(entityId: cityRoot, manifest: "City", withExtension: "json") { success in + setSceneReady(success) +} +``` + +For a CLI-based export, see [Export Assets With The CLI](../Tutorials/CLIExporterTutorial.md). + +## Where To Go Next + +- [Large Scene Streaming Demo](../Tutorials/LargeSceneStreamingDemo.md) +- [Geometry Streaming](../API/UsingGeometryStreamingSystem.md) +- [LOD + Batching + Streaming](../API/UsingLOD-Batching-Streaming.md) +- [Spatial Debugger](../API/SpatialDebugger.md) +- [Profiler](../API/UsingProfiler.md) +- [Tile-Based Streaming Architecture](../Architecture/tilebasedstreaming.md) +- [Asset Remote Streaming](../Architecture/asset_remote_streaming.md) + diff --git a/docs/LearningPaths/index.md b/docs/LearningPaths/index.md index 83fd9f89..46725471 100644 --- a/docs/LearningPaths/index.md +++ b/docs/LearningPaths/index.md @@ -9,10 +9,9 @@ Use the tutorials when you want to understand a specific engine demo or API. Use | Path | What You Build | Main Systems | | --- | --- | --- | | [Archviz To Vision Pro](ArchvizToVisionPro.md) | A Blender architectural visualization scene running on Apple Vision Pro | Xcode project creation, `GameData`, async asset loading, scene-authored data, XR input, spatial manipulation | +| [Bedroom Digital Twin](BedroomDigitalTwin.md) | A structured bedroom model with selectable objects, a light portal, and passthrough window behavior | Scene channels, light portals, passthrough ghost rendering, XR picking, mock twin state | +| [City Streaming On Vision Pro](CityStreamingOnVisionPro.md) | A city-scale scene that streams nearby tiles instead of loading the whole world at startup | Tile manifests, geometry streaming, LOD/HLOD, texture streaming, spatial debug, profiler | ## Planned Paths -- Digital Twin Walkthrough -- City Streaming - Interactive Product Viewer - diff --git a/docs/images/gifs/archviz-demo.gif b/docs/images/gifs/archviz-demo.gif new file mode 100644 index 00000000..d8972949 Binary files /dev/null and b/docs/images/gifs/archviz-demo.gif differ diff --git a/docs/images/gifs/coolsaber-demo.gif b/docs/images/gifs/coolsaber-demo.gif new file mode 100644 index 00000000..4305fa77 Binary files /dev/null and b/docs/images/gifs/coolsaber-demo.gif differ diff --git a/docs/index.md b/docs/index.md index 73e30171..14511dbb 100644 --- a/docs/index.md +++ b/docs/index.md @@ -32,7 +32,7 @@ Untold Engine is built for developers and teams who: Creator & Lead Developer: [Harold Serrano](http://www.haroldserrano.com) -[![untoldengine-image](images/engine-highlight-5.png)](https://vimeo.com/1176995991?fl=ip&fe=ec) +![archviz-demo](images/gifs/archviz-demo.gif) --- @@ -52,18 +52,18 @@ workflow, product experience, and customer-specific behavior. ## Watch It in Action — Apple Vision Pro Demos - - - - - - - - - - - -
+| Demo | Description | +| --- | --- | +| [CoolSaber](https://github.com/untoldengine/UntoldArcade/tree/main/CoolSaber) | PSVR2-driven lightsaber duels, blades clashing over SharePlay | +| [CoolWater](https://github.com/untoldengine/UntoldArcade/tree/main/CoolWater) | Real-time animated water with reflection, refraction, and ripples | +| [CoolCloth](https://github.com/untoldengine/UntoldArcade/tree/main/CoolCloth) | GPU cloth simulation (XPBD) you can punch a ball through | +| [Archviz Viewer](https://untoldengine.github.io/UntoldEngine/LearningPaths/ArchvizToVisionPro/) | Blender-authored architectural scene walked through in mixed reality | +| [Bedroom Digital Twin](https://untoldengine.github.io/UntoldEngine/LearningPaths/BedroomDigitalTwin/) | Tap-to-inspect digital twin bedroom with live mock status data | +| [City Streaming](https://untoldengine.github.io/UntoldEngine/LearningPaths/CityStreamingOnVisionPro/) | City-scale scene streamed in tile by tile with LOD/HLOD | + +Full source for every demo above lives in the [UntoldArcade](https://github.com/untoldengine/UntoldArcade) repo. + +![coolsaber-demo](images/gifs/coolsaber-demo.gif) ## Try the Engine Right Now @@ -100,8 +100,6 @@ The demos live under `Sources/Demos`. They are runnable when working inside the engine repository, but are not exposed as products to apps that add Untold Engine as a package dependency. -![untoldengine-image-2](images/engine-highlight-6.png) - --- ## Getting Started diff --git a/mkdocs.yml b/mkdocs.yml index 35711af8..b13fc974 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -46,6 +46,8 @@ nav: - Learning Paths: - Overview: LearningPaths/index.md - Archviz To Vision Pro: LearningPaths/ArchvizToVisionPro.md + - Bedroom Digital Twin: LearningPaths/BedroomDigitalTwin.md + - City Streaming On Vision Pro: LearningPaths/CityStreamingOnVisionPro.md - Tutorials: - Overview: Tutorials/index.md - Starter Demo: Tutorials/StarterDemo.md