DimSim scene editing/authoring in JS + moving inside dimos

[Viswa4599]

Brings DimSim into misc/DimSim/, wires it up as a first-class simulation backend, and replaces the legacy JSON authoring + sticky-state plumbing with a standard Three.js JS dev cycle for scenes and a JS-native eval system.

What this does

• DimSim lives inside dimos. Continues Paul's feat: include DimSim #2081 vendoring with a cleaned-up source tree (misc/DimSim/{src,cli,evals,scenes,public,docs}) — no separate repo to clone, dimos drives it directly via --simulation dimsim.
• Scene authoring is now JS. Each scene is one file at scenes/<name>/index.js that default-exports async build(api) and uses the engine's THREE / Rapier / physics helpers — no JSON, no editor sidebar plumbing.
• Apartment ported to JS. 97 MB apt.json decomposed into scenes/apartment/{data/*.js, textures/*} (~6 MB total) and fed through loadLevel; full interactivity (pickables, door states, TV toggle) preserved.
• Eval system collapsed to one file per workflow. scenes/<env>/evals/<name>.js is a runnable program that import { runEval } from '@dimsim/eval' and calls it; works both in the browser (via Vite-pinned harness chunk) and via deno run (via Deno import map) — same file, two runtimes.
• dimsim CLI shipped. dimsim dev, dimsim eval list, dimsim eval <workflow> (auto-connect), dimsim eval --headless ....

Docs

• misc/DimSim/docs/getting-started.md — 5-minute tour
• misc/DimSim/docs/scenes.md — create + edit scenes
• misc/DimSim/docs/evals.md — author eval workflows

Validation checklist (against dimos#1691)

• Write a scene from scratch in JS (scenes/warehouse/index.js)
• Edit the existing apartment scene in JS (scenes/apartment/)
• Robot embodiment API — setEmbodiment({...}) scene-side API + bridge ServerPhysics/ServerLidar live reconfigure
• Load a third-party map (Sketchfab GLB) into a scene
• Load an arbitrary GLB via loadGLTF
• Load assets from LFS

Follow-ups (separate)

• Decompose the apartment into pure Three.js + a registerable-object library so its interactive contents (pickables, doors, TV) can be authored in JS like everything else, not as apt-shape loadLevel data.
• Move objects and textures to LFS

[greptile-apps]

Greptile Summary

This PR brings DimSim in-tree under misc/DimSim/, replaces JSON-based scene authoring with JS modules (Three.js + Rapier), ports the apartment scene, introduces a dimsim CLI, and wires DimSim as a first-class simulation backend driven directly from the dimos repo.

• Python side: dimsim_process.py drops the external git-clone in favor of the vendored copy; a DIMSIM_LOCAL override is validated before use. websocket_server.py fixes a TypeError: float(None) crash on 2D-panel clicks.
• CLI/bridge: A Deno CLI (cli.ts) orchestrates dev, eval, and agent subcommands; bridge/server.ts multiplexes control vs sensor WebSocket streams with per-channel LCM isolation for parallel evals.
• Eval runner: runner.ts discovers workflows by filesystem walk and runs them sequentially (or in parallel via runEvalsMultiPage); a routing bug causes parallel-eval sockets to land on the sensor path where text frames are dropped, silently breaking the --parallel N path.

Confidence Score: 4/5

Safe to merge for single-page eval and dev workflows; the parallel eval path (--parallel N) is broken and will produce no results.

The Python integration, single-page eval flow, and bridge LCM relay are all solid. The one concrete defect is in runEvalsMultiPage: it connects sockets with ?ch=page-0 (the sensor routing param) instead of ?channel=page-0&ch=control, so every parallel-eval socket ends up in the sensor handler where text frames are silently discarded. Any CI run using --headless --parallel N (N > 1) will hang until timeout with zero results.

misc/DimSim/evals/runner.ts — the runEvalsMultiPage socket URL construction on line 133.

Important Files Changed

Filename	Overview
misc/DimSim/evals/runner.ts	Multi-page eval orchestrator; runEvalsMultiPage opens sockets with ?ch=page-0 (wrong param) instead of ?channel=page-0&ch=control, routing each connection to the sensor path where text frames are dropped — parallel evals silently produce no results. Single-page path is correct.
misc/DimSim/cli/bridge/server.ts	Bridge server handling WebSocket routing, LCM relay, and Rapier snapshot ingestion; single-page control/sensor separation is correct, but the multi-page channel routing relies on the channel query param which the runner currently doesn't supply correctly.
misc/DimSim/cli/cli.ts	Main CLI entry point; dev/eval/agent subcommands wired correctly; existing known issue with stale apt default scene in the dev subcommand (already tracked).
dimos/simulation/dimsim/dimsim_process.py	Replaces external git-clone approach with vendored misc/DimSim/; _resolve_dimsim_dir correctly handles the DIMSIM_LOCAL override and validates the resolved path before use.
dimos/visualization/rerun/websocket_server.py	Bug fix: replaces float(msg.get(x, 0)) pattern with a _num() helper that handles explicit None values (e.g., from 2D-panel clicks) that would previously crash with TypeError: float(None).
.github/workflows/dimsim-check.yml	Adds a CI check for the DimSim subtree: npm build, Deno type-check. Path-filtered correctly so it only runs on misc/DimSim/** changes.
dimos/simulation/dimsim/scene_client.py	Updates all robot embodiment presets to point at the new stub GLB (dimsim_unitree_stub.glb) that ships inside the repo, removing the dependency on external GLB files.

Sequence Diagram

sequenceDiagram
    participant PY as dimsim_process.py
    participant CLI as cli.ts (Deno)
    participant BRG as bridge/server.ts
    participant PW as Playwright (headless)
    participant ENG as engine.js (browser)
    participant RUN as runner.ts

    PY->>CLI: spawn deno run cli.ts --scene apartment --port 8090 --headless
    CLI->>BRG: "startBridgeServer({port, scene, headless})"
    BRG-->>BRG: open HTTP+WS server on :8090
    CLI->>PW: launchHeadless(url)
    PW->>BRG: GET / → injected HTML (window.__dimosScene)
    PW->>ENG: dynamic import /scenes/apartment/index.js
    ENG->>BRG: "WS connect ?ch=control (control socket)"
    ENG->>BRG: "WS connect ?ch=sensor (sensor socket)"
    ENG->>BRG: send Rapier snapshot (DSS2 binary)
    BRG-->>BRG: initServerSystems(snapshot)
    BRG->>ENG: pose updates (JSON)
    RUN->>BRG: "WS connect ?ch=control"
    RUN->>BRG: "send {type:runEval, workflowUrl}"
    BRG->>ENG: relay runEval message
    ENG-->>ENG: dynamic import workflow, poll success()
    ENG->>BRG: "send {type:evalResult, ...}"
    BRG->>RUN: relay evalResult
    RUN-->>CLI: EvalResult[]

Loading

_{Reviews (2): Last reviewed commit: "Update misc/DimSim/cli/bridge/server.ts" | Re-trigger Greptile}

[greptile-apps]

Stale default scene name breaks dimsim dev

The fallback is "apt" but the apartment scene was renamed to "apartment" (its directory is scenes/apartment/). Any developer running dimsim dev without --scene will load /scenes/apt/index.js, which returns 404, resulting in a blank scene. The Python-driven path always passes --scene apartment explicitly, so only standalone CLI usage is affected — but that's the primary path documented in docs/getting-started.md.

Suggested change

	const scene = (opts.scene as string) || "apt";
	const scene = (opts.scene as string) || "apartment";

[greptile-apps]

Unescaped scene name injected into HTML script tag

activeSceneName is interpolated directly into the <script> block without HTML/JS escaping. If a user passes --scene 'foo";window.open("http://evil")//' (or a specially crafted DIMSIM_SCENE env var), the quote terminates the string literal and injects arbitrary JS. Because this is a local dev server the blast radius is limited, but sanitising the value (e.g. allowing only [a-zA-Z0-9_-]) would prevent the class of mistake entirely.

[greptile-apps]

_runOne hangs indefinitely on WebSocket close/error

The returned Promise resolves only on a matching evalResult message. If the bridge crashes, the socket closes, or the eval workflow import fails silently on the browser side, the Promise never settles and the entire runEvals call stalls. Adding an onerror/onclose handler that rejects (or resolves with a failure result) would prevent the runner from hanging in CI.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!

[greptile-apps]

Multi-page eval sockets routed to sensor path, not control

?ch=page-0 sets the ch query param, which the bridge uses to distinguish control vs sensor sockets (isSensor = ch !== "control"). A channel name like "page-0" is not "control", so every socket opened here lands in the sensor onmessage handler — which immediately drops text frames with if (!(event.data instanceof ArrayBuffer) …) return. The runEval JSON command is a text frame, so it is silently discarded. All parallel eval workflows will hang indefinitely waiting for an evalResult reply that never arrives.

The multi-channel routing key is channel (not ch). The URL should be ?channel=${ch}&ch=control so the socket is both directed to the right channel state and treated as a control socket.

Suggested change

	const sockets = await Promise.all(
	options.channels.map((ch) => _connect(`${options.wsUrl}/?ch=${encodeURIComponent(ch)}`)),
	);
	const sockets = await Promise.all(
	options.channels.map((ch) => _connect(`${options.wsUrl}/?channel=${encodeURIComponent(ch)}&ch=control`)),
	);