Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
61001a7
Add a new feature: gen_sim/scene_engine
MuziWong Jul 28, 2026
ad708e8
Using vhacd by default in the simulation environment
MuziWong Jul 28, 2026
b664562
RAN black
MuziWong Jul 29, 2026
fe6c1af
style(geometry-generation): fix CI formatting
MuziWong Jul 29, 2026
4c0cfc7
Updated geometry generation client
MuziWong Jul 30, 2026
a67ac77
Modified the gym export to scene export
MuziWong Jul 30, 2026
c630f16
Make the 2D AABB optimization more robust
MuziWong Jul 30, 2026
8ae888d
Add optional --config in cli
MuziWong Jul 30, 2026
c38c0fc
Register scene-engine and preview-scene in embodichain.__main__.COMMANDS
MuziWong Jul 30, 2026
b86c3b5
Fixed with the suggestion from Copilot
MuziWong Jul 30, 2026
c99182c
Added __init__.py and ran black.
MuziWong Jul 30, 2026
e7f3fbe
Merge branch 'main' of github.com:DexForce/EmbodiChain into muzi/feat…
XuanchaoPENG Jul 31, 2026
79442f5
Fix import bug
MuziWong Jul 31, 2026
a0f6797
Add Viser support
XuanchaoPENG Jul 31, 2026
db438e7
Merge branch 'muzi/feat_scene_engine' of github.com:DexForce/EmbodiCh…
XuanchaoPENG Jul 31, 2026
d1b363a
test(scene_engine): add unit coverage
MuziWong Jul 31, 2026
23947e5
Add test for the newly-modified scene-preview
MuziWong Jul 31, 2026
b0774a4
docs(scene_engine): add usage documentation
MuziWong Jul 31, 2026
ede8172
Change logger
MuziWong Jul 31, 2026
73e285e
Correct the table id verification comment, make it more clear
MuziWong Jul 31, 2026
244ab1b
Delete a bad comment line in scene_segmentation_utils.py
MuziWong Jul 31, 2026
930af9b
Align the doc with the scene_engine_config
MuziWong Jul 31, 2026
05f8a1a
Align the scene_engine config setup with the simready_pipeline
MuziWong Jul 31, 2026
d0d39ca
fix(scene_engine): validate geometry output object IDs
MuziWong Jul 31, 2026
114734d
fix(scene_engine): restrict preview mesh paths
MuziWong Jul 31, 2026
a6a3219
Changed support region detection algo + Changed the 2d-aabbs-optimiza…
MuziWong Aug 3, 2026
3b1abc2
Reformatted the gravity settlement
MuziWong Aug 3, 2026
dd66de8
Reformatted assets group table aligner + Reformatted simready tools
MuziWong Aug 3, 2026
bbf2a06
Modified the scene segmentation: 1. Bold the asset outline 2. Avoid t…
MuziWong Aug 4, 2026
c45be97
Reformatted the scene export
MuziWong Aug 4, 2026
2436a37
Update the datastructure + Reformatted the pipeline
MuziWong Aug 4, 2026
5eb9134
Read .env for now
MuziWong Aug 4, 2026
2c9477d
Deleted all docs and all tests
MuziWong Aug 4, 2026
2d64a80
Deleted json config in setup.py
MuziWong Aug 4, 2026
f51bc5d
Merge remote-tracking branch 'origin/main' into muzi/feat_scene_engin…
MuziWong Aug 4, 2026
82acb65
Added md + rst in docs/
MuziWong Aug 4, 2026
8979f68
Modified help, description and epilog
MuziWong Aug 4, 2026
4f4102b
Added test files
MuziWong Aug 4, 2026
d710331
Added extra dependencies in gen_sim: scene_engine + Added CI
MuziWong Aug 4, 2026
2298e54
Fixed CI lint
MuziWong Aug 4, 2026
067d40e
Fixed scene-engine test file: make it have different name with the si…
MuziWong Aug 4, 2026
546be9b
Fixed dependencies in toml: scene_engine is under the management of g…
MuziWong Aug 4, 2026
27f94b9
Fixed scene export bug + Deleted md in cli/ + Added test for the modi…
MuziWong Aug 4, 2026
a5390af
fix: remove dexsim-provided gensim dependencies
MuziWong Aug 5, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,11 @@ jobs:
--extra-index-url https://download.blender.org/pypi/
pip install pytest-xdist
- name: Verify Scene Engine gensim installation
run: |
python -c "import matplotlib, numpy, open3d, requests, scipy, shapely, trimesh; from PIL import Image; import embodichain.gen_sim.scene_engine.pipeline.generate"
pytest tests/gen_sim/scene_engine -q
- name: Run default tests
run: |
echo "Default test suite (GPU-marked tests are skipped)"
Expand Down
1 change: 1 addition & 0 deletions docs/source/features/generative_sim/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ Generative Simulation collects EmbodiChain features for generating simulation-re
:maxdepth: 2

SimReady Asset Pipeline <simready_pipeline.md>
Scene Engine <scene_engine.md>
83 changes: 83 additions & 0 deletions docs/source/features/generative_sim/scene_engine.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Scene Engine

Scene Engine reconstructs a table-top scene from one image. It identifies the
table and visible objects, segments their masks, generates simulation-ready
meshes, refines the object layout on the table, and exports a scene that can be
loaded by EmbodiChain.

## Quick Start

Install EmbodiChain with the `gensim` extra first; see
[Installation](../../quick_start/install.md#optional-generative-simulation-gensim).

Configure the required services in `embodichain/gen_sim/.env`, then generate a
scene:

```bash
embodichain scene-engine \
--image /path/to/scene.png \
--output_root /path/to/scene_output
```

The same command is available through the package entry point:

```bash
python -m embodichain scene-engine \
--image /path/to/scene.png \
--output_root /path/to/scene_output
```

## Configuration

Scene Engine reads the LLM, segmentation, and geometry-generation settings
from `embodichain/gen_sim/.env`:

```bash
OPENAI_API_KEY="your-api-key"
OPENAI_MODEL="your-model"
OPENAI_BASE_URL="https://api.openai.com/v1"
SCENE_ENGINE_OPENAI_DEFAULT_QUERY="{}"
OPENAI_MAX_ATTEMPTS=3

SCENE_ENGINE_IMAGE_SEGMENTATION_BASE_URL="http://host:port"
SCENE_ENGINE_IMAGE_SEGMENTATION_TIMEOUT_S=30
SCENE_ENGINE_IMAGE_SEGMENTATION_MAX_ATTEMPTS=3
SCENE_ENGINE_IMAGE_SEGMENTATION_HEALTH_PATH="/health"
SCENE_ENGINE_IMAGE_SEGMENTATION_SINGLE_OBJECT_PATH="/predict"

SCENE_ENGINE_GEOMETRY_GENERATION_BASE_URL="http://host:port"
SCENE_ENGINE_GEOMETRY_GENERATION_TIMEOUT_S=600
SCENE_ENGINE_GEOMETRY_GENERATION_MAX_ATTEMPTS=3
SCENE_ENGINE_GEOMETRY_GENERATION_HEALTH_PATH="/health"
SCENE_ENGINE_GEOMETRY_GENERATION_OBJECTS_PATH="/generate_multiple_objects"
```

## Processing Flow

- **Scene understanding**: analyzes the image and segments the table and visible objects.
- **Scene generation**: generates meshes, prepares SimReady geometry, detects the table support surface, and refines the table-top layout.
- **Scene export**: copies the final GLBs and writes a portable z-up scene export.

## Output and Preview

The important final outputs are:

```text
scene_output/
|-- scene_understanding/ # Object analysis, masks, and stage JSON
|-- scene_generation/ # Generated, SimReady, and layout-debug artifacts
`-- scene_export/
|-- mesh_assets/ # Final GLBs
`-- scene_config.json # Exported scene description
```

Validate the export without opening a window:

```bash
embodichain preview-scene \
--output_root /path/to/scene_output \
--headless
```

For an interactive preview, omit `--headless`. Add `--viser` to publish the
scene through Viser.
10 changes: 10 additions & 0 deletions embodichain/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,16 @@ class Command:
target="embodichain.gen_sim.simready_pipeline.cli.start:main",
help="Convert a raw asset directory into a SimReady asset.",
),
Command(
name="scene-engine",
target="embodichain.gen_sim.scene_engine.cli.start:main",
help="Generate a scene export from an input image using gen_sim/.env.",
),
Command(
name="preview-scene",
target="embodichain.gen_sim.scene_engine.cli.preview:main",
help="Preview a generated Scene Engine scene export.",
),
Command(
name="preview-asset",
target="embodichain.lab.scripts.preview_asset:cli",
Expand Down
19 changes: 19 additions & 0 deletions embodichain/gen_sim/scene_engine/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# ----------------------------------------------------------------------------
# Copyright (c) 2021-2026 DexForce Technology Co., Ltd.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ----------------------------------------------------------------------------

from __future__ import annotations

__all__: list[str] = []
19 changes: 19 additions & 0 deletions embodichain/gen_sim/scene_engine/cli/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# ----------------------------------------------------------------------------
# Copyright (c) 2021-2026 DexForce Technology Co., Ltd.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ----------------------------------------------------------------------------

from __future__ import annotations

__all__: list[str] = []
Loading
Loading