Skip to content

Rewrite the Chung lab demo notebook for the OME-Zarr layout - #208

Draft
bendichter wants to merge 1 commit into
masterfrom
chunglab-demo-omezarr
Draft

Rewrite the Chung lab demo notebook for the OME-Zarr layout#208
bendichter wants to merge 1 commit into
masterfrom
chunglab-demo-omezarr

Conversation

@bendichter

Copy link
Copy Markdown
Member

Dandiset 000108 was reorganized after 2021-09-27_dandi-demo.ipynb was written. The image data moved from microscopy/*.SPIM.h5 to micr/*.SPIM.ome.zarr, and the JSON sidecars that gave each chunk's extent and offset were replaced by the OME-NGFF coordinateTransformations inside each store. The notebook has been on both exclusion lists since then, so it has neither badge and is not tested. This rewrites its data access for the current layout and removes both exclusions.

Draft because the dataset is not ours: I would like the Chung lab or DANDI staff to confirm the sample I picked is a reasonable one to show, and to look at the metadata gaps below.

What the notebook does now

The structure follows the original: find the chunks of one slab, show how they tile, read a whole-slab overview from the resolution pyramid, read a region at full resolution across a chunk boundary, and segment blood vessels in the LEC channel. What changed is how the data are found and placed.

Chunk geometry now comes from the OME-NGFF metadata rather than from sidecars, which makes the stitching considerably shorter than the code it replaces. Each store publishes its voxel size (2.564, 3.625, 2.564 um) and its position, and the chunks turn out to step 6649 um along y and overlap by 775 um, about 214 voxels, which matches the description in the original text. Reads are in micrometers, and the overlap is combined with a raised-cosine weight. The seam figure shows blending against a plain average side by side: the average leaves two visible intensity steps at the edges of the overlap and the blend does not.

The overview at level 6 is a couple of seconds of streaming, and it is displayed next to the photograph of the slab, which is a good check that the reconstruction matches the tissue.

Two changes go beyond a port. The interactive neuroglancer viewer is now a printed link, since a viewer bound to a local port is no use in Colab or CI, and the link is built the same way neuroglancer-stitched.py builds its links. And there is a short section reporting the slab z origins per stain, for the reason below.

Metadata gaps I ran into

These are in the dandiset, not in the notebook, and someone who knows the pipeline should probably look at them:

  • sample 41, stain LEC: chunks 1 to 5 report a z translation of 0, chunks 6 to 9 report 82000. The YO and NN chunks of the same sample all report 82000, and samples 40 and 42 sit at 80000 and 84000, so the slab spacing is otherwise regular and the five zeros look wrong.
  • sample 40, stain LEC, chunk 5: the level 0 dataset has no coordinateTransformations at all, so neither its voxel size nor its position is recorded.

Stitching a slab only needs the y positions, which are consistent everywhere I checked, so the notebook works regardless; it reads the geometry defensively and falls back to the voxel size implied by the pyramid when a level omits its transform. I added the reporting cell so a reader assembling slabs into a whole brain does not silently trust the z values. Happy to file this as a helpdesk issue if that is the better venue.

Verification

Both notebooks in the directory were run end to end through the CI harness on Python 3.13, in the same fresh-environment mode CI uses: the rewritten demo in 20 s, dashboard.ipynb in 162 s. The directory's requirements.in gains zarr, fsspec, aiohttp, pillow, requests and scikit-image, so dashboard.ipynb re-locks alongside it and the two stay in one image group, which means the published 000108-chunglab-demo image keeps its name and simply gains the second notebook.

🤖 Generated with Claude Code

https://claude.ai/code/session_011NuStoQykizhQCx2KBrMCN

Dandiset 000108 was reorganized after this notebook was written: the
image data moved from microscopy/*.SPIM.h5 to micr/*.SPIM.ome.zarr, and
the JSON sidecars that carried each chunk's extent and offset were
replaced by the OME-NGFF coordinateTransformations inside each store.
The notebook had been excluded from CI and from the Colab badge because
of it, and this rewrites the data access against the current layout.

The narrative is unchanged: find the chunks of one slab, look at how
they tile, read a whole-slab overview from the resolution pyramid, read
a region at full resolution across a chunk boundary, and segment blood
vessels in the LEC channel. The stitching is now driven by the voxel
size and position each store publishes, so it is much shorter than the
sidecar arithmetic it replaces, and the seam figure compares cosine
blending against a plain average to show what the blending buys.

Two changes of substance beyond the port. The interactive neuroglancer
viewer is now a printed link, since a viewer bound to a local port is
of no use in Colab or CI. And a cell reports the slab z origins per
stain, because they are not always recorded: in sample 41 the LEC
chunks disagree, five reporting 0 um and four 82000 um, and one chunk
of sample 40 carries no coordinateTransformations at all. Stitching
uses only the y positions, which are consistent, so the notebook is
unaffected, but the gap is worth surfacing rather than papering over.

The directory's requirements.in gains the packages the rewrite needs,
so dashboard.ipynb re-locks with it and the two continue to share one
container image. Both notebooks were run end to end on Python 3.13.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011NuStoQykizhQCx2KBrMCN
@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@github-actions

Copy link
Copy Markdown
Contributor

Index page preview

A preview of the generated index.html has been deployed for this PR:

View preview

Last updated: commit d513a66e199a6cb19b5d6b4bca1b7032a2156e45

github-actions Bot added a commit that referenced this pull request Aug 23, 2026
github-actions Bot added a commit that referenced this pull request Aug 23, 2026
@github-actions

github-actions Bot commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Executed notebooks

CI executes the notebooks changed in this PR through a Jupyter kernel and publishes each one, with its outputs, as soon as it finishes. Links appear below as notebooks complete; an hourglass means the notebook is still running.

2 of 2 finished. Commit d513a66. The executed copies live only in the preview and are removed when the PR closes; nothing is committed to the branch.

github-actions Bot added a commit that referenced this pull request Aug 23, 2026
@satra

satra commented Aug 23, 2026

Copy link
Copy Markdown
Member

could we set the overlay stitched slab to respect axis ratio of the underlying pixels? also after the blending demo, it would be nice to do a demo of the overlay with and without blending.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants