Write masks as pyramidal OME-TIFF with pixel size metadata#8
Open
Yu-AnChen wants to merge 2 commits into
Open
Conversation
Replace the pad_block + tifffile.imwrite output path with a new write_label_pyramid helper that: - Builds a factor-2 pyramid down to the tile size, downsampling by strided slicing so integer label IDs are preserved exactly. - Materializes each coarse level to a temp zarr and streams every level to the TIFF tile by tile, keeping peak memory bounded. - Embeds PhysicalSizeX/Y (µm) and TIFF resolution tags from the pixel size. Edge padding now lives inline in the writer, so pad_block is removed. TIFF encoding stays single-threaded (maxworkers=1). predictor is kept on for the uint32 label images and off for the binary discard mask, as before. Also normalize the micro sign in log messages from U+03BC to U+00B5 to match the OME metadata strings.
Replace the temp-zarr pyramid build with the Ashlar/ome-tiff-pyramid approach: each coarser level is generated by reading the previous level back from the output OME-TIFF as it is written, so no intermediate store is materialised to disk. Downsampling stays strided (b[::2, ::2]) to preserve integer label IDs exactly. Switch compression from zlib to zstd (lossless, ~20% smaller and faster on real masks) and stop forcing maxworkers=1, letting tifffile compress tiles across ~half the cores. On a 39760x84959 mask this is ~2.7x faster and ~23% smaller than the previous zlib/single-threaded path; output verified bit-exact and label-preserving under both zarr v2 and v3. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replace the
pad_block+tifffile.imwriteoutput path with a newwrite_label_pyramidhelper that:The explicit edge-tile padding is dropped (
pad_blockremoved): undersized edge tiles are yielded as-is and tifffile zero-pads them up to the full tile size internally. Tiles are compressed with zstd (lossless, and both faster and ~20% smaller than zlib on real masks) using tifffile's defaultmaxworkers— which compresses tiles across ~half the available cores, is CPU-affinity aware, and is overridable viaTIFFFILE_NUM_THREADS— rather than forcing single-threaded encoding.predictoris kept on for the uint32 label images and off for the binary discard mask, as before.On a 39760×84959 nucleus mask, writing the full pyramid is ~2.7× faster and ~23% smaller than the previous zlib / single-threaded path. Output verified bit-exact and label-preserving, and tested under both zarr v2 and v3.
Also normalize the micro sign in log messages from U+03BC to U+00B5 to match the OME metadata strings.