Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .codespellrc
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[codespell]
ignore-words-list = ND,nd,bu, dNe, slippy, tye
ignore-words-list = ND,nd,bu, dNe, slippy, tye, raison
skip = *.png,*.jpg,*.gif,*.pdf,*.svg
quiet-level = 3

4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -92,3 +92,7 @@ tests/.hypothesis
docs/_site
site
.mypy_cache

# Node.js (screenshot capture script in scripts/)
node_modules/
package-lock.json
14 changes: 12 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,16 @@
# Guidance for avoiding common pitfalls when producing and using datacubes.
# Datacube Guide

Status: This is a first draft of the guidance.
Practical guidance for producing, using, and visualizing datacubes.

Read the guide at [developmentseed.org/datacube-guide](https://developmentseed.org/datacube-guide/).

> Status: First draft, actively iterating.

## What's inside

- **Worst practices** — common pitfalls to avoid when producing and using multi-dimensional data products: chunking, metadata, datatypes, and default library configs (FSSpec, GDAL, Xarray).
- **Visualization** — a catalog and comparison of tools for visualizing Zarr-backed datacubes in the browser, covering both server-side dynamic tilers (TiTiler, Xpublish-tiles) and client-side rendering, with libraries for deck.gl, MapLibre/Mapbox, and Cesium, plus standalone viewer apps. Includes side-by-side comparisons and guidance on choosing an approach.
- **Benchmarking** — a small Python library (`datacube_benchmark`) for measuring read patterns and access costs.

## Installation

Expand Down
43 changes: 42 additions & 1 deletion docs/visualization/browzarr.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,44 @@
# Browzarr

Content coming soon. See [source repo](https://github.com/EarthyScience/Browzarr) for now.
A browser-native viewer for multi-dimensional Zarr and NetCDF datasets, with first-class 3D volumetric rendering. Maintained by the Max Planck Institute for Biogeochemistry (EarthyScience) under the Apache 2.0 license.

## At a glance

- **Repo** — [EarthyScience/Browzarr](https://github.com/EarthyScience/Browzarr)
- **Live demo** — [browzarr.io](https://browzarr.io)
- **Shape** — Viewer application (Next.js, static export). Not currently published as an embeddable npm package.
- **Render API** — three.js + react-three-fiber, GLSL3 shaders, with WebGPU support enabled via `webgpu-utils`
- **Zarr versions** — v2 and v3 via zarrita; also reads NetCDF (via `netcdf4-wasm`) and Icechunk stores (via `icechunk-js`)
- **Conventions** — CF `_ARRAY_DIMENSIONS` for axis naming

## What it does

Browzarr is the most "data-centric" of the client-side options listed here. The user points the hosted app at a remote Zarr URL or uploads a local file, and the dataset is rendered directly in the browser without a backing tile server. The distinguishing capability is genuine 3D volumetric rendering of Zarr cubes, not just slices of them.

## How it renders

Three rendering modes share the same data pipeline.

- **Data Cube** — full 3D volume rendering via fragment-shader raycasting, with both orthographic and perspective cameras. Implemented in `volFragment.glsl`.
- **FlatMap** — 2D orthographic slicing with pan and zoom. Implemented in `flatFrag.glsl`.
- **Sphere** — globe rendering with displacement mapping for elevation, useful for global datasets. Implemented in `sphereVertex.glsl`.

Additional point cloud and block modes handle sparse data. Animation is driven by GSAP for temporal transitions, and post-processing (shadows, bloom) is provided by react-three-postprocessing.

The data pipeline is normalize-to-uint8 then sample. Raw float values are scaled to [0, 1] and uploaded as 8-bit textures; the fragment shader looks them up against one of 70+ colormaps from `js-colormaps-es` (magma, viridis, turbo, Spectral, ...). Adjustable `cOffset` and `cScale` uniforms allow live remapping of the value range without re-uploading texture data, and a separate NaN colour and alpha control nodata appearance.

## Zarr handling

Reads use zarrita 0.6.x. An LRU cache (`ZarrLoaderLRU.ts`) holds recently accessed chunks. Geographic datasets get latitude and longitude bounds passed as shader uniforms, and the `useCoordBounds()` hook computes UV mapping for the sphere render mode. There is no GeoZarr support, no tile pyramid, no projection beyond geographic lat/lon, and no basemap concept; the Earth, when present, is just a textured sphere.

## Where it fits

Choose Browzarr when the goal is exploration of a multi-dimensional cube as a cube, with volumetric rendering and free 3D camera movement, rather than as a 2D web map. It is also the right starting point for users who want a hosted viewer to point at a Zarr URL with no integration work. It is not a substitute for a basemap-anchored 2D map: if the dataset belongs on top of a vector basemap with picking and panning, deck.gl-raster or zarr-layer are the fits.

## Links

- Source: [EarthyScience/Browzarr](https://github.com/EarthyScience/Browzarr)
- Hosted: [browzarr.io](https://browzarr.io)
- Authors: Jeran Poehls and Lazaro Alonso (MPI-BGC Jena)
- Funding: EU Horizon Europe (AI4PEX) and ESA SeasFire
- Related: [zarrita](https://github.com/manzt/zarrita.js), [icechunk-js](https://github.com/earth-mover/icechunk), [@earthyscience/netcdf4-wasm](https://github.com/EarthyScience/netcdf4-wasm)
41 changes: 39 additions & 2 deletions docs/visualization/carbonplan-maps.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,40 @@
# Carbonplan/maps
# carbonplan/maps

Content coming soon. See [source repo](https://github.com/carbonplan/maps) for now.
A React component library for rendering Zarr-backed raster data on Mapbox or MapLibre maps, built around `regl`. Maintained by CarbonPlan under the MIT license. The original "dynamic client" library that informed CarbonPlan's later [zarr-layer](zarr-layer.md), and still actively maintained.

## At a glance

- **Repo** — [carbonplan/maps](https://github.com/carbonplan/maps)
- **Shape** — React component library, single npm package (`@carbonplan/maps`)
- **Map host** — Mapbox GL JS v1.13.1 bundled in the default export, or bring-your-own Mapbox/MapLibre via the `/core` export with `<MapProvider>`
- **Render API** — `regl` (WebGL abstraction) with hand-written GLSL
- **Zarr versions** — v2 (primary) and v3 (via `version` prop), read with `zarr-js` (not zarrita)
- **Conventions** — Strict `ndpyramid` Web Mercator pyramids; reads `.zmetadata` with `multiscales` in `.zattrs`

## What it does

`@carbonplan/maps` exposes a small set of React components, `<Map>`, `<Raster>`, `<Fill>`, `<Line>`, `<RegionPicker>`, plus hooks like `useMap()`, `useRegl()`, `useControls()`, `useRegion()`, that let an application compose a Zarr-backed map view declaratively. It is the rendering backbone of CarbonPlan's published visualizations, including the CMIP6 downscaling, seaweed farming, and forest carbon explainers, so its design choices reflect a production track record rather than a research prototype.

## How it renders

The vertex shader handles Web Mercator projection math and tile geometry. The fragment shader looks up data values from a per-tile texture, normalizes against the user-supplied `clim` range, and samples a 1D RGB colormap texture (typically supplied via the sibling `@carbonplan/colormaps` package). Custom fragment-shader logic can be injected via the `frag` prop on `<Raster>`, which is how multi-band math (averaging, differencing, conditional masking) is expressed.

Multi-band selectors lift naturally into shader uniforms: passing `selector={{ month: [1, 2, 3] }}` loads three arrays simultaneously and exposes them as `month_1`, `month_2`, `month_3` GLSL uniforms.

## Zarr handling

Reads use `zarr-js`. Data is required to be in a Web Mercator pyramid built with `ndpyramid`, with consistent `pixels_per_tile` across zoom levels, and the library validates this at load time and throws if the projection is anything else. The `version` prop selects between Zarr v2 and v3 store layouts. There is no GeoZarr support, no untiled mode, and no in-browser CRS reprojection; the assumption is that all preprocessing happens upstream.

## Where it fits

`@carbonplan/maps` is documented here for completeness and is appropriate if you are already using it (e.g., extending one of CarbonPlan's existing visualizations) or if you are committed to a CarbonPlan-style workflow that pre-bakes data into `ndpyramid` Web Mercator pyramids.

For new MapLibre/Mapbox integrations, this guide does not recommend `@carbonplan/maps` as the starting point. The strict `ndpyramid` Web Mercator pyramid requirement carries the same trade-offs as any pre-rendering pipeline: a frozen projection, a frozen pyramid structure, and a regen step on every data update. CarbonPlan's own newer project, [zarr-layer](zarr-layer.md), removes those constraints (untiled mode, arbitrary CRS via proj4, vanilla-JS API) at the cost of being less mature. For the deck.gl ecosystem, [deck.gl-raster](deck.gl-raster.md) is the better fit since it shares a rendering stack with COG and parses GeoZarr metadata directly.

## Links

- Source: [carbonplan/maps](https://github.com/carbonplan/maps)
- Companion: [@carbonplan/colormaps](https://github.com/carbonplan/colormaps)
- Pyramid generator: [ndpyramid](https://github.com/carbonplan/ndpyramid)
- Newer sibling: [zarr-layer](zarr-layer.md), CarbonPlan's vanilla-JS MapLibre/Mapbox custom-layer approach
- Production users: CarbonPlan visualizations (CMIP6 downscaling, seaweed farming, forest carbon)
Loading