feat: improve type of Rust

.github/workflows/ci.yml

@@ -1,21 +1,37 @@
-name: Cargo Check
+name: CI
 on:
   push:
     branches:
       - main
+      - develop
   pull_request:
 jobs:
   check:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
       - name: Setup Rust
-        uses: actions-rs/toolchain@v1
+        uses: dtolnay/rust-toolchain@stable
         with:
-          toolchain: stable
+          components: rustfmt, clippy
+          targets: wasm32-unknown-unknown
       - name: Run fmt
         run: cargo fmt -- --check
       - name: Run clippy
-        run: cargo clippy
-      - name: Run check
-        run: cargo check
+        run: cargo clippy --all-targets -- -D warnings
+      - name: Run tests
+        run: cargo test
+      - name: Library alone (no CLI dependencies)
+        run: |
+          cargo clippy --no-default-features --all-targets -- -D warnings
+          cargo test --no-default-features
+      - name: Build wasm
+        run: cargo build --target wasm32-unknown-unknown --no-default-features --features wasm
+  msrv:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Setup Rust (the rust-version in Cargo.toml)
+        uses: dtolnay/rust-toolchain@1.85
+      - name: Build and test on the minimum supported Rust
+        run: cargo test --locked

.github/workflows/publish-crate.yml

@@ -23,10 +23,7 @@ jobs:
           fetch-depth: 0
 
       - name: Setup Rust
-        uses: actions-rs/toolchain@v1
-        with:
-          toolchain: stable
-          override: true
+        uses: dtolnay/rust-toolchain@stable
 
       - name: Install cargo-workspaces
         run: cargo install cargo-workspaces
@@ -38,7 +35,6 @@ jobs:
 
       - name: Bump version
         run: |
-          cd src/rust
           cargo workspaces version ${{ github.event.inputs.version_bump }} --no-git-commit --yes
 
       - name: Commit version bump

.github/workflows/publish-npm.yml

@@ -19,18 +19,15 @@ jobs:
           registry-url: "https://registry.npmjs.org"
 
       - name: Setup Rust
-        uses: actions-rs/toolchain@v1
-        with:
-          toolchain: stable
-          override: true
+        uses: dtolnay/rust-toolchain@stable
 
       - name: Install wasm-pack
         run: |
           curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
 
       - name: Build WASM
         run: |
-          wasm-pack build --target web --no-pack
+          wasm-pack build --target web --scope cityjson -- --no-default-features --features wasm
 
       - name: Publish to NPM
         env:

CHANGELOG.md

@@ -1,5 +1,48 @@
 # Changelog
 
+## [0.5.0] - unreleased
+### Modified (breaking)
+- The CityJSON schema's structure is carried in the types:
+  - geometry boundaries, and semantics, material and texture `values`, are nested exactly as deeply as their geometry type requires; `NestedArray` and the untagged value enums are gone, and each `Geometry` variant names its own value shapes (`PrimitiveValues`, `SolidValues`, `MultiSolidValues`);
+  - one index type per kind of index (`VertexIndex`, `UvIndex`, `MaterialIndex`, `TextureIndex`, `SurfaceIndex`, `TemplateIndex`);
+  - vertices are `[i64; 3]`, `transform` scale/translate `[f64; 3]`, `vertices-templates` `[f64; 3]`, a GeometryInstance's boundaries `[VertexIndex; 1]`, texture vertices `[f64; 2]`, extents `[f64; 6]`;
+  - `Lod` holds one of the schema's twenty LoD values and is required on every primitive; `version` is a `CityJSONVersion` (1.1 or 2.0); `attributes` is a JSON object; `extensions` maps a name to an `Extension { url, version }`; `contactType` and `role` are enums; a texture ring is `TexturedRing::Untextured` or `Textured { texture, uvs }`;
+  - City Object and semantic surface types are enums with an `Extension` case, which must start with `+`.
+- Objects the schema closes (geometries, `transform`, `appearance`, `geometry-templates`, materials, textures) reject members it does not name; every other object keeps them in an `other: serde_json::Map`.
+- `CityObjects` and the material/texture theme maps are `IndexMap`s, and object member order is preserved, so output follows input order and is deterministic.
+- The document model and CityJSONSeq sequencing are split. `CityJSON` holds only the document; `header()` (was `get_metadata`) and `features(order)` slice it into a sequence, `feature(id)` slices one feature, and `Collector` (`new`, `add`, `add_header`, `finish`) replaces `add_cjfeature`/`add_transform_correction`. `SeqReader`/`SeqWriter` read and write a sequence line by line, as `Line::Header`/`Line::Feature`. `get_cjfeature`, `sort_cjfeatures` and the hidden sorting state are gone; `cjseq_to_cj` returns a `Result`.
+- `SortingStrategy` is `AsIs` (input order, now the CLI default), `Random { seed }` (a shuffle reproducible from its seed; `cjseq cat --seed`) or `Lexicographical`; the unimplemented `Morton`/`Hilbert` are gone.
+- A City Object no feature reaches (its parent is missing, or does not list it as a child) is a `NotInAnyFeature` error from `features()` instead of being left out of `cat`'s output.
+- Warnings are a `Warning` type of their own, not `CjseqError` variants.
+- The public enums are `#[non_exhaustive]` (except `Line`).
+- `clap`, `rand` and `getrandom` are optional, behind a default `cli` feature that the binary needs; the library depends only on `serde`, `serde_json` and `indexmap`. `rust-version` is 1.85.
+- Errors are a `CjseqError` with structured variants (`IndexOutOfRange`, `DanglingReference`, `HierarchyMismatch`, `ValueCount`, `MisplacedMember`, located by `InCityObject`, `InTemplate`, `AtLine`); `Result<T>` lives at `cjseq::error::Result`. Malformed input is an error, never a panic.
+- The library is split into per-concern modules; every type is re-exported at the crate root. `CityJSON` and `CityJSONFeature` implement `FromStr` and `Default`.
+- wasm-bindgen bindings are behind an off-by-default `wasm` cargo feature, so library consumers building for wasm32 no longer inherit them.
+### Added
+- `validate()`/`errors()` on `CityJSON` and `CityJSONFeature`: every vertex, template, semantic surface, material, texture and texture-vertex index is in range, value arrays match the boundaries they describe, the parent/child hierarchy agrees with itself, and a feature carries no document-only members. The CLI has a `validate` subcommand.
+- `validate()` also checks the schemas' Extension name patterns: `+` then a capital letter for a City Object type, `+` then word characters for a semantic surface type, and `X.Y` or `X.Y.Z` for an Extension's version (`CjseqError::Pattern`).
+- `warnings()` on `CityJSON` and `CityJSONFeature` reports what the spec advises against but a valid file may still do: a core City Object with an Extension child (§ 8.7 rule 3), which the Noise Extension example published with the spec does. `cjseq validate` prints warnings without failing.
+- `Collector::warnings()`: CityJSONFeature lines may carry root members the schema does not name, and collecting drops them, one `DroppedMember` warning each; `cjseq collect` prints them to stderr.
+- `Display` for `CityJSONVersion` (`1.1`, `2.0`).
+- Features keep every descendant of their top-level City Object, not only its children.
+- `CityObject::new`, `children_roles`, `Geometry::surfaces`, `PartialEq` on the public types.
+- Round-trip, validation and sequencing tests over real CityJSONSeq fixtures; CI runs fmt, clippy with warnings denied, the tests and the wasm build.
+### Fixed
+- `collect` corrupted the texture coordinates of every textured feature after the first.
+- Address `location`s (Building, BuildingPart, BuildingUnit, Bridge, BridgePart) were never renumbered: `cat` and `collect` left them pointing at the wrong vertices. They are now a typed `CityObjectAddress`, renumbered and validated like any geometry.
+- `cat | collect` lost `default-theme-material`/`-texture` when the document had no geometry templates.
+- Collecting several sequences kept only a later header's transform; its geometry templates (and their appearance), extensions and root members are now merged, with GeometryInstances shifted to match, and conflicts reported.
+- `collect` turned an index past the end of a feature's own vertices or palettes into a valid-looking index into another feature's data; it is now an error. A City Object met in more than one feature (a member of two CityObjectGroups travels in both groups' features) is collected once when both copies are identical, and is a `DuplicateId` error instead of a silent overwrite when they are not; equal vertices and texture vertices are merged as features are added, so identical copies compare equal.
+- The header written by `cat` cut the appearance down to what the geometry templates use but left the templates' indices unrenumbered, so they pointed at the wrong materials and textures.
+- Metadata, address, `pointOfContact`, semantics, appearance and feature members the schema permits are kept instead of dropped or rejected; every `referenceSystem` the schema's URL pattern permits is accepted.
+- Coordinates round-trip exactly (`serde_json`'s `float_roundtrip`).
+- An empty document no longer gets a nonsense `translate` from `collect`, and an empty feature no longer has a NaN centroid.
+### Performance
+Against 0.4.0, release builds, median of 3 runs on one machine:
+- a 65 MB, 15,000-feature CityJSONSeq (`tests/data/small.city.jsonl` repeated): `collect` 0.95 s to 0.47 s, `cat` 0.57 s to 0.44 s, `filter --radius` 0.40 s to 0.25 s; `collect` gives the same objects, vertices, transform and real coordinates;
+- 80,000 features with one distinct material each: `collect` 68.3 s to 0.23 s, as materials and textures are merged through a hash map instead of a linear scan.
+
 ## [0.4.0] - 2025-09-12
 ### Modified
 - cjseq is now a library + has one binary. Some the functionalities that were in the `main.rs` are now in the `lib.rs` and the binary basically just adds functions to filter and calls the functions of `lib.rs`. This was done also to easily create a Python package.

Cargo.toml

@@ -1,9 +1,11 @@
 [package]
 name = "cjseq"
-version = "0.4.0"
+version = "0.5.0"
 description = "Create+process+modify+convert CityJSONSeq"
 authors = ["Hugo Ledoux <h.ledoux@tudelft.nl>"]
 edition = "2021"
+#-- the oldest Rust the locked dependencies build on (indexmap 2.14, clap 4.6)
+rust-version = "1.85"
 license = "MIT"
 readme = "README.md"
 homepage = "https://github.com/cityjson/cjseq"
@@ -12,24 +14,40 @@ repository = "https://github.com/cityjson/cjseq"
 [lib]
 crate-type = ["cdylib", "rlib"]
 
-# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+[[bin]]
+name = "cjseq"
+path = "src/main.rs"
+required-features = ["cli"]
 
 [dependencies]
-clap = { version = "4.5.4", features = ["derive"] }
-rand = "0.8.5"
+indexmap = { version = "2", features = ["serde"] }
 serde = { version = "1.0", features = ["derive"] }
-serde_json = "1.0"
-# serde_derive = "1.0"
+serde_json = { version = "1.0", features = ["float_roundtrip", "preserve_order"] }
+
+# CLI dependencies
+clap = { version = "4.5.4", features = ["derive"], optional = true }
+rand = { version = "0.8.5", optional = true }
 
 # WASM dependencies
-wasm-bindgen = { version = "0.2", features = ["serde-serialize"] }
-serde-wasm-bindgen = "0.6"
+wasm-bindgen = { version = "0.2", features = ["serde-serialize"], optional = true }
+serde-wasm-bindgen = { version = "0.6", optional = true }
 
 [dependencies.web-sys]
 version = "0.3"
 features = ["console"]
+optional = true
 
 # WASM-specific dependencies
 [target.'cfg(target_arch = "wasm32")'.dependencies]
-console_error_panic_hook = "0.1"
-getrandom = { version = "0.2", features = ["js"] }
+console_error_panic_hook = { version = "0.1", optional = true }
+getrandom = { version = "0.2", features = ["js"], optional = true }
+
+[features]
+default = ["cli"]
+#-- the `cjseq` binary; the library itself needs none of these
+cli = ["dep:clap", "dep:rand", "dep:getrandom"]
+wasm = ["dep:wasm-bindgen", "dep:serde-wasm-bindgen", "dep:web-sys", "dep:console_error_panic_hook"]
+
+[[test]]
+name = "cli"
+required-features = ["cli"]

README.md

@@ -3,7 +3,9 @@
 
 [![crates.io](https://img.shields.io/crates/v/cjseq.svg)](https://crates.io/crates/cjseq)
 
-`cjseq` is a Rust libray+binary for creating, processing, and modifying [CityJSONSeq](https://cityjson.org/cityjsonseq) files, as well as converting to/from [CityJSON](https://cityjson.org).
+`cjseq` is a Rust library and binary for creating, processing, and modifying [CityJSONSeq](https://cityjson.org/cityjsonseq) files, as well as converting to and from [CityJSON](https://cityjson.org).
+
+Its types follow the CityJSON 2.0 schema: a geometry's boundaries and its semantics, material and texture values are nested exactly as deeply as its type requires, each kind of index has its own type, and strings the schema constrains (LoD, City Object types, versions) are typed. What the types cannot express, `validate()` checks.
 
 ## Installation
 
@@ -13,10 +15,13 @@
 2. Run `cargo install cjseq`
 3. Then a binary called `cjseq` is installed system-wide
 
-### Installing the library
+### Using the library
 
-1. Install the [Rust compiler](https://www.rust-lang.org/learn/get-started)
-2. Run `cargo install cjseq`
+```sh
+cargo add cjseq --no-default-features
+```
+
+The default `cli` feature builds the `cjseq` binary and pulls in `clap` and `rand`; the library itself depends only on `serde`, `serde_json` and `indexmap`.
 
 ### Compiling the project
 
@@ -44,6 +49,8 @@ Alternatively, to use stdin as input:
 cat myfile.city.json | cjseq cat
 ```
 
+By default the features are written in the order of the input; `--order lexicographical` sorts them by id, `--order random` shuffles them (`--seed <n>` reproduces a shuffle).
+
 ### Convert CityJSONSeq to CityJSON
 
 The operator "collect" converts a CityJSONSeq stream to a CityJSON file:
@@ -56,12 +63,14 @@ cat ./data/3dbag_b2.city.jsonl | cjseq collect > 3dbag_b2.city.json
 cjseq collect ./data/3dbag_b2.city.jsonl > 3dbag_b2.city.json
 ```
 
-Notice that [globbing](https://en.wikipedia.org/wiki/Glob_(programming)) works for the `collect` command:
+Several sequences can be collected into one file, with a file list or [globbing](https://en.wikipedia.org/wiki/Glob_(programming)):
 
 ```sh
-cat ./data/*.city.jsonl | cjseq collect > hugefile.city.json
+cjseq collect ./data/*.city.jsonl > hugefile.city.json
 ```
 
+When their transforms differ, the features of each sequence are re-expressed in the first one's transform.
+
 ### Filter CityJSONSeq
 
 An input stream of CityJSONSeq can be filtered with the following operators:
@@ -85,10 +94,53 @@ As an example:
 cat myfile.city.jsonl | cjseq filter --bbox 85007 446179 85168 446290 > mysubset.city.jsonl
 ```
 
+### Validate
+
+`validate` reads a CityJSON file or a CityJSONSeq and prints every reference that does not resolve: indices past the end of `vertices` or of an appearance palette, semantics or material values that do not match their geometry, children and parents that do not name each other back, and so on.
+
+```sh
+cjseq validate myfile.city.jsonl
+```
+
+It also prints warnings, without failing, for what the spec advises against but a valid file may still do: a core City Object with an Extension City Object as a child (§ 8.7 rule 3).
+
+It complements, and does not replace, schema validation with [cjval](https://github.com/cityjson/cjval).
+
 ## Input constraints
 
-  1. the input CityJSON/Seq must be v1.1 or v2.0 (v1.0 will panic).
-  2. the input JSON must be CityJSON schema-valid, use [cjval](https://github.com/cityjson/cjval) to validate.
+  1. the input must be CityJSON v1.1 or v2.0; anything else is reported as an error.
+  2. input the schema forbids (a geometry nested to the wrong depth, a member a closed object does not name, a misspelt type) is reported as an error, with the line it is on for a CityJSONSeq.
+
+## The library
+
+```rust
+use cjseq::{CityJSON, Collector, Line, SeqReader, SeqWriter, SortingStrategy};
+use std::io::BufReader;
+
+fn main() -> cjseq::error::Result<()> {
+    //-- CityJSON -> CityJSONSeq
+    let cj: CityJSON = std::fs::read_to_string("data/3dbag_b2.city.json")?.parse()?;
+    cj.validate()?;
+    let mut out = SeqWriter::new(Vec::new());
+    out.write_header(&cj.header()?)?;
+    for feature in cj.features(SortingStrategy::AsIs) {
+        out.write_feature(&feature?)?;
+    }
+
+    //-- CityJSONSeq -> CityJSON
+    let file = std::fs::File::open("data/3dbag_b2.city.jsonl")?;
+    let mut collector: Option<Collector> = None;
+    for line in SeqReader::new(BufReader::new(file)) {
+        match line? {
+            Line::Header(h) => collector = Some(Collector::new(h)),
+            Line::Feature(f) => collector.as_mut().expect("header first").add(f)?,
+        }
+    }
+    let cj = collector.expect("a header").finish();
+    println!("{} features", cj.number_of_city_objects());
+    Ok(())
+}
+```
 
 ## WASM bindings
 
@@ -98,8 +150,8 @@ cat myfile.city.jsonl | cjseq filter --bbox 85007 446179 85168 446290 > mysubset
 cargo install wasm-pack
 ```
 
-Build the WASM bindings:
+Build the WASM bindings, which are behind the `wasm` cargo feature:
 
 ```sh
-wasm-pack build --release --target web --out-dir js
+wasm-pack build --release --target web --out-dir js -- --no-default-features --features wasm
 ```