Skip to content

Commit c0bea82

Browse files
gajopclaude
andcommitted
Slice 1: port terrain brushes to Rust + command-system infra
Port the four terrain brush commands (shape / level / smooth / metal) plus brush-settings to Rust and flip them to Rust-only via nativeCommandsOnly. Command system: - Heightmap brushes wrap their per-point writes in TerrainControl::set_height_map_func so the engine recalcs the touched area (face normals + LOS/pathing); raw add_height_map changes data but never recalcs. Metal needs no recalc. - Undo/redo for Rust-only commands forward UndoCommand/RedoCommand to the native manager (Lua wasn't driving Rust's undo stack). Those control commands are now braced structs so they deserialize from the {className} payload. - Native dispatch gated to the gadget (synced) state: every command reaches the gadget first, so the single shared native manager sees it once. Prevents cross-executed commands (e.g. SetMultipleCommandModeCommand) hitting it twice and desyncing the streaming/undo state. Infrastructure: - IO worker shell (boxed dyn IoJob / dyn IoOutcome seam, no job types yet) for off-engine-thread file/image work. - In-engine integration test harness (SBC_TEST_SPEC-gated), driven by the native Update callin (replaces the old Lua widget:Update poll_io driver). - Terrain brush tests assert the recalc signal (get_ground_normal), not just get_ground_height; terrain_drag_stroke covers streaming-mode grouping. Docs: command-system + async-io design docs, porting conventions (stepdown, comments), slice/review-queue status, and a todo.md for deferred work. Known issue: set_height_map_func recalc doesn't fire on undo (stale normals) — engine-side, tracked separately; terrain_shape_brush is red until it lands. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 8dcf68b commit c0bea82

36 files changed

Lines changed: 2099 additions & 96 deletions

docs/design/async-io.md

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
---
2+
name: Async IO
3+
description: How SpringBoard runs file IO and image work off the engine thread
4+
---
5+
6+
# Async IO
7+
8+
Some commands do **file IO** and **image decode/encode** — heightmap import /
9+
export, texture export, map compile. That work must not run on the engine thread
10+
(it would stall the frame), and the file/image work must not touch the engine
11+
(it runs on another thread, where engine calls are invalid).
12+
13+
One rule keeps these apart:
14+
15+
> **The background thread never touches the engine. The engine thread never does file IO.**
16+
17+
## The worker
18+
19+
A single background worker thread, owned by an `IoWorker` created once at plugin
20+
init and living for the whole plugin lifetime. Two channels connect it to the
21+
engine thread: jobs go out, outcomes come back.
22+
23+
The worker is **feature-agnostic**. It moves a `Box<dyn IoJob>` to the thread and
24+
a `Box<dyn IoOutcome>` back, knowing nothing about what either does. Each feature
25+
slice defines its own job and outcome types in its own module, so adding one
26+
never edits the worker.
27+
28+
Two trait bounds enforce the rule at compile time:
29+
30+
- `IoJob: Send` and owns its data — it has **no engine handle**, so the worker
31+
physically cannot reach the engine.
32+
- `IoOutcome::apply(&mut SBC)` takes the engine handle, and only runs back on the
33+
engine thread, where engine calls are valid.
34+
35+
## The three phases
36+
37+
Take heightmap export as the worked example: read every height, encode a PNG,
38+
write it to disk.
39+
40+
```mermaid
41+
sequenceDiagram
42+
participant Cmd as Command (engine thread)
43+
participant Worker as IoWorker (bg thread)
44+
participant Poll as poll_io (engine thread)
45+
46+
Note over Cmd: 1. submit
47+
Cmd->>Cmd: read heights into an owned buffer<br/>(fast memory read, no IO)
48+
Cmd->>Worker: submit(Box<dyn IoJob>)
49+
50+
Note over Worker: 2. run (no engine access)
51+
Worker->>Worker: encode PNG + write file<br/>on owned buffers only
52+
Worker-->>Poll: Box<dyn IoOutcome> (queued)
53+
54+
Note over Poll: 3. apply
55+
Poll->>Poll: drain ready outcomes
56+
Poll->>Poll: outcome.apply(&mut SBC)<br/>(engine calls valid here)
57+
```
58+
59+
1. **Submit (engine thread, on command dispatch).** Read whatever engine data
60+
the job needs into an owned buffer — a fast memory read, not IO. Build an
61+
`IoJob` (owns only `Send` data, holds no engine handle) and `submit` it.
62+
2. **Run (worker thread).** `IoJob::run` does the file read/write and image
63+
decode/encode on owned buffers only, returning a `Box<dyn IoOutcome>`. No
64+
engine access — the missing handle makes that a compile-time guarantee. A
65+
failed job still returns an outcome that applies as a logging no-op.
66+
3. **Apply (engine thread, on drain).** `IoOutcome::apply(&mut SBC)` applies the
67+
effect (e.g. `set_height_map` for import) where engine calls are valid.
68+
69+
Import is the mirror image: the job reads + decodes a file, the outcome writes
70+
the decoded heights into the engine.
71+
72+
## The drain point
73+
74+
`SBC::update` (the native `Update` callin — unsynced, once per render frame)
75+
drains any ready outcomes and applies them. Unsynced rather than `game_frame`
76+
because IO shouldn't be tied to the sim clock: `update` fires at a steady cadence
77+
regardless of pause or game speed.
78+
79+
## Status
80+
81+
The worker shell, the `IoJob` / `IoOutcome` seam, the `poll_io` handler, and the
82+
`widget:Update` driver all landed with slice 1 as common infrastructure, with no
83+
job types wired. The first concrete job/outcome types land with the heightmap
84+
slice (import / export); adding them does not touch the shell.
85+
86+
See also: [Command system](command-system.md) — async IO is kicked off by command
87+
dispatch and applied on the same boundary the command system runs on.

docs/porting/01-slices.md

Lines changed: 36 additions & 57 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ spread across the slices they belong to, not deferred as a catch-all.
3030
| # | Slice | Status |
3131
|--:|-------|--------|
3232
| 0 | [Command infrastructure](#0-command-infrastructure) — trait + dispatch + undo/redo + compound + bridge + widget-notify | done (stable) |
33-
| 1 | [Terrain](#1-terrain) — shape / level / smooth / metal brushes | done (wip; flipped to Rust-only) |
33+
| 1 | [Terrain](#1-terrain) — shape / level / smooth / metal brushes | review (in stable; flipped to Rust-only; heightmap recalc fixed via `set_height_map_func`) |
3434
| 2 | [Heightmap](#2-heightmap) — load (sync) + import / export (async IO) | todo |
3535
| 3 | [Map settings](#3-map-settings) — sun / atmosphere / water / map-rendering | todo (setters bound; undo needs gl getters) |
3636
| 4 | [Textures](#4-textures) — diffuse / shading / terrain texture / cache + grass + DNTS | blocked (texture-atlas GL) |
@@ -67,9 +67,9 @@ This slice delivered it.
6767
control commands). Transport (`{tag, data}` decode) lives at the `SBC` boundary
6868
in [sbc.rs](../../native/src/sbc/sbc.rs).
6969

70-
Convention: each slice is a directory; `mod.rs` files only wire submodules, never
71-
hold code. Feature-slice managers (e.g. terrain's `TerrainManager`) construct
72-
through `sbc.rs`.
70+
Tree-layout conventions (slice = directory, thin `mod.rs`, etc.) are in
71+
[conventions.md](conventions.md#code-structure). Feature-slice managers (e.g.
72+
terrain's `TerrainManager`) construct through `sbc.rs`.
7373

7474
**Lua files**:
7575
- [scen_edit/command/command.lua](../../scen_edit/command/command.lua) — base class
@@ -78,12 +78,18 @@ through `sbc.rs`.
7878

7979
The bridge sends every command to Rust via `Spring.InvokeNativeModule(json.encode(msg:serialize()))`. Lua execution **also** runs (parallel) unless the class name appears in `nativeCommandsOnly`. Slice 0 leaves that allowlist empty — feature slices populate it as they land.
8080

81-
**Deferred to slice 1 (first feature slice that flips a command to Rust-only):**
81+
**Still open — required for full 1:1 once Rust is the *only* executor of undo/redo:**
8282

83-
These exist on the Lua side and are required for full 1:1 parity once Rust is the only executor. Until then Lua handles them; slice 0 leaves them out by design, not by omission.
83+
These exist on the Lua side and are needed once Rust owns the undo stack. They
84+
are **not** yet ported. Slice 1 did not need them: flipping a command via
85+
`nativeCommandsOnly` only suppresses Lua's `cmd:execute()`
86+
([command_manager.lua](../../scen_edit/command/command_manager.lua) line ~129) —
87+
the `undoListAdd` / `notify` path (line ~133) is **not** gated, so during the
88+
parallel period Lua still owns undo/redo bookkeeping and the widget notify. These
89+
land with the slice that first moves the undo stack itself to Rust.
8490

8591
- `__cmd_id` allocation on every executed command (Rust counterpart to Lua's `idCount`).
86-
- Widget notify after `execute` / `undo` / `redo` / `clear_*` / `undo_list_add` (when it pops oldest) — Lua dispatches `WidgetCommandExecuted` / `WidgetCommandUndo` / `WidgetCommandRedo` / `WidgetCommandClearUndoStack` / `WidgetCommandClearRedoStack` / `WidgetCommandRemoveFirstUndo` to the widget. Rust needs the equivalent via `send_lua_uimsg`, matching `scen_edit/message/message_manager.lua`'s prefix-framed wire format.
92+
- Widget notify after `execute` / `undo` / `redo` / `clear_*` / `undo_list_add` (when it pops oldest) — Lua dispatches `WidgetCommandExecuted` / `WidgetCommandUndo` / `WidgetCommandRedo` / `WidgetCommandClearUndoStack` / `WidgetCommandClearRedoStack` / `WidgetCommandRemoveFirstUndo` to the widget. Rust needs the equivalent via `send_lua_uimsg` (a `lua_bridge` module — exists in wip, not yet in stable), matching `scen_edit/message/message_manager.lua`'s prefix-framed wire format.
8793
- `display()` method on commands (Lua returns `self.className`; `CompoundCommand` returns the first sub-command's display). Required for the `display` field of `WidgetCommandExecuted`.
8894

8995
**Out of scope for slice 0** (handled by their feature slices):
@@ -96,20 +102,26 @@ These exist on the Lua side and are required for full 1:1 parity once Rust is th
96102

97103
Brush-based heightmap and metal-map editing. Engine-side via `Spring.SetHeightMap` / `Spring.SetMetalAmount` / `Spring.AddHeightMap`, all bound natively.
98104

99-
**Status:** done in wip — 4 brush commands + brush-settings, flipped to Rust-only via `nativeCommandsOnly`. Needs in-game verification (the slice-0 widget-notify path is exercised here).
105+
**Status:** in stable, awaiting review — four brush commands + brush-settings,
106+
flipped to Rust-only. Also lands shared infra: the IO-worker shell (no job types
107+
yet), the in-engine test harness, and the Lua `poll_io` driver.
108+
109+
The three heightmap brushes wrap their writes in `TerrainControl::set_height_map_func`
110+
so the engine recalcs (without it, heights change in data but the terrain doesn't
111+
move) — see `SBC_PORT_MISSING_BINDINGS.md`. Metal needs no recalc.
100112

101113
**Model:**
102114
- [scen_edit/model/terrain_manager.lua](../../scen_edit/model/terrain_manager.lua) — brush state, listeners, generated metadata
103115
- [scen_edit/model/heightmap.lua](../../scen_edit/model/heightmap.lua)
104116
- [scen_edit/model/rendering/texture_undo_stack.lua](../../scen_edit/model/rendering/texture_undo_stack.lua) (also used by textures)
105117

106-
**Commands:**
107-
- [abstract_terrain_modify_command.lua](../../scen_edit/command/abstract_terrain_modify_command.lua) — shared base
108-
- [terrain_shape_modify_command.lua](../../scen_edit/command/terrain_shape_modify_command.lua)
109-
- [terrain_level_command.lua](../../scen_edit/command/terrain_level_command.lua)
110-
- [terrain_smooth_command.lua](../../scen_edit/command/terrain_smooth_command.lua)
111-
- [terrain_metal_command.lua](../../scen_edit/command/terrain_metal_command.lua)
112-
- [set_heightmap_brush_command.lua](../../scen_edit/command/set_heightmap_brush_command.lua)
118+
**Commands (Lua source → Rust):**
119+
- [abstract_terrain_modify_command.lua](../../scen_edit/command/abstract_terrain_modify_command.lua) — shared base; its brush-stamp logic became [brush_modify.rs](../../native/src/sbc/commands/heightmap/brush_modify.rs) (composition via closures, not inheritance) + [brush_filter_generator.rs](../../native/src/sbc/commands/heightmap/brush_filter_generator.rs)
120+
- [terrain_shape_modify_command.lua](../../scen_edit/command/terrain_shape_modify_command.lua)[terrain_shape_modify_command.rs](../../native/src/sbc/commands/heightmap/terrain_shape_modify_command.rs)
121+
- [terrain_level_command.lua](../../scen_edit/command/terrain_level_command.lua)[terrain_level_command.rs](../../native/src/sbc/commands/heightmap/terrain_level_command.rs)
122+
- [terrain_smooth_command.lua](../../scen_edit/command/terrain_smooth_command.lua)[terrain_smooth_command.rs](../../native/src/sbc/commands/heightmap/terrain_smooth_command.rs)
123+
- [terrain_metal_command.lua](../../scen_edit/command/terrain_metal_command.lua)[terrain_metal_command.rs](../../native/src/sbc/commands/heightmap/terrain_metal_command.rs)
124+
- [set_heightmap_brush_command.lua](../../scen_edit/command/set_heightmap_brush_command.lua)[set_heightmap_brush_command.rs](../../native/src/sbc/commands/set_heightmap_brush_command.rs) (registers the greyscale brush shape in [terrain_manager.rs](../../native/src/sbc/commands/heightmap/terrain_manager.rs); **not** flipped to Rust-only — Lua still needs the shape for its own preview)
113125

114126
---
115127

@@ -126,8 +138,10 @@ crate), replacing what the spring-launcher used to do over IPC.
126138
thread. Port directly.
127139
- **Import / export (async IO):** decode/encode an image file. File IO + image
128140
work runs on a background worker thread (must not touch the engine); the engine
129-
thread applies/reads heights. Needs the IO-worker + `widget:Update` poll (see
130-
"Async IO" below).
141+
thread applies/reads heights. The IO-worker shell + the `widget:Update` poll
142+
already exist (landed with slice 1 as common infra) — this slice just adds the
143+
first concrete `IoJob`/`IoOutcome` types (and the `image` crate dep). See
144+
[docs/design/async-io.md](../design/async-io.md).
131145

132146
**Model:**
133147
- [scen_edit/model/heightmap.lua](../../scen_edit/model/heightmap.lua) (shared with slice 1)
@@ -323,44 +337,9 @@ pure project logic (no engine API).
323337

324338
---
325339

326-
## Widget commands — not a slice
327-
328-
Widget commands aren't deferred as a catch-all. Each is part of the feature slice
329-
it serves and is listed there:
330-
331-
- `widget_terrain_change_texture_command` → slice 4 (textures)
332-
- `widget_follow_unit_command` → slice 5 (objects); genuinely widget-only, stays Lua
333-
- `widget_command_executed` / undo / redo / clear / remove-first → slice 0
334-
(the command-system widget-notify path; already handled)
335-
336-
The remaining genuinely widget-only ones (display text, unit say, draw texture,
337-
execute-unsynced-action) are unsynced UI helpers; they stay Lua until the view
338-
layer moves to Rust (Phase 2/3). Listed for completeness:
339-
- [widget_display_text.lua](../../scen_edit/command/widget_display_text.lua)
340-
- [widget_unit_say_command.lua](../../scen_edit/command/widget_unit_say_command.lua)
341-
- [widget_draw_texture_command.lua](../../scen_edit/command/widget_draw_texture_command.lua)
342-
- [widget_execute_unsynced_action_command.lua](../../scen_edit/command/widget_execute_unsynced_action_command.lua)
343-
344-
## Async IO (heightmap import/export, compile, image export)
345-
346-
Import/export/compile do file IO + image decode/encode. Rule: **the background
347-
thread never touches the engine; the engine thread never does file IO.**
348-
349-
- Engine thread (on command dispatch): read engine data into an owned buffer
350-
(export) — fast memory read, no IO. Hand the buffer to the worker.
351-
- Background worker thread: file read/write + image decode/encode on owned
352-
buffers only. No engine access.
353-
- Engine thread (on drain): apply results (e.g. `set_height_map` for import).
354-
355-
Drain point: there's no native per-frame callin (see the engine note, Category 0),
356-
so a throttled `widget:Update` pokes the plugin via
357-
`InvokeNativeModule(json.encode({tag="poll_io"}))` and the plugin drains completed
358-
jobs on that call. `widget:Update` (not `gadget:GameFrame`) because GameFrame
359-
doesn't fire while paused and SBC is effectively always paused.
360-
361-
## Where things go in Rust
340+
The async-IO model (heightmap import/export, compile, image export) is a design
341+
concern, not a slice — see [docs/design/async-io.md](../design/async-io.md). The
342+
worker shell landed with slice 1; the first job types land with slice 2.
362343

363-
- Per-command Rust file: [native/src/sbc/commands/](../../native/src/sbc/commands/)`<slice>/`. Each file owns its serde struct, the `inventory::submit!` registration, and the execute/unexecute logic.
364-
- Per-manager Rust file: a per-slice `model`/manager module.
365-
- The command-system internals + the single public `commands_api` surface live under [native/src/sbc/commands/command_system/](../../native/src/sbc/commands/command_system/).
366-
- Cross-slice managers (texture undo stack, heightmap) live where the first slice that needs them puts them; later slices reuse.
344+
Where ported code lives in the tree is a porting convention — see
345+
[conventions.md](conventions.md#code-structure).

docs/porting/README.md

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,15 @@ Phase 1 ports the model and command layers together in slices (one feature end-t
2626

2727
Phase 2 is in progress independently of the Rust port (some Chili→RmlUi work landed earlier).
2828

29-
Pending review items: [review-queue.md](review-queue.md). Rules: [conventions.md](conventions.md).
29+
Pending review items: [review-queue.md](review-queue.md). Rules: [conventions.md](conventions.md). Deferred improvements: [todo.md](todo.md).
30+
31+
## Design docs
32+
33+
Living documents describing how subsystems work (outlast the slice docs, which
34+
are purged once a slice lands):
35+
36+
- [Command system](../design/command-system.md) — how editor actions become commands and execute
37+
- [Async IO](../design/async-io.md) — running file IO / image work off the engine thread
3038

3139
## Where things live
3240

docs/porting/conventions.md

Lines changed: 39 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,14 @@ The registry is built at startup from `inventory::iter`. Adding a command = one
102102

103103
If `inventory` becomes problematic (e.g. platforms where life-before-`main` link sections misbehave), the fallback is a `build.rs` that scans `commands/*.rs` and generates the registry. Same property: command files stay self-contained.
104104

105+
**Where ported code lives in the tree:**
106+
107+
- Per-command file under [native/src/sbc/commands/](../../native/src/sbc/commands/)`<slice>/` — owns its serde struct, its `inventory::submit!` registration, and its execute/unexecute logic.
108+
- Per-manager file: a per-slice `model`/manager module.
109+
- The command-system internals + the single public `commands_api` surface live under [native/src/sbc/commands/command_system/](../../native/src/sbc/commands/command_system/).
110+
- Cross-slice managers (texture undo stack, heightmap) live where the first slice that needs them puts them; later slices reuse.
111+
- Each slice is a directory; `mod.rs` files only wire submodules, never hold code.
112+
105113
| Layer | Self-contained unit | Registration |
106114
|-------|--------------------|--------------|
107115
| Commands | one file per command | `inventory` keyed by class name |
@@ -142,10 +150,40 @@ If any check fails, fix it. **Don't pad the queue.** Three clean items beat ten
142150
- **Imports grouped**: std → external crates → crate-local, separated by blank lines.
143151
- **Errors carry context.** Never bare `?` that drops meaning.
144152
- **Tests live next to code**: `#[cfg(test)] mod tests` at the bottom of the file.
145-
- **No WHAT-comments.** Only WHY-comments survive review.
146153
- **Public surface minimal.** `pub(crate)` over `pub` when in doubt.
147154
- **Short functions.** If you scroll to read it, split it.
148155

156+
### Stepdown ordering
157+
158+
Write each file **top-down in call order**: the public entry point first, then
159+
the things it calls, then their callees, with private leaf helpers last. A reader
160+
scrolling top-to-bottom meets each function *before* the helpers it depends on —
161+
the file reads like a newspaper (headline first, detail below).
162+
163+
- In a command file: the `impl Command` block (`execute` / `unexecute` — what the
164+
framework calls) goes first, then its `stamp`/private methods, then leaf helpers
165+
(`brush`, `generate_*`), then free functions, then the `register_command!` line.
166+
- Keep one `impl` block per type — don't split a type's methods across several
167+
`impl` blocks to satisfy ordering; order the methods *within* the block instead.
168+
- Free helper functions go below the code that calls them; a leaf used by several
169+
callers goes after the last of them.
170+
171+
### Comments
172+
173+
Comment the **why**, not the **what**. The code already says what it does; a
174+
comment earns its place only by explaining something the code can't: a non-obvious
175+
reason, a constraint, a surprising interaction, a deliberate deviation.
176+
177+
- **No WHAT-comments.** `// loop over the points` above a `for` loop is noise.
178+
- **Write WHY-comments** for: why this approach over the obvious one, why an engine
179+
call is wrapped a certain way, why a value is what it is, what invariant must
180+
hold. (Example: the `set_height_map_func` wrapper comment explaining the recalc.)
181+
- **Doc-comments (`///`, `//!`)** state a contract — what a caller must know to use
182+
the item correctly — not a restatement of the body.
183+
- **Keep them true.** A stale comment is worse than none; update or delete it when
184+
the code moves. Design narrative belongs in `docs/design/`, not in long module
185+
headers that drift.
186+
149187
## Tests
150188

151189
- **Unit tests** (every item with non-trivial logic): co-located, cover happy path + error paths + edge cases (empty, boundary, large). Skip trivial getters.

0 commit comments

Comments
 (0)