Cut provisioning round-trips: merge working+drafts, add PriorHash cac…

CHANGELOG.md

@@ -31,6 +31,11 @@
 * Added first-class remote reboot support.
 * Expanded built-in provisioning namespaces and metrics.
 * Added schema hashing for provisioning schema caching.
+* Added per-namespace schema hashes and repurposed `GET_CAPABILITIES` to return a namespace hierarchy map, letting clients fetch schema lazily per namespace.
+* Added `NamespaceFilter` support to `GET_SCHEMA` so clients can fetch only the namespaces they don't already have cached.
+* Added `Draft` flag to `GET_STATE`: when `true`, the response includes drafts alongside working values in a single round-trip. Response body is now `{Values, Drafts?, Hash}`.
+* Added `PriorHash` cache short-circuit to `GET_STATE`: clients echo a previously-received `Hash`, and the server responds with `{Unchanged: true}` when the state would be byte-identical.
+* Added `IncludeState` flag to `SET_STATE` and `COMMIT`: when `true`, the response includes post-op `PostOpValues` / `PostOpDrafts` / `PostOpHash`, eliminating the follow-up `GET_STATE` round-trip after a save or commit.
 * Added provisioning payload compression support.
 * Embedded Heatshrink compression implementation.
 
@@ -127,6 +132,12 @@
 
 * Persisted path-table data created by previous releases may not be reusable after upgrade and may be rebuilt automatically from network announces.
 * Applications using internal persistence formats should verify compatibility.
+* **Provisioning wire format changed.** Existing v0.4-era clients will not interoperate with a v0.5 server:
+    * `GET_CAPABILITIES` now returns an array of namespace map entries (`{NsId, NsName, NsParent, NsFieldCount, NsSchemaHash}`), replacing the previous bare array of ids.
+    * `GET_STATE` responses are wrapped in `{Values, Drafts?, Hash}` (previously the response body was the `{ns_id: {fid: value}}` map directly). The old `Pending` request flag is replaced by `Draft`, with new "include drafts alongside working" semantics.
+    * `SET_STATE` requests use a new envelope `{State: {...}, IncludeState?, ReqCompress?}` (previously the top-level map was the state map itself).
+    * `COMMIT` requests use a map envelope `{NamespaceFilter?, IncludeState?, ReqCompress?}` instead of a bare array of ns ids.
+    * The `Pending` request key and `Applied`-slot-3 collision in the old commit response are gone. See [`docs/provisioning_client_guide.md`](docs/provisioning_client_guide.md) for the current wire format.
 * Existing applications using function-pointer callback handlers do not require immediate changes but should migrate toward std::function callbacks.
 
 ## Contributors

README.md

@@ -117,7 +117,7 @@ RNS::Bytes response = RNS::Provisioning::Provisioner::instance().handle_message(
 send_to_transport(response);
 ```
 
-Supported operations are `GET_SCHEMA`, `GET_INFO`, `GET_CAPABILITIES`, `GET_STATE`, `SET_STATE`, `COMMIT`, `DISCARD`, and `FACTORY_RESET`. The envelope is a 3-element MsgPack array `[op_id, seq, payload]`; see `src/microReticulum/Provisioning/Ops.h` for op-id and key constants.
+Supported operations are `GET_SCHEMA`, `GET_INFO`, `GET_CAPABILITIES`, `GET_STATE`, `SET_STATE`, `COMMIT`, `DISCARD`, `FACTORY_RESET`, and `REBOOT`. The envelope is a 3-element MsgPack array `[op_id, seq, payload]`; see `src/microReticulum/Provisioning/Ops.h` for op-id and key constants, and [`docs/provisioning_client_guide.md`](docs/provisioning_client_guide.md) for the full wire format (per-op request/response shapes, `Draft`/`IncludeState`/`PriorHash` round-trip optimizations, and heatshrink-compressed responses).
 
 ### Manual operations (no wire)
 

docs/provisioning_integration_guide.md

@@ -355,8 +355,8 @@ Mechanics:
 - Each `.register_namespace()` call pushes onto a per-`Provisioner` scope stack and uses the current scope top as the new namespace's parent.
 - Each `.end()` pops one level. Forgetting `.end()` is caught at `Provisioner::begin()` — the scope stack must be empty by the time `begin()` runs (or it gets cleared with a warning).
 - The Registry remains **flat** internally — every namespace is a top-level entry with an optional `parent_id`. Lookups stay O(1) by id.
-- On disk: `Interfaces.msgpack`, `Interfaces.LoRa.msgpack`, `Interfaces.UDP.msgpack`. Three files at the top level of the storage directory.
-- On the wire: `GET_STATE` keeps each namespace at the top level keyed by its own id. The tree shape is conveyed only through the schema's `parent_id`. Clients reconstruct the tree client-side.
+- On disk: `ns<id>.msgpack` — one file per namespace, keyed by numeric id. The tree shape is not reflected in filenames.
+- On the wire: `GET_STATE` returns values in a flat `{ns_id: {field_id: value, ...}, ...}` map (nested inside the response body's `Values` key). The tree shape is conveyed only through the schema's `parent_id`. Clients reconstruct the tree client-side.
 
 Cycle detection runs at registration time — declaring a parent that walks up to the new namespace itself is refused.
 
@@ -724,7 +724,8 @@ public:
     Registry& registry();
     Storage* storage();
 
-    static constexpr uint16_t SCHEMA_VERSION = 1;
+    static constexpr uint16_t SCHEMA_VERSION = 2;
+    uint32_t schema_hash() const;
 };
 ```
 
@@ -766,7 +767,7 @@ using CommitCallback = std::function<void(Namespace&)>;
 ### `Value` (tagged variant)
 
 ```cpp
-enum class Type : uint8_t { None=0, Bool=1, Int=2, Float=3, String=4, Bytes=5, Enum=6, BytesList=7 };
+enum class Type : uint8_t { None=0, Bool=1, Int=2, Float=3, String=4, Bytes=5, Enum=6, BytesList=7, Void=8 };
 
 class Value {
 public:

src/microReticulum/Provisioning/Namespace.h

@@ -71,6 +71,9 @@ namespace RNS { namespace Provisioning {
 		bool draft(fid_t field_id, Value& out) const;
 		bool has_draft(fid_t field_id) const;
 
+		// True iff at least one draft entry exists in this namespace.
+		bool has_any_draft() const { return !_draft.empty(); }
+
 		// Write to draft after validating against the field's constraint.
 		// Returns false if the field is unknown, read-only, or invalid.
 		bool set_draft(fid_t field_id, const Value& v);
@@ -110,6 +113,13 @@ namespace RNS { namespace Provisioning {
 		bool has_on_commit() const { return (bool)_on_commit; }
 		const CommitCallback& on_commit_callback() const { return _on_commit; }
 
+		// CRC32 of this namespace's schema entry as it would appear in the
+		// GetSchema payload. Computed by Provisioner::begin() after all fields
+		// are registered and surfaced in GetCapabilities so clients can cache
+		// each namespace's schema independently.
+		uint32_t schema_hash() const { return _schema_hash; }
+		void schema_hash(uint32_t h) { _schema_hash = h; }
+
 	private:
 		nid_t _id;
 		fstring_t _name;
@@ -120,6 +130,7 @@ namespace RNS { namespace Provisioning {
 		std::unordered_map<fid_t, Value> _working;
 		std::unordered_map<fid_t, Value> _draft;
 		CommitCallback _on_commit;
+		uint32_t _schema_hash = 0;
 		bool _dirty_for_persist = false;
 	};
 

src/microReticulum/Provisioning/Ops.h

@@ -69,11 +69,27 @@ namespace RNS { namespace Provisioning {
 		constexpr uint16_t NeedsReboot    = 2;
 		constexpr uint16_t DraftHasReboot = 2;	// alias for SetState
 		constexpr uint16_t FieldErrors    = 3;
+		// Post-op state returned in Commit and SetState responses when
+		// IncludeState was requested. Slots 4-6 avoid collision with
+		// Applied(1) / NeedsReboot(2) / FieldErrors(3) in the enclosing
+		// response body. Commit only emits Values+Hash (drafts are cleared
+		// by commit); SetState emits Values+Drafts+Hash (drafts persist).
+		constexpr uint16_t PostOpValues   = 4;
+		constexpr uint16_t PostOpDrafts   = 5;
+		constexpr uint16_t PostOpHash     = 6;
 
-		// GetState / SetState request flags
+		// GetState / SetState / Commit request flags
 		constexpr uint16_t NamespaceFilter = 1;
-		constexpr uint16_t Pending         = 2;
+		constexpr uint16_t Draft           = 2;	// GetState request: true = also include drafts alongside working
 		constexpr uint16_t State           = 3;	// the {ns: {field: value}} map
+		constexpr uint16_t PriorHash       = 4;	// GetState request: client-supplied CRC32 for cache short-circuit
+		constexpr uint16_t IncludeState    = 5;	// Commit request: return post-commit Values in the response
+
+		// GetState response body
+		constexpr uint16_t Values          = 1;	// {ns_id: {field_id: value}} map (working values)
+		constexpr uint16_t Drafts          = 2;	// sparse {ns_id: {field_id: value}} — only ns/fields with drafts
+		constexpr uint16_t Hash            = 3;	// CRC32 over the response body; echoed back as PriorHash
+		constexpr uint16_t Unchanged       = 4;	// present iff the client's PriorHash matched
 
 		// GetInfo
 		constexpr uint16_t FirmwareVersion = 1;
@@ -90,6 +106,8 @@ namespace RNS { namespace Provisioning {
 		constexpr uint16_t NsName           = 2;
 		constexpr uint16_t NsFields         = 3;
 		constexpr uint16_t NsParent         = 4;	// optional parent ns id (0 = root)
+		constexpr uint16_t NsFieldCount     = 5;	// GetCapabilities map: number of fields
+		constexpr uint16_t NsSchemaHash     = 6;	// GetCapabilities map: per-namespace CRC32
 		constexpr uint16_t FieldId          = 1;
 		constexpr uint16_t FieldName        = 2;
 		constexpr uint16_t FieldType        = 3;