Partial-edit tools for all memory surfaces (+ concurrency safety)

[rockfordlhotka]

RIght now memory files are entirely recreated any time they are changed. This is potentially risky, as a small change might accidentally cause changes elsewhere in the file.

It would be safer (and possibly more efficient) to allow edits of sections of a file, like a sentence or paragraph being replaced - maybe even just a word or phrase.

This new memory tool should apply to all types of memory.

---

Status

Partially delivered by #504, which covered only the shared-volume file surface:

• TextEdit (src/RockBot.Host/TextEdit.cs) — the exact-match replacement primitive, deliberately factored out of the file plumbing so the remaining surfaces can reuse it unchanged. Pure function: string in, string out. Ambiguity is an error rather than a guess; CRLF/LF mismatch is handled.
• file_edit tool — scoped to the shared PVC via FileSystemOptions.BasePath (/rockbot/shared). Memory, working memory, skills, and profile markdown live under AgentProfile__BasePath (/data/agent) and are not reachable by it.

Everything below is the remaining work.

Terminology correction

"Memory file" is a misleading frame for long-term memory. LTM entries are not free-form markdown — each is a JSON-serialized MemoryEntry written one-per-file at {/data/agent}/memory/{category}/{id}.json. The editable payload is the Content string field, not the file. So the tool shape is not path + old_string + new_string; it is id + old_string + new_string, applied as read-entry → TextEdit.Apply → entry with { … } → SaveAsync.

The same is true of skills: the markdown body is a JSON string field inside {name}.json, not a .md file. Nothing file_edit-flavored can be pointed at either one.

The problem is worse than "risky rewrite"

Long-term memory has no content-edit tool at all. The registered surface is save_memory, search_memory, delete_memory, update_memory_importance (MemoryTools.cs:90-96).

To correct a single word today, the agent must delete and re-save. That:

• mints a new Id
• resets CreatedAt and LastSeenAt to now
• resets ReinforcementCount to 1
• runs the replacement text through the LLM extraction pass (ExpandToMemoryEntriesAsync), which rewrites the content anyway

So the current workaround does not merely risk collateral edits — it destroys the entry's provenance and reinforcement history. Same class of loss as the consolidation problem.

Surfaces in scope

Surface	Tool	Store	Current behavior
Long-term / semantic memory	(none — delete + re-save)	FileMemoryStore.SaveAsync:53-88	whole-record rewrite
Working memory	save_to_working_memory	FileWorkingMemory.PersistGroupAsync:197-231	rewrites the whole group file
Skills	save_skill	FileSkillStore	full-body replace; summary cleared and LLM-regenerated
Scheduled-task directive	update_task_directive	FileScheduledTaskStore	"Replaces the existing directive entirely" — its own param description
Behavioral rules	add_rule / remove_rule	FileRulesStore.PersistAsync:98-107	regenerates all of rules.md

Notes per surface:

Working memory — WorkingMemoryEntry is (Key, Value, StoredAt, ExpiresAt, Category, Tags) with no UpdatedAt. Two consequences: there is no field to record an edit, and PersistGroupAsync rewrites the entire group file (session.json, patrol.json, …) containing every live entry in that namespace. A per-entry partial edit still rewrites the group unless storage is restructured — the win here is model-side only.

Open question: should editing a working-memory value extend its TTL? Suggested answer is no — a correction should not silently buy another lifetime — but this needs to be an explicit decision rather than an accident of implementation.

Skills — SkillBodyApplier.cs already implements section-level append / replaceSection / deleteSection ops on Skill.Content, sets UpdatedAt, and uses UpdatedAt as an optimistic-concurrency check before reverting (:49, :62, :91-97). It is a repair-ticket applier, not an LLM tool. Either promote it or mirror its concurrency discipline.

Rules — FileRulesStore.Load() strips leading bullets and drops # lines, so the file is not round-trip-safe: any hand-authored structure in rules.md is destroyed by the next add_rule. Partial editing helps, but the lossy load is a separate defect worth fixing alongside it.

Timestamp contract

MemoryEntry already defines the required semantics; the edit tool only has to honor them. A content edit is a record edit, not reinforcement:

Field	On a partial edit
UpdatedAt	set to now — "bumped on any edit, including dream rephrasing"
CreatedAt	unchanged — first-seen
LastSeenAt	unchanged — "advances only on real reinforcement … not on dream rephrasing, importance decay, or other record edits"
ReinforcementCount	unchanged — a correction is not a new observation
ImportanceScore, Metadata, SupersededBy, ArchivedAt	preserved verbatim

Nothing sets these for you: FileMemoryStore.SaveAsync serializes the record as given. UpdateMemoryImportance (MemoryTools.cs:391-395) is already exactly the existing with { …, UpdatedAt = UtcNow } pattern to copy.

Known coupling: UpdatedAt is overloaded. The dream importance-decay pass uses it as its "time since decay was last applied" anchor (DreamService.cs:1787-1808). Bumping it on a content edit makes the next pass compute eligibleElapsed ≈ 0, so the entry skips one decay cycle. It recovers on the following cycle — a delay, not a freeze — but worth knowing that bumping the timestamp is not purely bookkeeping.

Concurrency — the load-bearing risk

This is the part most likely to bite, and it is not solved by adding a tool.

FileMemoryStore serializes writes on a process-wide SemaphoreSlim but has no compare-and-swap, and SaveAsync is not atomic (plain File.WriteAllTextAsync, no temp+rename). A read-modify-write edit tool therefore races the dream service's background merges, decay pass, and contradiction sweeps — all of which call SaveAsync on the same entries.

update_memory_importance already has this race, but its outcome is benign (a lost score tweak). Losing a content correction to a concurrent consolidation is silent and defeats the entire point of the feature.

file_edit already solved this on the file surface and the pattern should be copied:

• per-key SemaphoreSlim in a static ConcurrentDictionary (FileEditToolExecutor)
• read-compare-write guard — FileText.WriteAtomicIfUnchangedAsync re-reads and byte-compares before writing, refusing with "modified by something else"
• atomic replace via sibling temp file + File.Move(overwrite: true)

Applying this means touching FileMemoryStore (and ideally ILongTermMemory), not just adding a tool method. This should be treated as in scope, not as a follow-up.

Index and embedding side effects

Mostly free, with one trap:

• FileMemoryStore.SaveAsync:85-87 re-embeds unconditionally on every save, in the background. A read-modify-SaveAsync edit gets re-vectorization automatically.
• The edit must go through SaveAsync. Patching the JSON on disk directly would leave the embedding stale against the new text and the in-memory index (:76) wrong.
• Memory embedding input is content + tags + category (GetDocumentText:495-503), so a content edit genuinely does invalidate the vector.
• Skill embedding input is name + summary only (FileSkillStore.GetDocumentText:497-503), so a skill body edit needs no re-vectorization — though the save path re-embeds regardless.
• BM25 has no persisted index — Bm25Ranker is stateless and scores at query time over the in-memory candidate list. Nothing to invalidate.
• Category changes are the structural gotcha: SaveAsync:68-74 moves the file when Category differs. A content-only edit is safe; a tool that also edits category must go through the same path.

Proposed shape

1. Add compare-and-swap + atomic write to FileMemoryStore (and the skill/working-memory stores as applicable), modeled on FileText.WriteAtomicIfUnchangedAsync.
2. Add edit_memory(id, old_string, new_string, replace_all=false) on top of TextEdit.Apply, honoring the timestamp contract above. Surface NotFound / Ambiguous as informative refusals, as file_edit does — a refused edit means re-read, not retry.
3. Repeat for working memory, skills, and the scheduled-task directive.
4. Fix the rules.md lossy round-trip, then apply the same treatment.
5. Skill-guide text directing the agent to prefer the edit tools over full replacement for existing content, mirroring the file_edit guidance added in Add file_edit tool and protect durable paths from the shared-volume sweep #504.