Repository navigation
Conversation
…an interrupted copy
…place an interrupted copy
| echo "Could not remove the partly copied volume ${new_volume}" >&2 | ||
| return 1 | ||
| fi | ||
| remove_partly_copied_memory "${new_volume}" "${new_container}" "${migration_label}" || return 1 |
There was a problem hiding this comment.
Could we check whether memory was successfully imported after the interrupted migration, or preserve the existing volume before deleting it? A stale .migration-started marker could otherwise cause a normal start to discard valid imported memory. I see this is documented under Known limits - is this behavior intentional within the scope of OMEGA-492, or should we handle it here?
There was a problem hiding this comment.
Could we check whether memory was successfully imported after the interrupted migration, or preserve the existing volume before deleting it?
Done in 7263f93. The copy goes entry by entry, so a partial copy leaves history.metta empty, missing, or a prefix of the old file. Before redoing the copy, start compares the two files. If omega-memory holds anything else, such as an imported archive, start stops, leaves the volume and the omega container as they are, and prints two commands: one clears the marker and keeps the memory, the other removes omega-memory so the copy runs again.
On singularitynet/omega:v0.1.20 both commands work, and a partly copied history is still copied again. The new cases are in d4f1da4.
There was a problem hiding this comment.
Thanks, that covers the case where history has changed. What about --only-vector, where imported vector memory may change while history.metta stays the same?
| if ! old_state=$(docker run --rm --label "${migration_label}" --entrypoint sh --volume "${old_volume}:/from:ro" "${image}" \ | ||
| -c "if [ -e /from/${marker} ]; then echo migrated; elif [ -e /from/${started_marker} ]; then echo interrupted; fi"); then | ||
| if [ -n "${memory_import_file}" ]; then | ||
| return 0 |
There was a problem hiding this comment.
Returning success here allows the import to proceed without checking or clearing a potentially existing .migration-started marker. If the marker check fails temporarily but the subsequent import succeeds, the next normal start can interpret the imported volume as an interrupted copy and delete it. Can we abort with an actionable error when migration state cannot be determined?
There was a problem hiding this comment.
Can we abort with an actionable error when migration state cannot be determined?
Yes, done in 7263f93. A failed marker read now stops the import with "Could not read the migration markers in omegaclaw-memory", the same way it stops a plain start. The marker can no longer outlive an import. Covered by test_memory_import_stops_when_markers_cannot_be_read in d4f1da4.
…able markers on import - Plain start must keep omega-memory when its history is not a prefix of the old one - Partly copied or missing history is still copied again - A memory import stops when the migration markers cannot be read
… on unreadable markers - Before redoing an interrupted copy, compare history.metta in omega-memory with omegaclaw-memory - Stop with commands to keep the memory or to copy again when it is not a partial copy - A memory import no longer skips a failed marker read, so a stale marker cannot remove imported memory later
This branch has not been deployed
2 participants
If the copy from
omegaclaw-memorydid not finish, the nextstartdeletes the partly copiedomega-memoryvolume and copies again. Docker does not delete a volume that a container uses, even a stopped one, so onpatch-to-v0.1.20thatstartstops with "volume is in use" and "Could not remove the partly copied volume omega-memory". Two kinds of containers can hold the volume at that point. One is a copy container that keeps running after the launcher is killed, for example by SIGHUP when its terminal closes. The other is anomegacontainer, left bystart --memory-importon v0.1.20 or started by hand.Changes in
scripts/omegaomega.memory-migrationlabel, sostartcan tell them from other containers.omegastill uses the volume,startstops, names it and prints thedocker rm -fcommand. Nothing else is removed, andomegastays.startremovesomegaif it exists, prints "Removed container omega, it used the partly copied volume omega-memory" and deletes the volume. A plainstartalready removesomegaright after the migration step, so only the order changes..migration-startedinomegaclaw-memory. Before, the import skipped the migration and kept the marker, so every later plainstarttreated the imported memory as a partial copy.Known limits
startnames them and does not remove them.startwith this change sees thatomega-memoryis not a partial copy, stops, and prints how to keep it or copy it again.--memory-importafter an interrupted copy starts from a new volume in every import mode, because the partial copy is dropped. If that import fails, the nextstartdoes not copyomegaclaw-memoryagain, the same as after an import with no earlier copy.Testing
tests/test_omega_launcher_migration.pynow records the volumes and labels of each container. 14 new test cases (lines 443-553 and 603-648) cover the cases above, and 11 of them fail with the launcher frompatch-to-v0.1.20.singularitynet/omega:v0.1.20on Docker 29.4.3:omegacontainer,startonpatch-to-v0.1.20fails with "volume is in use". With the change it removesomegaand copies again,history.mettainomega-memorymatchesomegaclaw-memory, and theomegacontainer starts. A runningomegacontainer gives the same result.startruns again while the copy container still works,patch-to-v0.1.20fails with "volume is in use". With the change the copy container is removed and the copy is redone.omega-memory,startstops and names it. The volume, the markers and a runningomegacontainer stay as they were. Afterdocker rm -fof that container the nextstartcopies.--memory-importwith an exported archive over an interrupted copy, the partly copied volume is replaced and the import completes. The next plainstartkeeps the imported memory and copies nothing.omega-memoryholds memory that is not a partial copy over an interrupted copy,startstops and leaves the volume and the stoppedomegacontainer as they were. Each of the two printed commands works: one keeps the memory, the other copiesomegaclaw-memoryagain. A partly copiedhistory.mettais still copied again.