Keep a pad until its file is deleted for good, and get a lost pad back from its file - #302
Merged
Merged
Conversation
trashed_at says when the file was seen going to a trash or being deleted; missing_since, when a sweep first missed it without that. Both nullable and indexed. Nothing writes them yet.
A .pad file moved to a trash or deleted past it, every .pad file under a folder that goes, and every file of a user about to be deleted get trashed_at on their row: once such a file is gone from the file cache, it is gone for good, however its trash was emptied. MarkLeavingPadsListener hears MoveToTrashEvent, BeforeNodeDeletedEvent and BeforeUserDeletedEvent and never throws. A folder's files are found in the file cache by the folder's storage and path. The in-memory binding table of the tests now holds a file cache too and evaluates joins, LIKE, IN and IS NULL.
GoneFileSweep runs every five minutes. Active rows marked as leaving Files whose file the file cache has nothing left of are gone for good: pad and row go. A pass over every active row, 200 a run with a cursor, keeps the dates in step with the file cache: it marks a file under a trash path, clears the mark of one back in Files, starts a seven-day grace period for a file missing without a mark, and ends it for one found again. After the grace period pad and row go too, unless more such files went missing since the brake's last release than its threshold (20): then they are kept and a warning says so, once. The file cache is asked once more right before a pad goes. A pad Etherpad no longer has counts as deleted, a refusal is logged and tried again, and Etherpad not answering ends the run with the slice taken again next time. With delete_on_trash off nothing is deleted, but the dates and the brake are kept. The grace period, threshold, release time, brake state and cursor live in the app config through IAppConfig.
… good The consistency check counts the active rows whose file went missing without being seen leaving Files, in their grace period or held, and says whether the brake holds; its message then names the brake. While it holds, the admin page offers to release it, after asking: POST /api/v1/admin/gone-file-brake/release. Rows missing until then no longer count, and their pads go once past their grace period. architecture.md describes the sweep under "Files gone for good", with the two columns and the events that mark a row; api-reference.md lists the new fields and the endpoint, and drops the consistency counters the check never returned. New strings in de, es, fr and it.
A file that goes missing without passing a trash, a delete or a user's deletion is left alone, pad and row with it, as before: no grace period, no brake, no missing_since column, and no admin action to release the brake. The migration adds trashed_at alone. The sweep's pass now reads only the active rows whose file the file cache has, and keeps their marks: it sets the mark of a file under a trash path and clears the mark of one back in Files. It asks only the database, so it runs even when Etherpad does not answer or the run's budget is spent. architecture.md describes the sweep under "Files gone for good".
Nextcloud runs an app's migrations only when info.xml's version rises, and a branch keeps it. sync-app.sh now asks which migrations the tree has that the stack has not run, and if there are any upgrades the app as a release would. It asks for the set, not the next one after the last run, so a stack that ran another branch's migrations still gets this one's.
A delete past the trash - a WebDAV DELETE with X-NC-Skip-Trashbin, the trash app off for the user, a move to the trash that fails - takes the file at once, so its pad goes at once too. The listener, now LeavingPadsListener, holds the files it marked before a delete until Nextcloud reports the delete done (NodeDeletedEvent), and the sweep deletes the pads of those gone from the file cache, within five seconds. A file the trash took is still in the file cache and is passed by. What does not fit, or finds Etherpad not answering, keeps its mark for the next run, and nothing is thrown at the delete. Nextcloud reports a deleted folder as a file, so what counts is what was marked before the delete. markTrashedUnder() names the files under the folder for that. docs/deleting-pads.md lists every way a .pad file can leave Nextcloud and what happens to its pad; architecture.md and the README link it.
A file moved to the trash keeps its pad, and a restored file has it back; the pad goes once the file is deleted for good. The page now says so, with the trash and a deletion for good apart, and what a protected and a public pad do while their file is in the trash.
The page keeps the trash and a deletion for good apart, and says what a restore gives back today: a new pad from the file's snapshot, or the old pad while its deletion still waits. It changes with the code that keeps pads through the trash.
groupfolders 22 gives each team folder its own storage, with its trash under a bare trash/ beside files/. The sweep's pass took a file there for one back in Files and cleared its mark, so a folder deleted from that trash for good left its pads behind. The pass now clears a mark only for a file surely in Files - files/, or a team folder's __groupfolders/<id>/ on the root storage - and leaves a path it cannot place as it is.
The admin's settle now runs the sweep of files gone for good as well, as its job would, so a spec can run what the background jobs do within minutes. pad-gone-for-good.spec.ts, on the container stack: a protected pad and a folder's pads go in the request that deletes past the trash; a folder in the trash keeps its pads until it is deleted from there; a deleted account takes the pads of its own files. With groupfolders enabled, the same for a team folder, and a pad an account made in a team folder stays when the account goes. Fixtures for accounts, groups and team folders through the OCS APIs, for deleting past the trash, for making a pad as another account, and for asking Etherpad whether a pad or group is still there.
Team folders are how many installations share pads, and a team folder's trash keeps its files apart from a user's, so the suite runs against them on every supported major: groupfolders 19.1.20, 20.1.18, 21.0.15 and 22.0.6 for Nextcloud 31 to 34, each checked against its release checksum. The full-text-search apps go through the same installer.
A restore now clears the marks of what it brings back: NodeRestoredEvent for core's trash, and the legacy post_restore hook, which groupfolders alone raises, for every restored item, folders included. Before, a restored file kept its mark until the sweep's pass reached it, and a file that then went unseen - a team folder deleted as a whole - lost its pad. A delete no longer marks before it happens: it looks the files up, a move to the trash marks those rather than looking them up again, and a delete past the trash marks them once they are gone, then deletes their pads. So the pass cannot clear a mark on a file a delete is about to take, and a folder's files are joined once per trash, not twice. A user about to be deleted has their home storage found through the mount cache, without setting up a home just to delete it. The sweep's pass takes slices of 1,000 rows, up to ten a run, and keeps a mark younger than five minutes. A pad Etherpad refuses to delete is tried again an hour later, so it neither holds the head of the queue nor warns every run. The request that deletes past the trash reports Etherpad not answering as info and anything else as a warning. The admin's settle runs the sweep in what the older sweep left of one budget, not in a second one.
A new Nextcloud major or a nightly has no groupfolders release pinned yet. The stack now comes up without it, with a note, and the team folder specs skip, rather than stopping before Nextcloud starts.
A pad whose file went without the app seeing it - a team folder deleted as a whole, a file removed outside Nextcloud - stays, by design. The consistency check now counts these apart (vanished_file_count: active rows never seen leaving Files) and returns up to 25 of them; the admin page lists them by pad id, so an admin can delete what is no longer needed in Etherpad. architecture.md and api-reference.md described checks and counters the consistency check never had; they now say what it does.
Core raises the legacy post_restore hook and NodeRestoredEvent for one restore; the listener now remembers what it cleared in the request, so a folder's files are looked up and cleared once. A move to the trash marks what the delete before it looked up and keeps it for the delete's end: a move that fails falls back to a delete past the trash, and its folder's pads then go in the same request too, as docs/deleting-pads.md says. For a move that succeeds the files are still in the file cache, and the sweep passes them by. Binding::$trashedAt and architecture.md say that a refused deletion moves the mark ahead, to the next try.
The pads under a folder that goes to the trash, is deleted or restored were found by a prefix match on the path. The file cache has no index for that on Postgres, so each folder read its whole storage: measured, 46 ms for a folder of 1,000 files on a storage of a million. A team folder from before groupfolders gave each its own storage shares the root storage with all others, and there that grows with every team folder. FolderPadFiles now walks down from the folder through the parent index, one level of folders at a time, and fetches only folders and .pad files: 0.6 ms for the same folder, growing with the folder alone.
An open that may write now asks Etherpad once whether it has lost the file's pad, before any address or session. Lost is no pad under that id - a protected pad, or a public one whose file holds saved content - or one without a single revision while the file's snapshot holds more: a public pad Etherpad made anew, empty, when someone visited its address. A public pad with nothing saved, an Ownpad link to a pad nobody opened yet, is made on the first visit as before; a pad merely behind its snapshot is not lost, as restores in 1.1.0-beta.1 left such files. A lost pad answers pad_missing. The viewer and the embed page show the recovery card without looking for an original, and recover-from-snapshot, which asks Etherpad again, makes a new pad from the file's content through the path a restore takes for a row whose pad is gone: the row moves onto it and the file names it. No other way of making a pad is added. A reader is not asked; a public share that may write is told only the owner can make the new pad. Logged once a minute per file, as a refusal is. pad-lost.spec.ts covers a public and a protected pad deleted in Etherpad, a public pad made anew empty, and the viewer's card.
A file back from the trash is surely the one its pad was, so a restore no longer leaves the question to the next open. A file whose row stayed active takes its pad back as it is, unless Etherpad lost it while the file was away: then the restore makes a new pad from the file at once, through the same replacement the recovery uses. Etherpad not answering, or a file that cannot be read, leave the row to the next open. A file back without a row, whose pad went with the trash as in 1.1.0-beta.1, now gets its new pad whatever delete_on_trash says: nothing is left to keep. RestoreService no longer reads the setting. A folder restored does not ask Etherpad for each of its files; one an open finds lost offers a new pad, as before.
A file back from the trash without a row gets a new pad only when no other file's row names its pad. A copy that was never opened has no row either, but its pad lives on with the original: it now comes back as it went, and its open still offers the original. A core restore arrives by the hook and then by the event. What the hook pass decided - the pad restored, made anew, or left as it is - the event pass now leaves, rather than ask Etherpad the same question again. It still tries again after a hook pass that could not read the file or failed on the database. The restore of an active row and the API's recovery share one check of whether the pad is lost. The restore's skip names what it found: pad_present, row_names_other_pad or copy_of_another_file. Etherpad refusing to answer at restore is reported, not taken for an unreadable file.
A pad's file is deleted for good when Nextcloud removes it from the file cache: from a trash, while it deletes the file or a folder above it, or with its account. The marks now come from those removals (CacheEntryRemovedEvent), one for each of a folder's files, instead of looking up .pad files by name before a trash or delete. A .pad renamed keeps its row, and its pad goes with it. A removal a scan makes - a file deleted outside Nextcloud - does not count: its pad stays, and the consistency check lists it. The request that deletes only marks, in blocks of 500; the job deletes the pads within minutes, for a delete past the trash too. A move to another storage, reported as a removal and an insert of the same id, takes its mark back. The sweep's pass over every row and its cursor, the folder walk, and the marks set on trash and cleared on restore are gone. The column is gone_after: the time from which the pad may go, moved ahead after a refusal. Nextcloud 34 up to 34.0.4 reports a removed folder's files under wrong ids (nextcloud/server#63969). Such removals are recognised and left out, so no other file's row is marked; the pads of the files inside a folder deleted for good there stay until the fix. pad-gone-for-good.spec.ts covers a renamed file deleted past the trash and from it, and skips the folder cases on the affected versions.
An account's files are looked up before it is deleted and marked only once Nextcloud reports it gone (UserDeletedEvent). A deletion the user backend refuses stops before that, and no longer leaves marks on files that stay. A mark is due five minutes after it was set, a margin before a step that cannot be undone; the admin page's settle takes it at once. Each run first clears the marks of files the file cache still has an hour after they were due - a deletion that did not happen after all - so a file a scan drops later keeps its pad, as any such file does.
A trash no longer deletes the pad. The file keeps its row, active, and its pad; a restore gives it the same pad back, with its history. The pad goes once the file is deleted for good, through the sweep of files gone for good. A file seen deleted for good is now a row in pending_delete, dated by deleted_at; the grace counts from deleted_at, a retry from updated_at, and the gone_after column goes. What the old trash needed goes with it: the snapshot written into a trashed file, the deletions a trash owed and their three jobs, the restore of a row that waited, restore_pending, the settling of a waiting row on open, and the waiting_binding error code. POST /api/v1/pads/trash and /api/v1/pads/restore, which ran that flow for a path, are removed; recover-from-snapshot stays. The health check and the admin page's check report pending_delete_count alone. A restore still makes a new pad from the file's snapshot where the file has none: a file without a row, trashed under an earlier version, and one whose pad Etherpad lost while it was away. A row in pending_delete whose file is opened or restored was not deleted after all and is active again. On upgrade the rows that waited under the old trash are taken over: a deletion owed whose file is still in a trash or back in Files becomes active, one whose file is gone for good stays for the sweep, and a restore left undecided becomes active. The lifecycle shell scripts, which drove the removed endpoints, and the spec of the trash's group clean-up go. pad-gone-for-good.spec.ts covers a .pad file in the trash keeping pad and group, getting the same pad back on restore, and taking both when deleted from the trash.
The trash keeps a pad now, and a session would keep giving it to whoever holds one, as the deleted file no longer does. A delete - to the trash or past it (BeforeNodeDeletedEvent) - now revokes the sessions of the protected pads it takes along: a file's own, or every protected pad under a folder, found through the file cache's parent index and the binding rows, whatever the files are called. It is built on the logout's revocation. PadSessionRevoker::revokeForPads() lists each group's sessions (listSessionsOfGroup) and deletes the live ones within the logout's budget; what does not fit expires on its own. A group loses its sessions only while it holds that pad alone, or nothing: a legacy .pad may name someone else's group. A public pad has no sessions and stays reachable by its address. pad-gone-for-good.spec.ts covers a protected file and a folder opened and moved to the trash: no live session is left, and the pads stay.
The trash keeps pads now, so the setting that said whether a trash deleted them says whether a file deleted for good takes its pad along. Its name says so: delete_on_permanent_delete, after Nextcloud's "Delete permanently", with the admin page's label and hint to match. On upgrade the value an admin gave delete_on_trash is taken over, unless the new setting is set already, and the old key is removed. The setting is read and written through IAppConfig alone, as a string, so it keeps one type whichever way it is written.
pad-lost.spec.ts now covers the restore of a public pad's file whose pad Etherpad deleted while the file was in the trash: the restore makes a new pad from the file's content, and the next open finds it without the card.
A restore left undecided never shipped; the table is there by the time the step runs.
The consistency check still asked for gone_after, which no migration makes: on a real database it failed. The in-memory table now knows the columns a database has, so a statement naming another fails there too, and the e2e case of a team folder deleted as a whole runs the check.
… cached A delete that fails raises no NodeDeletedEvent, which left every later removal of the process counting as a deletion, a scan's in a cron run too. A delete now counts the removals of its own node and of what is under it, found by the node's place in the file cache. Marks from removals skip files the file cache still has, so a removal under a wrong id marks no file that is there. The upgrade takes the three retry jobs of 1.1.0-beta.1 off the job list, and the docs say where the fix for Nextcloud 34's misnumbered removals stands.
The walk stopped past a hundred pads or ten thousand folders, but each query still read all it found: one folder with twenty thousand others in it read them all in the delete's request. The queries ask for one more than there is room for, which also tells that the walk stopped short.
…an still meet a recovery The admin page told admins to delete vanished pads in Etherpad; doing so leaves their rows, and the list never gets shorter. The hint now says so. A recipient's restored copy holds the content of its last sync, not all it had. And a mismatch between file and row can come from a forced sync writing in the moment after its last look at the row, not only from a rollback that fails.
…its mount A scan dropping entries of a trash - files missing on disk for a while, say - counted as a deletion from the trash, and the sweep took their pads. `occ files:scan` sends NodeRemovedFromCache right before it removes an entry; that entry and all under it no longer count, in a trash or under a delete whose window a failure left open. A deleted user's home was looked up in the mount cache, and a home it had no row for kept its pads. It is found from the home mount, as Nextcloud's own cleanup of a deleted user finds it.
A row seen deleted that two requests find at once is taken back by one of them; the other found its transition lost and refused the file with "Pad binding is not active." although the row was active. It reads the row again.
A row counts as never tried while updated_at equals deleted_at. A pad Etherpad refused in the second its file was seen deleted stayed so: it was tried again at the next run instead of an hour later, and its warning was given twice.
A file back without a row whose pad another row names was left as a copy, whatever became of the original. With the original deleted for good - a share recipient restoring their copy after the owner emptied the trash - its pad was about to go, and the copy was left none to open. It now gets a pad of its own from its content.
Code of this version running before its migration - deployed without a version bump - read only the new key, whose default is on, and a settle deleted pads the admin chose to keep. Until the new key is set, the old one's word stands.
With deleting off the check deleted nothing and answered as if all went well, while the count of pending deletes kept growing. Its message now says that deleting is off.
A group was checked against the first of its pads a delete took along: two Ownpad files of one group deleted together left it holding another pad, and its sessions stayed. It is checked against all of them.
…nd the settle's answer
A pad Etherpad made anew in place of the file's - at revision 0, with other text than the file saved - is not the file's document. Written over the file, its default text would leave the saved content to the file's versions, and the open would no longer find the pad lost and offer a new pad from the file. The sync now refuses it with pad_missing, by the rule the open goes by. A pad behind the snapshot that holds a revision still syncs, so the files a restore in 1.1.0-beta.1 left with the old pad's revision count still come up to date.
|
ⓘ Qodo reviews are paused because the subscription is no longer active. Ask your workspace admin to reactivate the subscription to resume reviews. Manage billing |
PR Summary by QodoKeep pads through trash and recover lost pads from file snapshots
AI Description
Diagram
High-Level Assessment
Files changed (163)
|
Code Review by Qodo
1. Trashed pads keep active edit sessions
|
![qodo-free-for-open-source-projects[bot]](https://avatars.githubusercontent.com/in/1068000?s=60&v=4){kind=small}
Comment on lines
+130
to
+135
| if (!$this->bindingService->isFileGone($binding->fileId)) { | ||
| return false; | ||
| } | ||
| $context = ['app' => 'etherpad_nextcloud', 'fileId' => $binding->fileId, 'padId' => $binding->padId]; | ||
| try { | ||
| $this->padLifecycle->discardIfPresent($binding->padId, $budget, retried: true); |
There was a problem hiding this comment.
1. Restored files can lose their pad 🐞 Bug ≡ Correctness
GoneFileSweep::discard() checks that the file is absent before calling Etherpad, but does not recheck the binding before deleting the pad. If a restore puts the file back and reactivates its row between those steps, the sweep deletes the restored file's pad while deleteInState() leaves its active row in place.
Agent Prompt
## Issue description
A restore can reactivate a binding after the sweep checks the file cache but before it deletes the pad, leaving the restored file bound to a deleted pad.
## Fix Focus Areas
- lib/Service/GoneFileSweep.php[129-157]
- lib/Service/RestoreService.php[74-90]
## Recommended Fix
Serialize the restore and sweep decisions for a binding so the sweep cannot delete its pad after restoration has reclaimed it. Ensure the final state check protects the Etherpad deletion, not just the database-row deletion.
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Collaborator
Author
There was a problem hiding this comment.
Left as it is: this interleaving cannot happen. The sweep deletes a pad only while the file cache has nothing under the file's id, checked when the row is picked and again right before the delete. A restore needs the file's entry in the trash, and a restore or a move updates that entry in place, keeping its id; an id the file cache no longer has does not come back. So a file whose pad the sweep is deleting cannot be restored under that id.
An account deleted without a logout kept its sessions, and with them the protected pads of other people's files shared with it, until they expired. They go now as the delete starts, as on a logout: Nextcloud removes the account's settings, the cached Etherpad author among them, before it reports the account gone.
…does not know A restore that could not finish on a main after 1.1.0-beta.1 left its row restore_pending, and a deletion owed may have no date. Neither is reached by anything here: an open refuses them, and no list shows them. The migration dates them as seen deleted now, so the next step makes those whose file is there active, and the sweep takes the pads of the rest.
The form read the new key alone, with its default on, while the sweep falls back to the old key until the migration takes it over. An admin who had switched deleting off and saved the form before the migration wrote the new key on, and the takeover kept that. The form reads the setting through AppConfigService now, as the sweep does.
The session TTL, the revoker's budget, a revoked share and the list of events still said only a logout ends a session. The event list names the logout's listener, which it had left out, beside the new one.
The window NodeRemovedFromCache opened stayed open for the rest of the process: a removal under that path later on - the trash expiring, a delete - no longer counted, and the pad was left and listed as vanished. Nextcloud reports the dropped entry after all under it, so its own removal now ends the window, once it is judged itself. The listener's list of what never counts names the scan's drop, the event lists name NodeRemovedFromCache, and the lookup of a deleted account's home says it sets up that storage as Nextcloud's own cleanup does at the same event.
The forced sync's table repeated the rule of a pad made anew, which the lifecycle test holds case by case; the sync keeps one refused case, one written behind the snapshot and one new file. A copy's restore looped over row states the code never reads. Two of the four end-to-end checks of the forced sync took the same branch of the rule as the visit; the visit and the 1.1.0-beta.1 template keep it.
Collaborator
Author
|
/agentic_review |
|
Code review by qodo was updated up to the latest commit 33145b7 |
A recovery made its new pad from the file as it read it, and after the seeding asked only whether the file had moved. A file written meanwhile - a sync of a pad made anew that someone wrote into, an upload - lost what was written to the older snapshot. The file is read again before the claim: holding something else, the new pad goes, nothing is claimed or written (file_changed), and the next open asks again.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two changes to what happens to a
.padfile's pad. They share one lifecycle, so they come as one PR:.padfile's saved content is always enough to get a pad back. Whether Etherpad lost the pad, the trash of an earlier version deleted it, or the file was copied in from elsewhere, a new pad can be made from what the file holds.Until now the trash deleted the pad, after writing its content into the file, and a restore made a new pad from that snapshot. The restored pad was a new one, without its history, and a public pad came back under a new address. Every step could fail halfway: Etherpad down, the file locked, or a restore racing the sweep. So rows waited in
pending_deleteorrestore_pending, three retry jobs worked them off, and opening a file had to settle a waiting row first. That machinery is gone. The diff adds about 7,500 lines and removes 8,000.What a delete does to a pad
DELETE, a folder with.padfiles in it, a user's own folders or a team folderocc trashbin:cleanup, a team folder's trash tooX-NC-Skip-Trashbin: true, or the trash app off for the uservanished_file_count).This fixes four cases where
mainleaves pads behind for good:.padfiles inside a folder that went through the trash;.padfile;The new
docs/deleting-pads.mdcovers every way of deleting a file, for admins and users.Getting a pad back from the file
The new pad is made from the content saved in the file. It is always a new pad: the pad id the file names is never taken over, even if Etherpad has a pad of that name.
A file restored from the trash gets its new pad without asking. A file back from the trash is surely the one its pad was, so there is nothing to decide:
Any other file asks when it is opened. Viewer and embed page show a card that offers to make a new pad from the file's content:
A copy stays a copy. A never-opened copy of another
.padfile restored from the trash is not given a pad of its own: its pad lives on with the original, and its open offers the original or a new pad, as for any copy.All of these take one path: a new pad from the file's snapshot, the row made or moved onto it, the file naming it (
RestoreService::restoreOntoNewPad()). No other way of making a pad is added.How
Seeing a file deleted for good.
GoneFilesListenerlistens to the file cache. Nextcloud reports every entry it removes (CacheEntryRemovedEvent), including each descendant of a removed folder. A scan also removes entries, for files that vanished outside Nextcloud, so only these removals count:files_trashbin/files/on a home storage);__groupfolders/trash/);trash/, told apart from other folders of that name by the storage id).BeforeNodeDeletedEventand itsNodeDeletedEvent: a delete past the trash, or one the trash was meant for but did not take (an app vetoing it, the move failing). The node's storage and path are looked up in the file cache when the delete starts. A delete that fails (a locked file) raises noNodeDeletedEvent; its window stays open only on its own entries, so a scan later in the same cron run never counts as a delete.MoveToTrashEvent, which the trash app sends before it tries.BeforeNodeRestoredEventandNodeRestoredEvent: what it removes from the trash does not count. The window closes by the source's path, which has no id once restored.BeforeUserDeletedEventfrom the account's home mount, as Nextcloud's own cleanup finds it, and marked atUserDeletedEvent.Some details of the listener:
pending_delete, dated bydeleted_at..padfile takes its pad along.CacheEntryInsertedEvent), and the insert takes the mark back.UPDATE … WHERE file_id IN), when a delete through a node is done (NodeDeletedEvent, or\OCP\Files::postDelete, which a trash's deletes send too), and at the end of the process, and only for files the file cache no longer has. The rows are looked up first, so a removal of a file without one costs no look into the file cache, and the listener keeps only the files it marked: a cleanup over millions of entries holds a handful. The request never calls Etherpad, and the listener never throws.occ files:scansendsNodeRemovedFromCacheright before an entry goes, and that entry and all under it are left out, until the entry's own removal, which Nextcloud reports after all under it. A removal there later in the same process counts again.The trashbin's
OC_Hooks were not an option. They are old hooks with relative paths, and groupfolders,occ trashbin:cleanupand account deletion do not send them. Its typed restore events are used.Deleting the pad.
GoneFileSweepruns on every tick of a five-minute cron (GoneFileSweepJob, declared ininfo.xml, with an interval of four minutes: Nextcloud runs a job only once more than its interval has passed, and five would run it on every other tick), within 20 s, in batches of 200 and at most ten per run.checkTokenfirst, and if it answers, the pad waits its hour and the run goes on.settle-pending) runs the sweep at once, and says so when deleting is off, including rows still in their five minutes and rows Etherpad refused within the hour, after the admin fixed Etherpad, say. A run tries each row once.delete_pad_with_fileoff, no pad is deleted and the rows keep waiting.Sessions.
RevokeSessionsOnDeleteListenerruns for a move to the trash and for a delete past it alike.BeforeNodeDeletedEventit collects the protected pads the delete takes along. For a file that is its own row. For a folder it is every active protected row below it, found by walking the file cache'sparentindex one level at a time (ProtectedPadsOfNode), whatever the files are called. An instance without an active protected pad takes no walk; a new index on state and access mode answers that. The walk stops past 100 pads or at 10,000 folders, with a line, and its queries read no more than it has room for.NodeDeletedEvent, once the delete is done,PadSessionRevoker::revokeForPads()reuses the logout's loop: at most two seconds, and up to 100 deletes, newest first, since every open makes a session and the newest are those of whoever is at it now. Whatever does not fit expires on its own. A delete that fails, a file locked by a sync say, takes no sessions.ManagedPadLifecycle::groupHoldsOnly()). A legacy Ownpad file can name someone else's group; a legacy group whose pads all leave together loses its sessions.RevokeSessionsOnAccountDeleteListener). An account deleted without a logout kept them, and with them the protected pads of other people's files shared with it, until they expired. They go as the delete starts: Nextcloud removes the account's settings, the cached Etherpad author among them, before it reports the account gone. A delete the user backend then refuses has lost them as a logout would.Recognising a lost pad. An open that may write asks Etherpad before any address or session (
ManagedPadLifecycle::howLost()). Each question waits at most three seconds. Only a definite answer stops the open: Etherpad slow, silent or refusing the question opens the pad as before, with a line atdebug. Any other fault of the check opens it too, with awarning. A pad counts as lost when:A file holds saved content once its snapshot is past a pad's first revision (
snapshot_revabove 0), or once it holds any text: a file made from a template by 1.1.0-beta.1 holds its content atsnapshot_rev: 0.It does not count as lost when:
A lost pad answers
400with the codepad_missing, which the card acts on. A reader is not asked, and is shown what the pad server has. A public share that may write is told in one sentence that the file's owner can make the new pad. The refusal is logged once a minute per file.Restore.
.padwhose file is still there. A copy whose original is gone - deleted for good, or vanished - gets a pad of its own. One that cannot be read, such as a legacy Ownpad link without metadata, is left to its open.pending_deleterow whose file is back means the delete did not happen. The row becomes active again. An open does the same.Safety
pad_missingfor a pad without a single revision whose text is not the text the file saved, by the rule the open goes by (ManagedPadLifecycle::isMadeAnew()), and writes nothing. A pad behind its snapshot that holds a revision still syncs, which is what brings files a 1.1.0-beta.1 restore left with the old pad's revision count up to date.file_moved,file_changed); the next open asks again.recover-from-snapshotanswers403to anyone who may not change the file, such as the recipient of a read-only share, before Etherpad is asked. A new pad means writing the file and moving its row.ManagedPadLifecycle::seed()), and restore, recovery and templates all write that count assnapshot_rev. Before, a pad made from a template saidsnapshot_rev: 0. Lost before its first sync, it was handed out; Etherpad made it anew, and the next sync wrote that pad over the template's content. Files 1.1.0-beta.1 made from a template still say 0, and count by their saved text. If Etherpad does not report the count, a template's file starts at 0, and counts by its text too.POST /api/v1/pads/trashran for anyone who could see the file. Called by the recipient of a read-only share, it deleted the owner's pad. This PR removes the endpoint.Upgrading from 1.1.0-beta.1
Version000005Date20260928120000:pending_delete.restore_pending, which amainafter beta.1 wrote for a restore that could not finish, and apending_deleterow without a date. Otherwise an open would refuse them and nothing would list them.delete_on_trashbecomesdelete_pad_with_file, keeping the admin's value. The value is read and written only throughIAppConfig, as a string. Until it is taken over, the old key's value stands, in the sweep and in the admin form, so this code running before its migration keeps an admin's opt-out, and saving the form does not undo it.Hot,Warm,ColdPendingDeleteRetryJob) are taken off the job list. Otherwise the first cron run would drop each with a warning.test_faultkey a debug instance of beta.1 may have set is removed, with the faults.state(ep_bind_state_idx) is replaced by one onstateandaccess_mode(ep_bind_state_mode_idx), which also serves every lookup bystatealone.API and admin changes
POST /api/v1/pads/trashandPOST /api/v1/pads/restore. They ran the old trash and restore for a path; only the shell scripts used them, and those are deleted too.POST /api/v1/admin/test-faultand every test fault it set (trash_*andrestore_*), with thetest_faultsetting. Only the removed shell scripts set them.pad_missing(PadLostException),400, on open, and on a sync that would write a pad Etherpad made anew over its file.POST /api/v1/pads/recover-from-snapshot/{fileId}:403to whoever may not change the file;409as before;409withstatus=skippedandreason=file_movedorfile_changedwhen the file moved or was written while the new pad was seeded.delete_on_trashis nowdelete_pad_with_file, in the admin API and the form. The admin page says it applies once a file is deleted permanently.settle-pendingnow only deletes the pads of files deleted for good. It returnschecked,settledandpending_delete_count.vanished_file_count, withsamples.vanished_files), reports issues on them, and the admin page lists them by pad id.binding_without_file_countandsamples.bindings_without_fileare gone: they also counted files seen deleted for good, which are on their way, and every such file made the check report issues, for good while deleting is off.restore_pending_countin the health check, and the error codewaiting_binding.docs/api-reference.mdno longer listsfile_without_binding_countandinvalid_frontmatter_count. The check never returned them.Left as it is
occ trashbin:cleanup, deletesfiles_trashbinas one folder, so it is affected too..padfiles inside a folder deleted for good there, or in a trash emptied as a whole, keep their pads, and the consistency check lists them.occ files:scansays what it drops; the background scan and the watcher do not, and what they drop in a trash still counts as a deletion from it. Files in the trash of a team folder kept on Nextcloud's own storage go with it as from a trash, and take their pads.meta-by-id,resolve) still hand out the pad address from the file, without asking Etherpad. That is left for a later change.Tests
GoneFilesListenerTest(with a move to the trash, a restore, a file gone again after coming back, a scan's drop ending with its own entry, only marked files kept),ProtectedPadsOfNodeTest,RevokeSessionsOnDeleteListenerTest(sessions taken once the delete is done, none for a failed one),RevokeSessionsOnAccountDeleteListenerTest,KeepPadsThroughTheTrashMigrationTest(withrestore_pendingand undated rows);AdminSettingsRepositoryTest: the form shows the old opt-out until the setting is taken over;GoneFileSweepTest,RestoreServiceTest,BindingServiceTest;PadSessionRevokerTest: a slow Etherpad still revokes the first group's sessions;pad_missing.pad-gone-for-good.spec.ts, 11 cases:pad-lost.spec.ts, 9 cases:snapshot_rev: 0;pad-orphan-recovery.spec.ts: a copy restored from the trash still offers the original.protected-pad-group-cleanup.spec.tsis removed; its scenario no longer exists.sync-app.shruns the migrations a branch adds, through this app's own upgrade:occ upgradealso upgraded every app the app store had a newer release of, and moved the pinned ones.tests/integration/e2e-lifecycle-*.shscripts are removed, since they drove the removed endpoints, and dropped fromrelease-check.sh.e2e-protected-cookie-contract.shno longer cleans up throughpads/trash.restore_pendingor undated rows skipped, the form reading the new key alone, sessions taken after the account is gone;200, notpad_missing);snapshot_rev: 0;200, notpad_missing);200, notpad_missing, and those cases fail;occ user:delete: no live session left; without the listener, the session stays.Numbers and verification
js/is rebuilt with Node 24 and byte-identical with a fresh build.pending_deleterow with its file in the trash (Etherpad stopped during the trash); apending_deleterow whose file had also been deleted from the trash;delete_on_trash = no; atest_faultkey; its three retry jobs; the indexep_bind_state_idx.pending_delete, and the sweep deletes its pad.delete_pad_with_fileisno, stored as a string;delete_on_trash,test_faultand the three jobs are gone,GoneFileSweepJobis registered,ep_bind_state_idxhas given way toep_bind_state_mode_idx, and the upgrade logs no warning..padmissing on disk and dropped byocc files:scankeeps its row and pad; with the scan's exception switched off, the same run marks the row;404;occ upgraderegistersGoneFileSweepJobfrominfo.xml, with its row removed beforehand;sync-app.shruns a new migration of this app alone, and the pinned apps keep their versions;