feat(ipc/sem): add semaphore support for dragonOS - #2172
feat(ipc/sem): add semaphore support for dragonOS#2172mistcoversmyeyes wants to merge 30 commits into
Conversation
e0bc761 to
a266b20
Compare
d9c382f to
fbdee92
Compare
|
@codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fbdee927ce
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
fslongjin
left a comment
There was a problem hiding this comment.
Request changes: this PR establishes a useful base for System V semaphore support, but it does not yet satisfy the Linux 6.6 compatibility and concurrency-safety contract stated in #2142.
The blocking issues are:
- IPC_SET permission updates can partially commit on an error, changing the owner even though the syscall returns EINVAL; the shared helper also affects SHM.
- SEM_UNDO is rejected with ENOSYS and the new test codifies that incompatibility, while Linux maintains per-process/shared undo state and replays it at process exit.
- A single namespace-wide spinlock protects the registry and every semaphore set, so unrelated sets are serialized; the lock also covers allocation-heavy queue simulation and scheduler wakeups.
- User-controlled semaphore-set allocation is infallible and can reach the kernel panic allocation handler instead of returning ENOMEM.
- SEM_STAT and SEM_STAT_ANY mask their direct table index, causing out-of-range indices to alias valid objects.
The basic syscall wiring, atomic multi-operation simulation, timeout/removal paths, and test breadth are valuable. However, the issues above are architectural or user-visible Linux semantic mismatches rather than optional refinements. Please address them, add the corresponding regression tests, and rerun the guest suite. The current Integration Test check also reports 5666 passed, 1 failed, and 180 skipped; I am not attributing that failure to this PR without further evidence, but the PR description should not claim that Integration Test passed while the check remains red.
| .iter() | ||
| .any(|op| (op.sem_flg as u32) & SemFlags::SEM_UNDO.bits() != 0) | ||
| { | ||
| return Err(SystemError::ENOSYS); |
There was a problem hiding this comment.
[P1] Rejecting every SEM_UNDO operation with ENOSYS is not Linux-compatible System V semaphore behavior. Linux 6.6 maintains sem_undo/semadj state, shares the undo list for CLONE_SYSVSEM, clears adjustments on SETVAL/SETALL/IPC_RMID, and replays them from exit_sem() when a task exits. This is essential crash-recovery behavior: without it, a lock holder exiting can leave peers blocked indefinitely. Please implement the full lifecycle before treating #2142 as complete; the new test should verify Linux behavior instead of expecting ENOSYS.
There was a problem hiding this comment.
Confirmed. Full SEM_UNDO lifecycle support is required for Linux 6.6 compatibility, including shared undo state for CLONE_SYSVSEM, semadj updates and limits, cleanup on SETVAL/SETALL/IPC_RMID, and replay on task exit. I am implementing this now and will replace the current ENOSYS test with lifecycle coverage. Keeping this thread open until the implementation and tests are complete.
| /// SysV SHM manager (phase one: per-namespace SHM only) | ||
| pub shm: SpinLock<ShmManager>, | ||
| /// SysV semaphore manager | ||
| pub sem: SpinLock<SemManager>, |
There was a problem hiding this comment.
[P1] A namespace-wide spinlock is too broad for semaphore-set state. Every operation on every set, including update_queue(), is serialized here; queue simulation allocates a HashMap, may rescan waiters quadratically, and calls Waker::wake() while this lock is held. A user can therefore stall unrelated semaphore sets in the same namespace. Please keep the manager lock limited to ID/key/quota lookup, store stable Arc<KernelSemSet> objects with per-set locking, use a non-allocating operation fast path, and collect wakeups for execution after releasing the set lock, following Linux's registry/array locking and wake_q separation.
There was a problem hiding this comment.
Please keep the manager lock limited to ID/key/quota lookup, store stable
Arc<KernelSemSet>objects with per-set locking, use a non-allocating operation fast path, and collect wakeups for execution after releasing the set lock.
I am currently evaluating the necessity and designing the manager/per-set lock separation. No implementation changes for this lock split have started yet.
I am also coordinating this with the latest semaphore queue work. The design will preserve the unlocked queue-growth preparation and fallible-allocation semantics from b3f637c7, rather than duplicate or replace that work. I will post the finalized scope before touching the locking code so we can avoid overlapping changes.
|
While reviewing the queue-rescan concern, I found a separate Linux 6.6 compatibility issue in the current unified waiter queue. Linux classifies queued requests into const operations (the entire operation group has no altering operation) and alter operations when they are enqueued: ipc/sem.c#L2127-L2153. Its update path handles wait-for-zero requests before scanning pending alter requests: ipc/sem.c#L1029-L1039. After a queued alter operation succeeds, Linux immediately checks wait-for-zero requests made satisfiable by that change before continuing with further alter operations: ipc/sem.c#L974-L992. The current DragonOS implementation stores both classes in one Proposed minimal design for this PR:
This fixes the observable ordering difference without attempting to eliminate the irreducible worst-case rescan for complex multi-operation groups. |
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a2d659b25f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| let (waiter, waker) = Waiter::new_pair(); | ||
| let timer = | ||
| deadline_ticks.map(|deadline| Timer::new(TimeoutWaker::new(waker.clone()), deadline)); |
There was a problem hiding this comment.
当 semop 可立即完成时,这里仍会无条件为 Waker 分配 Arc;带非零超时的 semtimedop 还会额外分配 TimeoutWaker 和 Timer。这使无竞争的信号量快路径以及无效 ID、越界或权限失败等错误路径都承担不必要的堆分配开销;应仅在 simulate_semop 返回 Blocked 且确实需要睡眠后创建这些对象。
AGENTS.md reference: AGENTS.md:L10-L10
Useful? React with 👍 / 👎.
|
There are currently conflicts. Please rebase your code onto the latest |
Signed-off-by: longjin <longjin@dragonos.org>
cfa9335 to
73467d1
Compare
Signed-off-by: longjin <longjin@dragonos.org>
|
@codex review |
Signed-off-by: longjin <longjin@dragonos.org>
…rs directly Signed-off-by: longjin <longjin@dragonos.org>
Signed-off-by: longjin <longjin@dragonos.org>
|
Pushed three signed-off fixes: 8d1e231 (fallible namespace fs copy), 2c64d68 (unlocked semaphore storage preparation and direct FIFO unlink), and b2717d7 (the CI failure's root cause). The failing Dunitest run 33979086990 reported a 5000 ms timeout in PtyExecDirectUnameEmitsOutputAndExits although the test took only 9 ms and captured the complete uname output. CollectFdUntilChildExit counted every ready poll as 10 ms. Persistent PTY HUP therefore exhausted an imaginary timeout before the child became waitable. Linux and DragonOS both close files before publishing final exit status; HUP is not proof that waitpid must already succeed. The collector now uses a real monotonic deadline and stops polling a drained, closed output descriptor while continuing bounded waits for process exit. No timeout increase, kernel HUP workaround, or test skip was introduced. Deterministic pipe and PTY regressions (close output, remain alive for 200 ms, then exit normally) both failed immediately with the old helper on Linux; they pass after the fix. Validation: make kernel, formatting and diff checks pass; QEMU semaphore suite 74/74 and PTY suite 29/29, plus 20 repetitions of each (2,060 passing test executions). Host checks cover allocation failure and the actual queue/table methods; these are structural checks, not guest-wide OOM injection. Three-role adversarial review found no actionable new defect in this delta. The three new review threads have English replies and are resolved. Earlier unrelated unresolved discussions remain open. Only code and regression tests were pushed. New CI and automated review results are pending. |
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b2717d700f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: longjin <longjin@dragonos.org>
|
Pushed signed-off commit dabd730 addressing the two new SEM_UNDO review threads. Existing operations now borrow live debt and commit only affected slots; final-owner replay releases the namespace lock between sets while keeping unprocessed debt visible to semctl. Both threads have detailed English replies and are resolved. Validation: make kernel and formatting/diff checks pass; the full semaphore dunitest suite passes 76/76 in QEMU plus 20 complete repetitions (1,520 executions). New semantic regressions also pass on the pre-fix guest and Linux host. Actual-source structural checks separately demonstrate removal of the full-size Existing allocation and preservation of pending replay visibility. Three-role adversarial review passed. Only code and tests were pushed; new CI and automated review are pending. |
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dabd730798
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: longjin <longjin@dragonos.org>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 277351ed23
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: longjin <longjin@dragonos.org>
|
@codex review |
|
Codex Review: Didn't find any major issues. Hooray! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Signed-off-by: longjin <longjin@dragonos.org>
Split semaphore ABI, namespace management, atomic execution and wait queues into focused modules. Centralize terminal publication without rescanning known queues, and represent undo retirement with an explicit phase. Preserve syscall paths, locking and deferred reclamation. Move existing tests with their owning modules and add atomic-attempt regression coverage. Signed-off-by: longjin <longjin@dragonos.org>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 303064c738
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: longjin <longjin@dragonos.org>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a455a7301a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: longjin <longjin@dragonos.org>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 55a48ef282
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Preserve the old IPC namespace and actor across namespace publication, then release the fs reference guard before replay. Validate copied SETALL values before checking for concurrent removal. Signed-off-by: longjin <longjin@dragonos.org>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 506bf42a68
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| match outcome? { | ||
| None => { | ||
| drop(record_slot); | ||
| set.update_queue(&mut wakes); |
There was a problem hiding this comment.
当集合已积累大量阻塞者,而调用者反复执行可立即成功的全零等待或最终值不变的操作时,这里仍会在持有 namespace 级 ipcns.sem 锁期间调用 update_queue(),逐项扫描两个无界等待队列;SEM_UNDO 分支还丢弃了 SemAttempt::Completed 中已有的 values_changed 标志,非 SEM_UNDO 分支也未使用该标志。由于原子操作未改变任何 semval,现有等待者不可能因此变为可执行,应仅在 values_changed 为真时扫描队列,避免普通用户将无效果的成功调用放大为 O(W) 的全局锁内工作。
AGENTS.md reference: AGENTS.md:L10-L10
Useful? React with 👍 / 👎.
Related
Summary
semget,semctl,semop, andsemtimedop.Scope
SEM_UNDOis out of scope and currently returnsENOSYS.Acceptance
ENOSYS.semopandsemtimedopshare consistent operation semantics.IPC_RMID.Testing