Skip to content

Use IOSurface on macOS/iOS#329

Open
madsmtm wants to merge 2 commits intomasterfrom
madsmtm/iosurface
Open

Use IOSurface on macOS/iOS#329
madsmtm wants to merge 2 commits intomasterfrom
madsmtm/iosurface

Conversation

@madsmtm
Copy link
Member

@madsmtm madsmtm commented Jan 31, 2026

Make Buffer<'_> be backed by a shared memory buffer that can be shared directly with the compositor. This makes the CoreGraphics backend zero-copy (QuartzCore was copying from the CGImageProvider internally before when committing the transaction).

This also implements triple buffering of the surface, to avoid creating a bunch of unnecessary buffers. The compositor seems to want to work on two buffers at the same time? A bit unsure why, but I know that we do get tearing if we try to touch things while it's working on it.

Fixes #83.

This requires #321, because IOSurface does not support RGBX (so the alpha channel must be opaque to render correctly).

Tested on:

  • macOS 15.7.3 M2
  • macOS 10.12.6 Intel
  • iOS simulator

Replaces #95 and #96.

@madsmtm madsmtm added enhancement New feature or request DS - CoreGraphics macOS/iOS/tvOS/watchOS/visionOS backend labels Jan 31, 2026
Comment on lines +301 to +317
// Block until back buffer is no longer being used by the compositor.
//
// TODO: Allow configuring this: https://github.com/rust-windowing/softbuffer/issues/29
// TODO: Is this actually the check we want to do? It seems like the compositor doesn't
// properly set the usage state when the application loses focus, even if you continue
// rendering there?
//
// Should we instead set up a `CVDisplayLink`, and only allow using the back buffer once a
// certain number of frames have passed since it was presented? Would be better though not
// perfect, `CVDisplayLink` isn't guaranteed to actually match the display's refresh rate:
// https://developer.apple.com/library/archive/documentation/GraphicsImaging/Conceptual/CoreVideo/CVProg_Concepts/CVProg_Concepts.html#//apple_ref/doc/uid/TP40001536-CH202-DontLinkElementID_2
//
// Another option would be to keep a boundless queue as described in:
// https://github.com/commercial-emacs/commercial-emacs/blob/68f5a28a316ea0c553d4274ce86e95fc4a5a701a/src/nsterm.m#L10552-L10571
while self.back.surface.is_in_use() {
std::thread::yield_now();
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way we wait for the compositor to stop using the buffer(s) here is kinda bad, but that can be resolved later, and shouldn't be a problem for applications that either:

  1. Only re-render when something changes.
  2. Do proper frame-pacing (Winit doesn't yet provide that though).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I run the animation example, it seems to sometimes work, but sometimes get stuck in this loop after the three buffers have been allocated.

Write directly into a shared memory buffer that can be shared directly
with the compositor and avoids copying bytes when presenting (QuartzCore
was copying stuff before).

This also implements double and triple buffering of the surface, to
avoid creating a bunch of unnecessary buffers. The compositor seems to
sometimes work on two buffers at the same time? A bit unsure why.

The way we wait for the compositor to stop using the buffer(s) is kinda
bad, but that can be resolved later, and shouldn't be a problem for
applications that do proper frame-pacing (which Winit doesn't yet
provide though).
@madsmtm madsmtm requested a review from ids1024 February 5, 2026 23:37
@ids1024
Copy link
Member

ids1024 commented Feb 6, 2026

Hm. When I try this on an M1 Mac Mini running Tahoe 26.2, I'm not seeing it work correctly. Examples create windows with white contents, except it becomes partly visible if I drag another window over it. But that doesn't show up in a screenshot, which just shows the window white.

@madsmtm
Copy link
Member Author

madsmtm commented Feb 6, 2026

It requires #321 (the alpha mode is AlphaMode::Opaque), if you want to test things you can do:

diff --git a/src/pixel.rs b/src/pixel.rs
index bcc223d..eb08f30 100644
--- a/src/pixel.rs
+++ b/src/pixel.rs
@@ -101,7 +101,7 @@ impl Pixel {
     /// ```
     pub const fn new_rgb(r: u8, g: u8, b: u8) -> Self {
         // FIXME(madsmtm): Change alpha to `0xff` once we support transparency.
-        Self { r, g, b, a: 0x00 }
+        Self { r, g, b, a: 0xff }
     }
 
     /// Create a new pixel from a blue, a green and a red component.

@ids1024
Copy link
Member

ids1024 commented Feb 6, 2026

Ah. Is there any reason we have to wait for #321 to change this to 0xff? I guess we specified that the alpha component should be 0 before #289 as well, but there aren't really platforms where that's necessary.

Though hopefully we can get both these PRs merged soon.

@madsmtm
Copy link
Member Author

madsmtm commented Feb 6, 2026

Idk., I just preferred to wait for #321 (and I won't merge this one before that)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DS - CoreGraphics macOS/iOS/tvOS/watchOS/visionOS backend enhancement New feature or request

Development

Successfully merging this pull request may close these issues.

Investigate more optimal way to implement CoreGraphics backend

2 participants