fix(wayland): support DMA-BUF modifiers for wlroots capture#5132
fix(wayland): support DMA-BUF modifiers for wlroots capture#5132joeknock90 wants to merge 2 commits into
Conversation
This comment was marked as off-topic.
This comment was marked as off-topic.
|
This is continuing to work well for me. I'm not sure when I should remove the DRAFT designation. Any input from maintainers? |
If you think it's ready for review or close to being mergable, you can remove draft. Typically I don't really review anything that's in draft unless asked, just because there is so much to review already. And with that said, I don't really use Linux myself so would appreciate anyone of the other contributors also reviewing and/or testing the changes. |
Considering I relied heavily on Gemini to create this that would certainly make me much more comfortable. I'll take it out of draft though. I'm happy with the results personally, and if/when HDR support gets merged for Sway it would be nice to have this working as it relies on the sway's vulkan renderer. |
Bundle ReportBundle size has no change ✅ |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #5132 +/- ##
=========================================
Coverage ? 16.95%
=========================================
Files ? 111
Lines ? 24163
Branches ? 10698
=========================================
Hits ? 4097
Misses ? 16890
Partials ? 3176
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report in Codecov by Sentry.
|
|
Can you provide more precise instructions to test this PR? For me build from pipeline doesn't work at all on headless sway instance on Arch. Both with and without Vulkan. Upstream builds work without Vulkan, but don't work with it. Also it might be more beneficial to revive this PR: #4788 which implements more modern capture method. |
|
For my testing I cloned the pacman-repo repository and edited the PKGBUILD to use my repo and branch, then installed with makepkg and pacman. I've been testing on headless sway as well, using the WLR_RENDERER=vulkan variable. I verified that the env variable is set in the headless sway session from the terminal. |
Go here -> https://github.com/LizardByte/Sunshine/actions/runs/26135435168?pr=5132#artifacts Download and install the artifact for your system that starts with |
|
Yes, I've installed build from PR pipelines, but it doesn't work both with and without Vulkan. I've attached the log. Is there any reason to try to build it from PKGBUILD of cloned repo? There might be additional problems, because I'm running it inside of Proxmox container, but this setup works without Vulkan. |
|
I've tested the build from the PR pipeline (Archlinux + wlroots-git commit 7265a79e + sway-git commit e51f9d7) for about an hour with WLR_RENDERER=vulkan. Notably HDR still doesn't work, I think it's not yet supported by the wlroots headless backend based on this log from sway: |
I didn't attempt HDR support in this PR. I'm not entirely sure it's possible in sway yet? I think sway-git might have added some support but I'm not sure It would work with the screen capture method being used. Someone can correct me if I'm wrong there. |
Sway 1.12 just released with HDR support, but yeah, I don't think it's possible with the wlroots headless backend (the log I posted is just from Sway, before even starting Sunshine). I just mentioned it in case people were following this PR for this reason, I doubt anything can be done about it until wlroots supports it. |
|
@joeknock90 could you fix the lint errors? https://github.com/LizardByte/Sunshine/actions/runs/26135435165/job/77045005439?pr=5132 |
|
I believe that clang-format was just looking for a space? Pushed and rebased. |
ReenigneArcher
added
the
ai
Explicitly query supported DMA-BUF modifiers from the compositor and use them when creating GBM buffers. This fixes capture failures on Sway when using the Vulkan renderer, which rejects the implicit INVALID modifier.
ReenigneArcher
force-pushed
the
fix_wlroots_vulkan
branch
from
52ce706 to
f55f5a0
Compare
|
There was a problem hiding this comment.
Pull request overview
Fixes wlroots screencapture failure under the Vulkan renderer (Sway) by querying DMA-BUF format/modifier advertisements from zwp_linux_dmabuf_v1 and using them with gbm_bo_create_with_modifiers, so capture buffers no longer default to the implicit INVALID modifier that Vulkan rejects.
Changes:
- Add
format/modifierlistener tozwp_linux_dmabuf_v1and accumulate per-format modifier lists oninterface_t(binding capped at version 3 to ensure those events are delivered). - Plumb the collected
supported_modifiersmap throughdmabuf_t::listenintocreate_and_copy_dmabuf. - Prefer
gbm_bo_create_with_modifiersusing the advertised modifiers, falling back to the existinggbm_bo_createpath on failure.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/platform/linux/wayland.h | Adds includes, declares dmabuf listener callbacks, and stores supported_modifiers + listener on interface_t/dmabuf_t. |
| src/platform/linux/wayland.cpp | Implements format/modifier callbacks, binds dmabuf at v3, and uses modifier list in GBM buffer creation with fallback. |
| src/platform/linux/wlgrab.cpp | Passes interface.supported_modifiers through to dmabuf.listen. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if (supported_modifiers) { | ||
| auto it = supported_modifiers->find(dmabuf_info.format); | ||
| if (it != supported_modifiers->end() && !it->second.empty()) { | ||
| current_bo = gbm_bo_create_with_modifiers(gbm_device, dmabuf_info.width, dmabuf_info.height, dmabuf_info.format, it->second.data(), it->second.size()); | ||
| } | ||
| } |
| dmabuf_interface = (zwp_linux_dmabuf_v1 *) wl_registry_bind(registry, id, &zwp_linux_dmabuf_v1_interface, std::min(version, 3u)); | ||
| zwp_linux_dmabuf_v1_add_listener(dmabuf_interface, &dmabuf_listener, this); |
Explicitly query supported DMA-BUF modifiers from the compositor and use them when creating GBM buffers. This fixes capture failures on Sway when using the Vulkan renderer, which rejects the implicit INVALID modifier.
Explicitly query supported DMA-BUF modifiers from the compositor and use them when creating GBM buffers. This fixes capture failures on Sway when using the Vulkan renderer, which rejects the implicit INVALID modifier.
DISCLAIMER: I am not a developer, programmer, or really any sort of smart person at all. I punched this issue into Gemini-CLI. I manually approved all of he changes it made with very little understanding of them, but did learn a little bit. Never thought I'd be a vibe coder 🤮
For full transparency, here is the prompt I used:
I am currently testing my branch and it seems to be working well for me so far.
Screenshot
N/A
Issues Fixed or Closed
Fixes #4050
Roadmap Issues
N/A
Type of Change
Checklist
AI Usage