Skip to content

[WebGPU] Fix OrtReleaseEnv self-deadlock after failed adapter request on Linux#29591

Merged
skottmckay merged 4 commits into
mainfrom
copilot/fix-ortreleaseenv-deadlock
Jul 8, 2026
Merged

[WebGPU] Fix OrtReleaseEnv self-deadlock after failed adapter request on Linux#29591
skottmckay merged 4 commits into
mainfrom
copilot/fix-ortreleaseenv-deadlock

Conversation

Copilot AI commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

On Linux with the WebGPU EP, if SessionOptionsAppendExecutionProvider is called when no Vulkan adapter is available, a subsequent OrtReleaseEnv on the same thread hangs forever in a futex wait.

Root causes

1. C++ exceptions thrown inside Dawn WaitAny callbacks (primary)

ORT_ENFORCE was called directly inside the RequestAdapter/RequestDevice callback lambdas. Dawn's WaitAny does not release its internal EventManager mutex on exception, so throwing through it leaves that mutex permanently locked. The deadlock fires later when Cleanup() calls wgpuInstanceReleaseEventManager::ShutDown() → tries to re-acquire the same mutex on the same thread.

2. Zombie WebGpuContext left in the factory map (secondary)

When Initialize() threw, the WebGpuContext entry remained in contexts_ with ref_count=1 and no owner — a resource leak that would also re-trigger the deadlock path at Cleanup().

Changes (webgpu_context.cc)

  • Adapter callback: replace the throwing lambda with a non-throwing one that writes into a local RequestAdapterResult struct; ORT_ENFORCE is moved to after WaitAny returns:

    // Before — throws inside Dawn's callback dispatch:
    [](wgpu::RequestAdapterStatus status, wgpu::Adapter adapter,
       wgpu::StringView message, wgpu::Adapter* ptr) {
        ORT_ENFORCE(status == wgpu::RequestAdapterStatus::Success, ...);
        *ptr = std::move(adapter);
    }, &adapter
    
    // After — captures result, throws outside Dawn:
    [](wgpu::RequestAdapterStatus status, wgpu::Adapter adapter,
       wgpu::StringView message, RequestAdapterResult* result) {
        result->status = status;
        if (status == wgpu::RequestAdapterStatus::Success)
            result->adapter = std::move(adapter);
        else
            result->message = std::string{message};
    }, &adapter_result
    // ORT_ENFORCE(adapter_result.status == ...) called here, after WaitAny
  • Device callback: same pattern applied to RequestDevice.

  • CreateContext cleanup: wrap Initialize() in a try/catch; on failure decrement ref_count and erase the entry from contexts_ if it reaches zero.

Motivation and Context

Fixes the hang reported when registering the WebGPU EP with no usable Vulkan adapter and then calling OrtReleaseEnv. The process would park on a futex with __owner set to its own TID — a non-recursive mutex locked twice on the same thread — and never exit.

Two root causes addressed:

1. (Primary) ORT_ENFORCE inside Dawn WaitAny callbacks threw C++
   exceptions, which propagated through Dawn code that doesn't release
   its internal mutexes on exception. This left an EventManager mutex
   locked. When OrtEnv::~OrtEnv() later called Cleanup() ->
   wgpuInstanceRelease -> EventManager::ShutDown(), it tried to acquire
   that same mutex -> self-deadlock.

   Fix: capture adapter/device request status in a local struct and
   check it (throwing if needed) AFTER WaitAny returns, so no
   exception ever propagates through Dawn's internal stack.

2. (Secondary) When Initialize() threw, the WebGpuContext remained in
   the contexts_ map with ref_count=1 and no owner. Cleanup() would
   later destroy it, triggering the (now-absent) deadlock path, and
   also leaking the entry permanently.

   Fix: wrap Initialize() in a try/catch in CreateContext(); on failure
   decrement the ref_count and erase the entry if it reaches zero.

Fixes #29553
Copilot AI changed the title [WIP] Fix OrtReleaseEnv deadlock after failed Vulkan adapter request [WebGPU] Fix OrtReleaseEnv self-deadlock after failed adapter request on Linux Jul 7, 2026
Copilot AI requested a review from skottmckay July 7, 2026 05:33
Comment thread onnxruntime/core/providers/webgpu/webgpu_context.cc Outdated
Copilot AI requested a review from skottmckay July 7, 2026 06:22
@skottmckay skottmckay marked this pull request as ready for review July 7, 2026 06:33
@skottmckay skottmckay requested a review from edgchen1 July 7, 2026 06:38
Comment thread onnxruntime/core/providers/webgpu/webgpu_context.cc Outdated
@skottmckay skottmckay enabled auto-merge (squash) July 8, 2026 03:36
@skottmckay skottmckay merged commit ef73996 into main Jul 8, 2026
86 checks passed
@skottmckay skottmckay deleted the copilot/fix-ortreleaseenv-deadlock branch July 8, 2026 03:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[WebGPU] OrtReleaseEnv self-deadlocks after a failed Vulkan adapter request

3 participants