Skip to content

Conversation

@qjia7
Copy link
Contributor

@qjia7 qjia7 commented Oct 30, 2025

This pull request updates the execution provider registration logic in inference_session.cc to improve support for the WebGPU execution provider, specifically for the CopyTensors C API. The main change ensures that the WebGPU data transfer is registered not only with the session's data transfer manager but also with the environment's data transfer manager, enabling broader compatibility.

@qjia7 qjia7 force-pushed the data_transfer_mgr branch from 04434ea to 47b3a2d Compare November 11, 2025 07:39
@qjia7 qjia7 marked this pull request as ready for review November 12, 2025 10:12
Copy link
Contributor

@fs-eire fs-eire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Environment::data_transfer_mgr_ is different from InferenceSession::data_transfer_mgr_. They are the same type, but the one inside Environment should not depend on any session instance. The changes in this PR brings dependency on a specific session for Environment and I believe this is not what we want.

There is an existing method CreateAndRegisterInternalEps in class Environment, which should have already called RegisterExecutionProviderLibrary on WebGPU. Why the data transfer is not correctly registered - I can take a look at it.

@fs-eire
Copy link
Contributor

fs-eire commented Nov 13, 2025

Please add override CreateDataTransfer:

virtual OrtStatus* CreateDataTransfer(_Outptr_result_maybenull_ OrtDataTransferImpl** data_transfer) noexcept {
*data_transfer = nullptr;
return nullptr; // Default implementation does nothing
}

in class WebGpuEpFactory (file in https://github.com/Microsoft/onnxruntime/blob/main/onnxruntime/core/session/plugin_ep/ep_factory_webgpu.h) for the purpose of this change.

@qjia7
Copy link
Contributor Author

qjia7 commented Nov 13, 2025

Please add override CreateDataTransfer:

virtual OrtStatus* CreateDataTransfer(_Outptr_result_maybenull_ OrtDataTransferImpl** data_transfer) noexcept {
*data_transfer = nullptr;
return nullptr; // Default implementation does nothing
}

in class WebGpuEpFactory (file in https://github.com/Microsoft/onnxruntime/blob/main/onnxruntime/core/session/plugin_ep/ep_factory_webgpu.h) for the purpose of this change.

WebGPU DataTransfer requires a BufferManager.
For graph capture, the BufferManager is tied to the execution provider instance not session-independent. That's the problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants