Skip to content

Add CameraFeed support for Web #106784

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

shiena
Copy link
Contributor

@shiena shiena commented May 24, 2025

fixed: godotengine/godot-proposals#12493

Current Limitation:
The platform/web/js/libs/library_godot_camera.js library includes certain functionalities that are inherently asynchronous. We are currently investigating how to execute these functions synchronously, as this is a requirement for our project, but haven't yet found a solution.
I was able to build with JSPI and call asynchronous functions synchronously in Chrome. However, JSPI only works in Chrome and Edge 137.
https://webassembly.org/features/
https://learn.microsoft.com/en-us/microsoft-edge/web-platform/release-notes/137#javascript-promise-integration-jspi-in-webassembly

@shiena shiena force-pushed the feature/support-web-camera branch 3 times, most recently from 140bdef to f939c20 Compare May 24, 2025 23:16
@shiena
Copy link
Contributor Author

shiena commented May 25, 2025

I've tried enabling Asyncify as follows, which I believe should fix it, but the build is still failing.

diff --git a/platform/web/SCsub b/platform/web/SCsub
index cf872e65ff..5bb105e1cd 100644
--- a/platform/web/SCsub
+++ b/platform/web/SCsub
@@ -106,6 +106,8 @@ else:
     sys_env.Append(LIBS=["idbfs.js"])
     build = sys_env.add_program(build_targets, web_files + ["web_runtime.cpp"])
 
+sys_env.Append(LINKFLAGS=["-sASYNCIFY=1"])
+
 sys_env.Depends(build[0], sys_env["JS_LIBS"])
 sys_env.Depends(build[0], sys_env["JS_PRE"])
 sys_env.Depends(build[0], sys_env["JS_POST"])
cache:INFO: generating system asset: symbol_lists/8eab473934c10451394fc82e8912c6b3712c9ff9.json... (this will be cached in "C:\Users\shien\dev\emsdk\upstream\emscripten\cache\symbol_lists\8eab473934c10451394fc82e8912c6b3712c9ff9.json" for subsequent builds)
cache:INFO:  - ok
unexpected expression type
UNREACHABLE executed at C:\b\s\w\ir\cache\builder\emscripten-releases\binaryen\src\passes\Asyncify.cpp:1142!
em++: error: 'C:/Users/shien/dev/emsdk/upstream\bin\wasm-opt --strip-target-features --post-emscripten -Os --low-memory-unused --asyncify --pass-arg=asyncify-propagate-addlist --pass-arg=asyncify-imports@env.invoke_*,env.__asyncjs__*,*.fd_sync,*.emscripten_promise_await,*.emscripten_idb_load,*.emscripten_idb_store,*.emscripten_idb_delete,*.emscripten_idb_exists,*.emscripten_idb_clear,*.emscripten_idb_load_blob,*.emscripten_idb_store_blob,*.emscripten_sleep,*.emscripten_wget_data,*.emscripten_scan_registers,*.emscripten_lazy_load_code,*._load_secondary_module,*.emscripten_fiber_swap,*.SDL_Delay --zero-filled-memory --pass-arg=directize-initial-contents-immutable --no-stack-ir bin\godot.web.template_debug.wasm32.nothreads.wasm -o bin\godot.web.template_debug.wasm32.nothreads.wasm -g --mvp-features --enable-bulk-memory --enable-exception-handling --enable-multivalue --enable-mutable-globals --enable-reference-types --enable-sign-ext --enable-simd' failed (returned 3221226505)
scons: *** [bin\godot.web.template_debug.wasm32.nothreads.js] Error 1

@Calinou
Copy link
Member

Calinou commented May 25, 2025

Asyncify makes binaries a lot larger, so we need to be able to use it selectively only for functions that need it.

cc @adamscott

@shiena shiena force-pushed the feature/support-web-camera branch 3 times, most recently from a6e548e to 9354b30 Compare May 26, 2025 19:48
@AThousandShips AThousandShips added this to the 4.x milestone May 27, 2025
@shiena shiena force-pushed the feature/support-web-camera branch 5 times, most recently from be89cda to 706e8d0 Compare June 2, 2025 04:48
@shiena shiena marked this pull request as ready for review June 2, 2025 05:19
@shiena shiena requested review from a team as code owners June 2, 2025 05:19
@shiena
Copy link
Contributor Author

shiena commented Jun 2, 2025

Ready for review.
The demo site requires Chrome 137+ due to its use of JSPI.
I wanted to use ASYNCIFY=1 in SCsub but ran into errors, so I'm currently using ASYNCIFY=2 for JSPI.

Demo site: https://shiena.github.io/GodotCameraFeedSample/
Demo source code: https://github.com/shiena/GodotCameraFeedSample

@shiena shiena force-pushed the feature/support-web-camera branch from 706e8d0 to 6854b08 Compare June 2, 2025 05:31
@shiena shiena requested a review from a team as a code owner June 2, 2025 05:31
Copy link
Collaborator

@Faless Faless left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The demo site requires Chrome 137+ due to its use of JSPI.
I wanted to use ASYNCIFY=1 in SCsub but ran into errors, so I'm currently using ASYNCIFY=2 for JSPI.

This means it can't be merged, we at least want support the latest version of Chrome, Firefox and iOS (and we also try to support the latest version of Firefox ESR, but that has been on and off in the past).

That said, it should be possible to implement the API by relying on callbacks instead of requiring asyncify.

Comment on lines +37 to +43
EM_ASYNC_JS(void, godot_js_camera_get_cameras, (void *context, CameraLibrary_OnGetCamerasCallback p_callback_ptr), {
await GodotCamera.api.getCameras(context, p_callback_ptr);
});

EM_ASYNC_JS(void, godot_js_camera_get_capabilities, (void *context, const char *p_device_id_ptr, CameraLibrary_OnGetCapabilitiesCallback p_callback_ptr), {
await GodotCamera.api.getCameraCapabilities(p_device_id_ptr, context, p_callback_ptr);
});
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't use the EM_*_JS functions, as they break dynamic linking.

@shiena
Copy link
Contributor Author

shiena commented Jun 2, 2025

That said, it should be possible to implement the API by relying on callbacks instead of requiring asyncify.

I'm trying to get a list of CameraFeed objects using CameraServer.feeds() after setting CameraServer.monitoring_feeds = true in GDScript. Once I have a CameraFeed, I then want to retrieve its available formats using CameraFeed.formats.

However, both of these methods return a JavaScript Promise, which makes using callbacks inherently slow.
If there's an alternative approach, I'd be happy to try it out immediately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Suppport CameraFeed for Web
4 participants