docs: Binary image segmentation example #1795
Conversation
|
pkg.pr.new packages benchmark commit |
…mentation-example
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 12 out of 15 changed files in this pull request and generated no new comments.
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 12 out of 15 changed files in this pull request and generated 3 comments.
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
apps/typegpu-docs/src/examples/image-processing/background-segmentation/shaders.ts
Show resolved
Hide resolved
apps/typegpu-docs/src/examples/image-processing/background-segmentation/shaders.ts
Show resolved
Hide resolved
apps/typegpu-docs/src/examples/image-processing/background-segmentation/shaders.ts
Show resolved
Hide resolved
aleksanderkatan
requested review from
cieplypolar,
iwoplaza,
lursz and
reczkok
apps/typegpu-docs/src/examples/image-processing/background-segmentation/index.html
Outdated
Show resolved
Hide resolved
apps/typegpu-docs/src/examples/image-processing/background-segmentation/index.html
Outdated
Show resolved
Hide resolved
apps/typegpu-docs/src/examples/image-processing/background-segmentation/index.ts
Outdated
Show resolved
Hide resolved
| } | ||
|
|
||
| // monkey patching ONNX: https://github.com/microsoft/onnxruntime/issues/26107 | ||
| const oldRequestAdapter = navigator.gpu.requestAdapter; |
There was a problem hiding this comment.
Should we wrap it in try block?
| }); | ||
|
|
||
| const feeds = { 'input.1': myPreAllocatedInputTensor }; | ||
| const fetches = { '1959': myPreAllocatedOutputTensor }; |
There was a problem hiding this comment.
idk
the outputs are named 1959, 1960..., this was the first one.
| "monaco-editor": "^0.53.0", | ||
| "morphcharts": "^1.3.2", | ||
| "motion": "^12.23.24", | ||
| "onnxruntime-web": "1.23.0-dev.20250917-21fbad8a65", |
There was a problem hiding this comment.
Bump reminder (when patched)
| }; | ||
|
|
||
| const tileData = tgpu.workgroupVar(d.arrayOf(d.arrayOf(d.vec3f, 128), 4)); | ||
| export const computeFn = tgpu['~unstable'].computeFn({ |
There was a problem hiding this comment.
On my mobile device (Android 16, Snapdragon 888, any browser), masking doesn't quite work
There was a problem hiding this comment.
I tried to find some reason behind this, and I can't find anything. The resolution is not a problem, rewriting the video frame to a new texture also didn't help. The mask just is like this on your phone. This might be an onnx web runner issue.
|
Turns out Chrome has a built-in feature for background blur! |
Changes:
camera-tresholdingto unstable API (the file looked almost 1:1 likechroma-keyingso I guess why not differentiate it a bit),background-segmentationexample, which uses ONNX runtime with unet2p model. It copies the camera fee, blurs it and selects either blurred or unblurred pixels based on the mask generated by the model.ONNX has some issues with providing it with a device (microsoft/onnxruntime#26107). I monkey-patched some requests, and I clean this up in
cleanUp.ONNX with WebGPU does not yet work on Safari (microsoft/onnxruntime#26480), so I just throw an error.
On my phone, it works in ~1fps and is unusable, but on one of the shared devices I borrowed (Samsung Galaxy X Fold 5) it worked perfectly fine.