Complete port file

[ypujante]

Hi @kainino0x

I saw your response about "misusing" the port system for your needs at the moment. I understand that this is experimental. I also want to help because I think using webgpu as a proper port is the right way to go (down the line).

Here is the port that I wrote (and you are more than welcome to reuse if you so wish):

# note that the file must be called emdawnwebgpu.py (<port_name>.py)
import os
from typing import Union, Dict, Optional

TAG = 'v2025.04.12-03.06.48'
HASH = 'd5627fb822317a8573ad133100a15917517ad44ca39c6927b4dd37b3a3476082a077f8474dcde348bb42a4058c92b9c25beeffb5bd71b83c44df93efea48c1cf'
ZIP_URL = f'https://github.com/kainino0x/dawn/releases/download/{TAG}/emdawnwebgpu_pkg-{TAG}.zip'

# contrib port information (required)
URL = 'https://dawn.googlesource.com/dawn'
DESCRIPTION = 'Dawn is an open-source and cross-platform implementation of the WebGPU standard'
LICENSE = 'BSD 3-Clause License'

VALID_OPTION_VALUES = {
  'enableCPPBindings': ['true', 'false'],
  'optimizationLevel': ['0', '1', '2', '3', 'g', 's', 'z']  # all -OX possibilities
}

OPTIONS = {
  'enableCPPBindings': 'A boolean to enable CPP bindings (disabled by default)',
  'optimizationLevel': f'Optimization level: {VALID_OPTION_VALUES["optimizationLevel"]} (default to 2)',
}

# user options (from --use-port)
opts: Dict[str, Union[Optional[str], bool]] = {
  'enableCPPBindings': False,
  'optimizationLevel': '2'
}

port_name = 'emdawnwebgpu'


def get_lib_name(settings):
  return f'lib_{port_name}_{TAG}-O{opts["optimizationLevel"]}.a'


def get_root_path(ports):
  return os.path.join(ports.get_dir(), port_name, 'emdawnwebgpu_pkg')


def get_include_path(ports):
  return os.path.join(get_root_path(ports), 'webgpu', 'include')


def get_cpp_include_path(ports):
  return os.path.join(get_root_path(ports), 'webgpu_cpp', 'include')


def get_source_path(ports):
  return os.path.join(get_root_path(ports), 'webgpu', 'src')


def get(ports, settings, shared):
  # get the port
  ports.fetch_project(port_name, ZIP_URL, sha512hash=HASH)

  def create(final):
    source_path = get_source_path(ports)
    include_path = get_include_path(ports)

    includes = [include_path]
    srcs = ['webgpu.cpp']
    flags = ['-std=c++17', '-fno-exceptions']
    flags.append(f'-O{opts["optimizationLevel"]}')

    ports.build_port(source_path, final, port_name, includes=includes, srcs=srcs, flags=flags)

  lib = shared.cache.get_lib(get_lib_name(settings), create, what='port')
  if os.path.getmtime(lib) < os.path.getmtime(__file__):
    clear(ports, settings, shared)
    lib = shared.cache.get_lib(get_lib_name(settings), create, what='port')
  return [lib]


def clear(ports, settings, shared):
  shared.cache.erase_lib(get_lib_name(settings))


def linker_setup(ports, settings):
  if settings.USE_WEBGPU:
    raise Exception('dawn may not be used with -sUSE_WEBGPU=1')

  src_dir = get_source_path(ports)

  settings.JS_LIBRARIES += [
    os.path.join(src_dir, 'library_webgpu_enum_tables.js'),
    os.path.join(src_dir, 'library_webgpu_generated_struct_info.js'),
    os.path.join(src_dir, 'library_webgpu_generated_sig_info.js'),
    os.path.join(src_dir, 'library_webgpu.js'),
  ]
  # TODO(crbug.com/371024051): Emscripten needs a way for us to pass
  # --closure-args too.


def process_args(ports):
  # It's important that these take precedent over Emscripten's builtin
  # system/include/, which also currently as webgpu headers.
  args = ['-isystem', get_include_path(ports)]
  if opts['enableCPPBindings']:
    args += ['-isystem', get_cpp_include_path(ports)]
  return args


def check_option(option, value, error_handler):
  if value not in VALID_OPTION_VALUES[option]:
    error_handler(f'[{option}] can be {list(VALID_OPTION_VALUES[option])}, got [{value}]')
  if isinstance(opts[option], bool):
    value = value == 'true'
  return value


def check_required_option(option, value, error_handler):
  if opts[option] is not None and opts[option] != value:
    error_handler(f'[{option}] is already set with incompatible value [{opts[option]}]')
  return check_option(option, value, error_handler)


def handle_options(options, error_handler):
  for option, value in options.items():
    value = value.lower()
    opts[option] = check_option(option, value, error_handler)


if __name__ == "__main__":
  print(f'''# To compute checksums run this
curl -sfL {ZIP_URL} | shasum -a 512
''')

You use it this way (with the main.cpp file from this repository)

> mkdir /tmp/dawn
> emcc -sASYNCIFY=1 -sASYNCIFY_STACK_SIZE=65536 -sEXPORTED_RUNTIME_METHODS=ccall --use-port=emdawnwebgpu.py:enableCPPBindings=true main_dawn.cpp -o /tmp/dawn/index.html

A few points:

• by using port options, you do not need 2 separate ports: enabling cpp bindings is an option (and it is false by default)
• the port file downloads the zip file and does what needs to be done => from a user point of view you only need the port file (not the zip file)
• this means that the port file is outside the zip file (and could be also an artifact generated part of your CI)
• I can host this port file in my emscripten-ports project for the time being for people who just wants to use the port without having to download the zip file
• It is my strong opinion that this port file should become a contrib port in Emscripten once -sUSE_WEBGPU is deprecated so that people can simply install Emscripten and do emcc --use-port=contrib.dawn. Updating the port file on Emscripten usually becomes simply updating the version/hash and that would be fine to do when Dawn has "stable" releases

[ypujante]

As a a side note, I understand that the issue with the port as written, is that we cannot use --closure-args (which, unless I am wrong, is an optimization, so not really a showstopper for this experiment).

I opened an issue to add it to the port system in Emscripten.

[kainino0x]

Thanks for these notes. It's going to be a while before we're ready to turn this into a real port, if we do at all. However I will need to pick up some of the things you did here, especially the optimization flag, and perhaps also

Are optimization levels passed in automatically? Or does the user have to do that? It occurs to me it would be nice to support -g too. Is there any way to inherit those from the linker settings?

Thank you for filing the bug for --closure-args!

[ypujante]

Are optimization levels passed in automatically? Or does the user have to do that? It occurs to me it would be nice to support -g too. Is there any way to inherit those from the linker settings?

No they are not, and this is why I make it an option of the port. If not provided at all (like in your port for example) then -O2 is the default. That is why I introduced it as an option.

I usually do this in my CMakeLists.txt that way it is automatic

set(PORTS_OPTIMIZATION_LEVEL "$<IF:$<CONFIG:Debug>,g,2>")
set(port_options "--use-port=${PROJECT_SOURCE_DIR}/../../ports/Dawn/dawn.py:optimizationLevel=${PORTS_OPTIMIZATION_LEVEL}:enableCPPBindings=true")

Thank you for filing the bug for --closure-args!

No problem. I genuinely want to help.

[kainino0x]

Are optimization levels passed in automatically? Or does the user have to do that? It occurs to me it would be nice to support -g too. Is there any way to inherit those from the linker settings?

No they are not, and this is why I make it an option of the port. If not provided at all (like in your port for example) then -O2 is the default. That is why I introduced it as an option.

I came up with a hack for this. What's your opinion?
https://dawn-review.googlesource.com/c/dawn/+/228417/25/src/emdawnwebgpu/pkg/emdawnwebgpu.port.py#103

# Derive the compile flags for webgpu.cpp based on the linker flags of the
# invocation that called --use-port.
def _compute_flags(settings):
    debug = f'-g{settings.DEBUG_LEVEL}'

    if settings.SHRINK_LEVEL == 2:
        opt = '-Oz'
    elif settings.SHRINK_LEVEL == 1:
        opt = '-Os'
    else:
        opt = f'-O{settings.OPT_LEVEL}'

    #if settings.LTO == 'full':
    #    lto = ['-flto=full']
    #elif settings.LTO == 'thin':
    #    lto = ['-flto=thin']
    #else:
    #    lto = []

    return [debug, opt] #+ lto

EDIT: realized the LTO flags aren't needed, Emscripten seems to do that for us. But deducing -O and -g is still nice.

[ypujante]

I don't know the internals of Emscripten, so I do not know if this "hack" will work in all cases. Maybe @sbc100 might be able to have an opinion.

That being said, even if the "hack" is actually solid code, I do prefer the fact that it is an explicit option with the port so that compiling the port in Debug mode is an option. For example, when I am working on my WebGPUShaderToy project, which uses WebGPU under the cover, I want my code to be compiled in Debug mode, but not the WebGPU port (or other ports like ImGui and GLFW) as it makes the code bigger and slower overall. If I bump into an issue and I feel like it might be a problem with Dawn or I just want to understand better what is happening, I can then turn on debugging for Dawn.

[kainino0x]

Good points. A manual override would make sense.

[ypujante]

You will probably see it in the other thread but I wanted to let you know that I have now released a complete version of the port including support for CLOSURE_ARGS and adding tag (and hash) as options to the port, so that it is easy to change the version without editing the file (although editing the file is not hard...)

The project is emscripten-ports
The port file is here

I also wanted to direct your attention to the example I wrote (inspired by this demo), in particular the main loop. I noticed that in your demo, you have a complex piece of logic (involving JavaScript to invoke the frame API). I really don't think it is necessary (as demonstrated by my example): since you are using ASINCIFY you can simply loop normally, but just making sure you call emscripten_sleep on each iteration to give control back to JavaScript.

[kainino0x]

Thanks!

I am probably going to try to make this demo work without Asyncify at some point, so I didn't want to specialize too much. It's a fair point that if you have asyncify then you don't technically need that hack to make asyncify work, but I think it's something people are going to want to do. But before I do that I need to extract a bunch of the random tests from here and bring them into Dawn's testing, so I haven't yet.

emscripten_sleep is not the proper way to run a main loop though, we should always use requestAnimationFrame to (1) match the framerate of the browser, (2) stop rendering when the page isn't visible, and (3) have the browser schedule work just before presenting. It IS possible to write that kind of main loop that's driven by requestAnimationFrame but it is not as simple.

[ypujante]

In regards to ASYNCIFY, I also personally don't like to use it and prefer the emscripten_set_main_loop approach. That being said, due to the change in APIs (from -sUSE_WEBGPU=1) and the fact that your example rely on Instance::WaitAny call, it was a bit hard for me figure out how to make it work without it. Now that I have something in place, I can spend some more time trying to figure it out (the important part of this effort was the port, not the example).

Thank you for the heads up about emscripten_sleep. Like I just said, in my other projects I have used the emscripten_set_main_loop approach. I don't think there is a way to invoke requestAnimationFrame from C (would have to use JavaScript) I think.