Problems and possible solutions for using wizard on Windows

[yezhengkai]

Problems and possible solutions

When I run BinaryBuilder.run_wizard() on Windows, I encounter the problems described in the following subsections. In each subsection, I also describe possible solutions so that I can generate build_tarballs.jl using BinaryBuilder.run_wizard().

symlink

symlink requires administrator privileges on Windows, so I just opened a shell with administrator privileges to solve the problem.
But comment in the symlink function states that "creating file/directory symlinks requires Administrator privileges while junction points apparently do not", so using junction points might be another way to run programs as a non-administrator.

tar for importing docker image

This problem is described in detail in #520. Here I use wsl tar to solve the problem.

(original import_docker_image)

function import_docker_image(......)
    ......
    try
        if Sys.iswindows()
            run(pipeline(pipeline(
                `wsl tar -c -C \$\(wslpath \'$(rootfs_path)\'\) .`,
                `docker import - -c $(dockerfile_cmds) $(docker_image(rootfs))`;
            ); stdout=devnull))
        else
            run(pipeline(pipeline(
                `tar -c -C $(rootfs_path) .`,
                `docker import - -c $(dockerfile_cmds) $(docker_image(rootfs))`;
            ); stdout=devnull))
        end
    finally
        unmount(rootfs, workspace_root)
    end
    return
end

listen function

According to the examples in official doc and the discussion in the github issue, when using the listen function on windows, we may need to add prefix (\\.\pipe\), as shown in the following code snippet.

(original setup_bb_service)

function setup_bb_service(......)
    fpath = joinpath(prefix, "metadir", "bb_service")

    server = Sys.iswindows() ? listen("\\\\.\\pipe\\$(fpath)") : listen(fpath)
    ......
end

Docker mount volume

There was a problem with the path separator for initial working directory and read-only volumes in the container, which resulted in incorrect directory names in the container. To solve this problem I replaced separator from \\ to /, as shown in the following code snippet.

(original DockerRunner)

function DockerRunner(......)
    ......

    if cwd !== nothing
        docker_cmd = `$docker_cmd -w /$(replace(abspath(cwd), "\\" => "/"))`
    end

    # Add in read-only mappings and read-write workspaces
    for shard in shards[2:end]
        outside = mount_path(shard, workspace_root)
        inside = replace(map_target(shard), "\\" => "/")
        docker_cmd = `$docker_cmd -v $(outside):$(inside):ro`
    end
    ......
end

Environment info

julia versioninfo

julia> versioninfo()
Julia Version 1.7.3
Commit 742b9abb4d (2022-05-06 12:58 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-12.0.1 (ORCJIT, skylake)

julia package status

(@v1.7) pkg> st
      Status `C:\Users\kai\.julia\environments\v1.7\Project.toml`
  [6e4b80f9] BenchmarkTools v1.3.2
  [12aac903] BinaryBuilder v0.5.6
  [5fb14364] OhMyREPL v0.5.12
  [65465c31] PackageCompatUI v1.0.2
  [14b8a8f1] PkgTemplates v0.7.29
  [295af30f] Revise v3.4.0
  [22787eb5] Term v1.0.4

docker info

$ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.9.1)
  compose: Docker Compose (Docker Inc., v2.10.2)
  extension: Manages Docker extensions (Docker Inc., v0.2.9)
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
  scan: Docker Scan (Docker Inc., v0.19.0)

Server:
 Containers: 2
  Running: 0
  Paused: 0
  Stopped: 2
 Images: 7
 Server Version: 20.10.17
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.102.1-microsoft-standard-WSL2
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 12.36GiB
 Name: docker-desktop
 ID: 2WFH:QGDR:WPRJ:AT3D:25UV:XSWD:LKXK:NXVR:U6FC:SHSQ:WKA5:NR5O
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5000
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support

[cserteGT3]

With the above DockerRunner change I still hit the docker_entrypoint.sh issue:

docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "/docker_entrypoint.sh": stat /docker_entrypoint.sh: no such file or directory: unknown.

My system is Windows 11, with julia v1.9.2 and current BinaryBuilder and BinaryBuilderBase master. I'm trying to build the Arb, so if it works, I can move on to MuJoCo. I'm happy to provide more details, but wanted to ask if there's any suggestions first.