Update Docker-Moby Client and Server to support zstd

[TikiTDO]

Tool name

Docker-Moby

Tool license

Apache

Add or update?

• Add
• Update

Desired version

21+

Approximate size

No response

Brief description of tool

This PR tagged for 21.x adds support for zstd images. Currently you can not setup workflows to use container images compressed with zstd. Havng this would be very desirable as this can greatly reduce time necessary to pull large images.

I'm not sure why docker is being held back at 20, but it would be very nice to have it updated.

URL for tool's homepage

No response

Provide a basic test case to validate the tool's functionality.

Should be able to use a workflow like this:


name: Example
on: [push]
jobs:
  setup:
    name: Setup
    runs-on: ubuntu-latest
    steps:
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
        with:
          driver-opts: image=moby/buildkit:master

      - name: Build and push the CI image
        id: build
        uses: docker/build-push-action@v4
        with:
          file: Dockerfile
          push: true
          tags: ghcr.io/username/repo:latest
          outputs: type=image,oci-mediatypes=true,compression=zstd,compression-level=3,force-compression=true

  lint_and_unit_test:
    name: Lint and unit test
    needs: setup
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/username/repo:latest
    steps:
      - name: Sanity Test
        run: echo I should run

Platforms where you need the tool

• Azure DevOps
• GitHub Actions

Runner images where you need the tool

• Ubuntu 20.04
• Ubuntu 22.04
• macOS 11
• macOS 12
• macOS 13
• Windows Server 2019
• Windows Server 2022

Can this tool be installed during the build?

No, it's needed to pull newer zstd images.

Tool installation time in runtime

No response

Are you willing to submit a PR?

I can submit a PR if this isn't being held back for some special reason.

[ilia-shipitsin]

@TikiTDO ,

we install moby-engine provided by Ubuntu: https://github.com/actions/runner-images/blob/main/images/linux/scripts/installers/docker-moby.sh#L17

if you wish to switch to some other moby package, it is ok for your private repo (just using "apt install ..." as usual), but it probably won't be desired for the rest of users.

please keep it in mind when proposing PR

[mikhailkoliada]

We are using and providing only the Moby versions supported by the Microsoft's repository (always latest). Please contact https://github.com/microsoft/linux-package-repositories if you have concerns

[TikiTDO]

@ilia-shipitsin In this case my problem happens while the runner is trying to start the image that the actual workflow will run in. The only viable solution for me would be to build my own runner and launch it on a self-hosted machine, which seems like a lot of extra work for something that seems to be due to happen soon anyway. I tend to not like opening issues otherwise because it tends to be a waste of my time.

@mikhailkoliada Based on this link it appears that the newest moby-cli is in the official Microsoft repository. This issue is more about at least understanding when it might ship because it's affecting a workflow I'm working on, so I want to know if this is something I can wait on imminently, or if I'm going to be waiting another year and can put the feature on the back burner.

[ilia-shipitsin]

@mikhailkoliada , let's keep it open for a while since it is a feature request anyway

I'm against changing moby build to some custom build, but I beleive there might be other options

1. changing moby during runtime
2. maybe zstd is an option which can/should be enabled for moby ?
3. maybe microsoft folks will help us

[TikiTDO]

@ilia-shipitsin It appears that the new version is actually in the MS repo, but it's in testing. No custom builds necessary. I imagine it will be released imminently, but I couldn't find any discussion on it. See this link. That said, the zstd PR is fairly small so it may be possible to backport it to moby 20.

[vpolikarpov-akvelon]

Hey @TikiTDO, good news. The latest Ubuntu image update that has been rolled out recently includes Docker Moby 23.0.6.

[TikiTDO]

Awesome, thanks for letting me know @vpolikarpov-akvelon