cannot output arch linux image to local directory

[ReenigneArcher]

Contributing guidelines

• I've read the contributing guidelines and wholeheartedly agree

I've found a bug, and:

• The documentation does not mention anything about my problem
• There are no open or closed issues that are related to my problem

Description

Since today, I cannot output the built image to a local directory. This worked yesterday, so I'm not quite sure what is causing the problem.

I use the outputs option to copy the built images to the local runner as a way to copy the release assets out of the image.

      - name: Build and push
        id: build
        uses: docker/build-push-action@v6
        with:
          context: ./
          file: ${{ matrix.dockerfile }}
          push: ${{ inputs.publish_release }}
          platforms: ${{ steps.prepare.outputs.platforms }}
          build-args: |
            BRANCH=${{ steps.prepare.outputs.branch }}
            BUILD_DATE=${{ steps.prepare.outputs.build_date }}
            BUILD_VERSION=${{ inputs.release_version }}
            COMMIT=${{ inputs.release_commit }}
            CLONE_URL=${{ steps.prepare.outputs.clone_url }}
            RELEASE=${{ inputs.publish_release }}
          tags: ${{ steps.prepare.outputs.tags }}
          cache-from: type=local,src=/tmp/.buildx-cache
          cache-to: type=local,dest=/tmp/.buildx-cache
          outputs: 'type=local,dest=image'

Expected behaviour

Copy image files out of built image as it did before.

Actual behaviour

Cannot copy files out of image, permission denied.

Repository URL

https://github.com/LizardByte/Sunshine

Workflow run URL

https://github.com/LizardByte/Sunshine/actions/runs/16061306236/job/45329065378#step:10:3681

YAML workflow

https://github.com/LizardByte/.github/blob/master/.github/workflows/__call-docker.yml called from https://github.com/LizardByte/Sunshine/blob/2259141bd2493eb1b88e74469e0784fcef06a3e0/.github/workflows/ci.yml#L56

Workflow logs

#33 [sunshine 6/6] WORKDIR /home/lizard
#33 DONE 0.0s

#34 exporting to client directory
#34 copying files
#34 copying files 3.36MB 0.0s done
#34 ERROR: error from receiver: failed to symlink image/etc/ca-certificates/extracted/cadir/002c0b4f.0: symlink GlobalSign_Root_R46.pem image/etc/ca-certificates/extracted/cadir/002c0b4f.0: permission denied
------
 > exporting to client directory:
------
ERROR: failed to build: failed to solve: error from receiver: failed to symlink image/etc/ca-certificates/extracted/cadir/002c0b4f.0: symlink GlobalSign_Root_R46.pem image/etc/ca-certificates/extracted/cadir/002c0b4f.0: permission denied

BuildKit logs

Additional info

Dockerfile: https://github.com/LizardByte/Sunshine/blob/master/docker/archlinux.dockerfile

I just started extracting the full image using outputs a few days ago, but it definitely worked. I am not sure if this failure is due to some weird change in the arch linux base image or what. Any guidance/solution would be appreciated. I'd be happy to provide any more details if necessary.

[crazy-max]

as a way to copy the release assets out of the image.
...
I just started extracting the full image using outputs a few days ago

You should not output the full rootfs if you just want your built binaries. In such case you should have a scratch stage that just copies what you need as shown in https://docs.docker.com/build/building/export/#export-binaries-from-a-build

In your Dockerfile right before the last sunshine stage https://github.com/LizardByte/Sunshine/blob/2259141bd2493eb1b88e74469e0784fcef06a3e0/docker/archlinux.dockerfile#L115 you would have a stage named artifacts with only the artifacts like:

...
FROM scratch AS artifacts
COPY --link --from=sunshine-build /build/sunshine/pkg/sunshine*.pkg.tar.zst /sunshine.pkg.tar.zst
COPY --link --from=sunshine-build /build/sunshine/sunshine.pkg.tar.gz /sunshine.pkg.tar.gz

FROM sunshine-base AS sunshine
...

You can then remove https://github.com/LizardByte/Sunshine/blob/2259141bd2493eb1b88e74469e0784fcef06a3e0/docker/archlinux.dockerfile#L119-L121 that are not used in your runnable image

Then in the build step: https://github.com/LizardByte/.github/blob/9e4ccf81b58554cd7a4a2167e49cae94627b6121/.github/workflows/__call-docker.yml#L264-L283

You should set target input to artifacts depending of what you want to build and I guess in this case it would be:

      - name: Build and push
        id: build
        uses: docker/build-push-action@v6
        with:
          context: ./
          file: ${{ matrix.dockerfile }}
          push: ${{ inputs.publish_release }}
          platforms: ${{ steps.prepare.outputs.platforms }}
          build-args: |
            BRANCH=${{ steps.prepare.outputs.branch }}
            BUILD_DATE=${{ steps.prepare.outputs.build_date }}
            BUILD_VERSION=${{ inputs.release_version }}
            COMMIT=${{ inputs.release_commit }}
            CLONE_URL=${{ steps.prepare.outputs.clone_url }}
            RELEASE=${{ inputs.publish_release }}
          tags: ${{ steps.prepare.outputs.tags }}
          cache-from: type=local,src=/tmp/.buildx-cache
          cache-to: type=local,dest=/tmp/.buildx-cache
          no-cache-filters: ${{ steps.prepare.outputs.no_cache_filters }}
          outputs: ${{ steps.prepare.outputs.artifacts == 'true' && './artifacts' || '' }}
          target: ${{ steps.prepare.outputs.artifacts == 'true' && 'artifacts' || '' }}

Then you can remove the next step "Arrange Artifacts": https://github.com/LizardByte/.github/blob/9e4ccf81b58554cd7a4a2167e49cae94627b6121/.github/workflows/__call-docker.yml#L285-L306

And just list the artifacts:

      - name: List artifacts
        run: |
          tree -nh ./artifacts

[crazy-max]

Also looking at your Dockerfiles https://github.com/LizardByte/Sunshine/tree/master/docker and workflows I would suggest to take a look at Bake: https://docs.docker.com/build/bake/

Btw I'm a user of sunshine myself and I would be happy to contribute 🤗

[ReenigneArcher]

You should not output the full rootfs if you just want your built binaries. In such case you should have a scratch stage that just copies what you need as shown in https://docs.docker.com/build/building/export/#export-binaries-from-a-build

In your Dockerfile right before the last sunshine stage https://github.com/LizardByte/Sunshine/blob/2259141bd2493eb1b88e74469e0784fcef06a3e0/docker/archlinux.dockerfile#L115 you would have a stage named artifacts with only the artifacts like:

I did that before but it build the images twice (very rarely would it resume where it left off). This caused mutliarch builds to take 5 hours, now they are 2.5 hours. It has been bugging me for a long time and I literally just fixed that a few days ago in LizardByte/.github#475 ... 😆

The arrange artifacts is still necessary due to how they are output when doing a multiarch image. It would be nice if we could just specify the exact files/directories that we want to pull out of the image instead of extracting out the full image (whatever stage it's at).

Btw I'm a user of sunshine myself and I would be happy to contribute 🤗

That would be awesome! I'm definitely not an expert on Docker and we could use all the help we can get. Our workflow is a bit complicated as I wanted to reduce duplicated workflows among our various repos and currently the callled docker workflow works pretty well throughout our org. I haven't heard of "bake" but it seems interesting. Part of the complexity is we have different build requirements for PRs versus push events, as we don't PR builds to take so long.

[ReenigneArcher]

The problem seems to have resolved itself

[ReenigneArcher]

It seems the problem is not solved, but actually intermittent. Not sure why I only ever see this with an Arch Linux image.