Multi-source `ApplicationSet` write-back error

[rvandernoort]

Describe the bug
A clear and concise description of what the bug is.

Hi, I'm trying to add multiple sources to an ApplicationSet from which I only want to update the versions of a single source using git-write back and kustomize. Overall my setup works perfectly as long as I do not put the additional helm chart souces in the ApplicationSet, however I would like to join them. Is this currently possible or should I implement a workaround in which I put the redis instance in a seperate ApplicationSet?

To Reproduce
Steps to reproduce the behavior:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: rover-test-1
spec:
  goTemplate: true
  goTemplateOptions: ['missingkey=error']
  generators:
    - list:
        elements:
          - environment: staging
          - environment: production
  template:
    metadata:
      name: rover-test-1-{{.environment}}
      annotations:
        argocd-image-updater.argoproj.io/image-list: rover-test=private-registry/rover-test-1
        argocd-image-updater.argoproj.io/rover-test.update-strategy: semver
        argocd-image-updater.argoproj.io/rover-test.allow-tags: regexp:^v[0-9]+.[0-9]+.[0-9]+$|^[a-f0-9]{40}$
        argocd-image-updater.argoproj.io/rover-test.ignore-tags: '^[a-f0-9]{40}$'
        argocd-image-updater.argoproj.io/write-back-method: git
        argocd-image-updater.argoproj.io/write-back-target: kustomization
    spec:
      project: hw-{{.environment}}
      destination:
        namespace: rover-ns-{{.environment}}
        name: cluster-name

      sources:
        - repoURL: 'git@github.com:organisation/gitops.git'
          path: ./deployments/rover-test/overlays/{{.environment}}
          targetRevision: main

        - repoURL: 'registry-1.docker.io/bitnamicharts'
          chart: redis
          targetRevision: 20.11.*
          helm:
            valueFiles:
              - $values/deployments/rover-test/base/redis/values.yaml
        - repoURL: 'https://github.com/orginsation/gitops.git'
          ref: values
          targetRevision: main

which eventually results in the following error message:

time="2025-04-03T10:25:08Z" level=info msg="Successfully updated image 'registry/rover-test-1:v3.4.1' to 'registry/rover-test-1:v3.4.2', but pending spec update (dry run=false)" alias=rover-test application=rover-test-1-staging image_name=registry/rover-test-1 image_tag=v3.4.1 registry=registry
time="2025-04-03T10:25:08Z" level=debug msg="Using commit message: argocd-image-updater: automatic update version of rover-test-1-staging\n\nupdates image registry/rover-test-1 tag 'v3.4.1' to 'v3.4.2'\n
time="2025-04-03T10:25:08Z" level=info msg="Committing 1 parameter update(s) for application rover-test-1-staging" application=rover-test-1-staging
time=\"2025-04-03T10:25:08Z\" level=error msg=\"Could not update application spec: could not get creds for repo 'registry-1.docker.io/bitnamicharts': credentials for 'registry-1.docker.io/bitnamicharts' are not configured in Argo CD settings\" application=rover-test-1-staging"

However, I do believe I have the credentials correctly configured, since I have other ApplicationSets that do get updated correctly like:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: external-dns
spec:
  goTemplate: true
  goTemplateOptions: ['missingkey=error']
  generators:
    - list:
        elements:
          - name: main
            type: main
            target: main
            cluster: main-cluster
         - ...
  template:
    metadata:
      name: external-dns-{{.name}}
    spec:
      project: hw-infra
      destination:
        namespace: external-dns
        name: '{{.cluster}}'

      sources:
        - repoURL: "registry-1.docker.io/bitnamicharts"
          chart: external-dns
          targetRevision: v8.7.*
          helm:
            valueFiles:
              - $values/deployments/external-dns/values/values-{{.name}}.yaml
        - repoURL: 'https://github.com/organisation/gitops.git'
          ref: values
          targetRevision: '{{.target}}'
        - repoURL: 'https://github.com/organisation/gitops.git'
          path: ./deployments/external-dns/overlays/{{.name}}
          targetRevision: '{{.target}}'

which updates the patch version whenever a new version is available.

I have installed image-updater using kustomize and patch the registries with the credentials using the following patch:

  - target:
      kind: ConfigMap
      name: argocd-image-updater-config
    patch: |-
      - op: add
        path: /data
        value:
          registries.conf: |
            registries:
            - ...
            - name: Docker Hub
              api_url: https://registry-1.docker.io
              ping: yes
              credentials: secret:argocd/docker-credentials#credentials

with my credential secret like this:

apiVersion: v1
kind: Secret
metadata:
  name: docker-credentials
  namespace: argocd
data:
  credentials: redacted
type: Opaque

Expected behavior
A clear and concise description of what you expected to happen.

Preferably I would like that the kustomization writeback to only write back to my kustomization file that are located at the first source. However I think the issue lies in the fact that image-updater tries to also writeback at those external helm resources, which I do not necessarily want. I can understand that possibly this requires some additional configuration to specify a select source to update.

Additional context
Add any other context about the problem here.

Version
Please tell us about the version you encountered the issue with

argo 2.14.9
image-updater 0.16.0

Logs
Please paste any relevant logs here

time="2025-04-03T10:25:08Z" level=info msg="Successfully updated image 'registry/rover-test-1:v3.4.1' to 'registry/rover-test-1:v3.4.2', but pending spec update (dry run=false)" alias=rover-test application=rover-test-1-staging image_name=registry/rover-test-1 image_tag=v3.4.1 registry=registry
time="2025-04-03T10:25:08Z" level=debug msg="Using commit message: argocd-image-updater: automatic update version of rover-test-1-staging\n\nupdates image registry/rover-test-1 tag 'v3.4.1' to 'v3.4.2'\n
time="2025-04-03T10:25:08Z" level=info msg="Committing 1 parameter update(s) for application rover-test-1-staging" application=rover-test-1-staging
time=\"2025-04-03T10:25:08Z\" level=error msg=\"Could not update application spec: could not get creds for repo 'registry-1.docker.io/bitnamicharts': credentials for 'registry-1.docker.io/bitnamicharts' are not configured in Argo CD settings\" application=rover-test-1-staging"

I appreciate any support or insight you can provide!
Kind regards,
Rover

[chengfang]

have you tried config step https://argocd-image-updater.readthedocs.io/en/latest/basics/update-methods/#specifying-a-repository-when-using-a-helm-repository-in-repourl ?

[rvandernoort]

Hi yes I've tried this and it resulted in the following error:

time="2025-04-04T07:44:29Z" level=info msg="Setting new image to registry/rover-test-1:v3.4.3
time="2025-04-04T07:44:29Z" level=trace msg="Setting Kustomize parameter registry/rover-test-1:v3.4.3
time="2025-04-04T07:44:29Z" level=info msg="Successfully updated image 'registry/rover-test-1:v3.4.2' to 'registry/rover-test-1:v3.4.3', but pending spec update (dry run=false)
time="2025-04-04T07:44:29Z" level=debug msg="Using commit message: argocd-image-updater: automatic update version of rover-test-1-staging\n\nupdates image organisation/rover-test-1 tag 'v3.4.2' to 'v3.4.3'\n"
ime="2025-04-04T07:44:29Z" level=info msg="Committing 1 parameter update(s) for application rover-test-1-staging
time="2025-04-04T07:44:31Z" level=info msg="Initializing git@github.com:organisation/gitops.git to /tmp/git-rover-test-1-staging2137139863"
time="2025-04-04T07:44:31Z" level=debug msg="Using target revision 'main' from matching source 'git@github.com:organisation/gitops.git'
time="2025-04-04T07:44:31Z" level=trace msg="targetRevision for update is 'main'
time="2025-04-04T07:44:31Z" level=info msg="git fetch origin main --force --prune --depth 1
time="2025-04-04T07:44:33Z" level=info msg=Trace args="[git fetch origin main --force --prune --depth 1]
time="2025-04-04T07:44:33Z" level=info msg="git checkout --force main
time="2025-04-04T07:44:33Z" level=info msg=Trace args="[git checkout --force main]
time="2025-04-04T07:44:33Z" level=info msg="git clean -ffdx
time="2025-04-04T07:44:33Z" level=info msg=Trace args="[git clean -ffdx]
time="2025-04-04T07:44:33Z" level=info msg="updating base /tmp/git-rover-test-1-staging2137139863
time="2025-04-04T07:44:33Z" level=error msg="Could not update application spec: could not find kustomization in /tmp/git-rover-test-1-staging2137139863

Now it seems like image-updater does not use the correct path and cannot find the kustomization file.

[chengfang]

From your original post, it seems you're using helm. Not sure why the error message is about "could not find kustomization".

For multi-source app with helm, I have a sample app https://github.com/chengfang/image-updater-examples/blob/main/write-helmvalues/app/write-helmvalues.yaml, which configures to write to a helm values file in a configured git repo.

[rvandernoort]

I'm using both kustomize and helm in conjunction here:

• kustomize for our own workloads
• helm for the external resources like redis

Therefore, I want to use the kustomize writeback to update the kustomize versions only when we publish a new version of our workload.

I do not want to use image-updater for the helm source as I use the targetRef for manual helm chart updates.

Summarizing: I want image-updater to write back to the kustomize source and I want to ignore the helm source.

[seanlucey]

Is there any solution on this? I'm also seeing this behaviour. I upgrade my helm chart from 0.11 to 0.12 yesterday and since then it's failing to get git credentials. I've had to revert back to the older version as this is not working across the board. Image Updater is deployed to the same namespace as ArgoCD - and I've confirmed git credentials in ArgoCD work.

[dkarpele]

@rvandernoort it seems image-updater doesn't properly support multiple sources of different types (Helm and Kustomize) in one Application. I have prepared a very simple Application example that behaves just weird.

If you get trace log level you will see:

time="2025-04-26T18:53:35+02:00" level=trace msg="Could not get Source of type Helm or Kustomize from multisource configuration. Returning first source from the list" application=sources

This line is from

argocd-image-updater/pkg/argocd/argocd.go

Lines 678 to 693 in 61ea007

	// getApplicationSource returns the main source of a Helm or Kustomize type of the application
	func getApplicationSource(app *v1alpha1.Application) *v1alpha1.ApplicationSource {
	if app.Spec.HasMultipleSources() {
	for _, s := range app.Spec.Sources {
	if s.Helm != nil || s.Kustomize != nil {
	return &s
	}
	}
	log.WithContext().AddField("application", app.GetName()).Tracef("Could not get Source of type Helm or Kustomize from multisource configuration. Returning first source from the list")
	return &app.Spec.Sources[0]
	}
	return app.Spec.Source
	}

So this function returns first found type source. As a result, in my case I have a file .argocd-source-specs.yaml in the directory of first found application (001-simple-kustomize-app) with the content:

kustomize:
  images:
  - nginx:1.17.10@sha256:6fff55753e3b34e36e24e37039ee9eae1fe38a6420d8ae16ef37c92d1eb26699
  - gcr.io/heptio-images/ks-guestbook-demo:0.2@sha256:e6fbd640820c32c847a53b93eb7b404f25ffb56df1cc16d0711eec3a405d6509

So image-updater included "helm" image into kustomize application file!