Pod keeps restarting #1128
Description
Describe the bug
Image updater is not pulling the latest digest.
To Reproduce
kubectl describe application/apple-bean-app-api-dev
Name: apple-bean-app-api-dev
Namespace: argocd
Labels: <none>
Annotations: argocd-image-updater.argoproj.io/app-api.update-strategy: digest
argocd-image-updater.argoproj.io/image-list: app-api=asdabacrmgmt.azurecr.io/apple-bean/app-api
API Version: argoproj.io/v1alpha1
Kind: Application
Metadata:
Creation Timestamp: 2025-05-05T15:29:33Z
Finalizers:
resources-finalizer.argocd.argoproj.io
Generation: 14913
Owner References:
API Version: argoproj.io/v1alpha1
Block Owner Deletion: true
Controller: true
Kind: ApplicationSet
Name: apps-dev
UID: b3808337-f2a7-4e01-a35f-8be4512ac625
Resource Version: 85238345
UID: 7dfc2e26-331c-475e-8c9f-aa493a27a48a
Spec:
Destination:
Namespace: apple-bean
Server: https://newcoffee-aks-dev-cluster-abcdef.hcp.westus2.azmk8s.io:443
Project: default
Source:
Path: apple-bean/app-api/dev
Repo URL: git@github.com:asdabtech/infra-azure-aks.git
Target Revision: main
Sync Policy:
Automated:
Prune: true
Self Heal: true
Sync Options:
CreateNamespace=true
PrunePropagationPolicy=foreground
Status:
Controller Namespace: argocd
Health:
Last Transition Time: 2025-05-10T11:57:15Z
Status: Degraded
History:
Deploy Started At: 2025-05-10T11:24:22Z
Deployed At: 2025-05-10T11:24:28Z
Id: 1496
Initiated By:
Automated: true
Revision: c884af36c334295ca99462baaac67833b6ea1f40
Source:
Path: apple-bean/app-api/dev
Repo URL: git@github.com:asdabtech/infra-azure-aks.git
Target Revision: main
Deploy Started At: 2025-05-10T11:26:08Z
Deployed At: 2025-05-10T11:26:17Z
Id: 1497
Initiated By:
Automated: true
Revision: c884af36c334295ca99462baaac67833b6ea1f40
Source:
Kustomize:
Images:
asdabacrmgmt.azurecr.io/apple-bean/app-api:dev@sha256:2d4e50c4feceb85c28bcb81cdcf665a6be84fc3b175c51565efb4adb84d4426b
Path: apple-bean/app-api/dev
Repo URL: git@github.com:asdabtech/infra-azure-aks.git
Target Revision: main
Deploy Started At: 2025-05-10T11:26:36Z
Deployed At: 2025-05-10T11:26:43Z
Id: 1498
Initiated By:
Automated: true
Revision: c884af36c334295ca99462baaac67833b6ea1f40
Source:
Path: apple-bean/app-api/dev
Repo URL: git@github.com:asdabtech/infra-azure-aks.git
Target Revision: main
Operation State:
Finished At: 2025-05-10T11:42:27Z
Message: successfully synced (all tasks run)
Operation:
Initiated By:
Automated: true
Retry:
Limit: 5
Sync:
Prune: true
Revision: 92c9b77c615376325ca3efd8f7a7b4332afd0e20
Sync Options:
CreateNamespace=true
PrunePropagationPolicy=foreground
Phase: Succeeded
Started At: 2025-05-10T11:42:18Z
Sync Result:
Resources:
Group: apps
Hook Phase: Running
Kind: Deployment
Message: deployment.apps/app-api unchanged
Name: app-api
Namespace: apple-bean
Status: Synced
Sync Phase: Sync
Version: v1
Revision: 92c9b77c615376325ca3efd8f7a7b4332afd0e20
Source:
Path: apple-bean/app-api/dev
Repo URL: git@github.com:asdabtech/infra-azure-aks.git
Target Revision: main
Reconciled At: 2025-05-10T12:37:51Z
Resources:
Group: apps
Health:
Message: Deployment "app-api" exceeded its progress deadline
Status: Degraded
Kind: Deployment
Name: app-api
Namespace: apple-bean
Status: Synced
Version: v1
Source Hydrator:
Source Type: Kustomize
Summary:
External UR Ls:
http://dev-api.asdabbeta.io/
Images:
asdabacrmgmt.azurecr.io/apple-bean/app-api:dev
asdabacrmgmt.azurecr.io/apple-bean/app-api@sha256:e0a52450992a9b579c0a5674362d8eddc47d48e6c3e0ba989320626b0946b8c1
Sync:
Compared To:
Destination:
Namespace: apple-bean
Server: https://usw2-aks-dev-cluster-1u5y0oe3.hcp.westus2.azmk8s.io:443
Source:
Path: apple-bean/app-api/dev
Repo URL: git@github.com:asdabtech/infra-azure-aks.git
Target Revision: main
Revision: 92c9b77c615376325ca3efd8f7a7b4332afd0e20
Status: Synced
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ImagesUpdated 59m ArgocdImageUpdater Successfully updated application 'apple-bean-app-api-dev'
Normal OperationStarted 58m argocd-application-controller Initiated automated sync to '92c9b77c615376325ca3efd8f7a7b4332afd0e20'
Normal ResourceUpdated 58m argocd-application-controller Updated sync status: Synced -> OutOfSync
Normal ResourceUpdated 58m argocd-application-controller Updated sync status: OutOfSync -> Synced
Normal OperationCompleted 58m argocd-application-controller Sync operation to 92c9b77c615376325ca3efd8f7a7b4332afd0e20 succeeded
Normal ResourceDeleted 53m argocd-server me@me.com deleted resource apps/ReplicaSet 'app-api-658dcd4556'
Normal ResourceUpdated 43m argocd-application-controller Updated health status: Progressing -> Degraded
and here is the output of the kustomize:
kustomize build --enable-helm
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
reloader.stakater.com/auto: "true"
name: app-api
spec:
replicas: 1
selector:
matchLabels:
app: app-api
template:
metadata:
labels:
app: app-api
spec:
containers:
image: asdabacrmgmt.azurecr.io/apple-bean/app-api@sha256:e0a52450992a9b579c0a5674362d8eddc47d48e6c3e0ba989320626b0946b8c1
name: app-api
ports:
- containerPort: 80
resources:
limits:
cpu: "1"
memory: 512Mi
requests:
cpu: 50m
memory: 1Mi
nodeSelector:
product: apple-bean
tolerations:
- effect: NoSchedule
key: product
operator: Equal
value: apple-bean
Expected behavior
Every time a developer pushes their code to the main branch, the automation tags it with dev. I just want the image updater to force it to fetch the latest dev tag.
Additional context
Add any other context about the problem here.
Version
targetRevision: 0.12.1
Logs
Failed to pull image "asdabacrmgmt.azurecr.io/apple-bean/app-api@sha256:e0a52450992a9b579c0a5674362d8eddc47d48e6c3e0ba989320626b0946b8c1": [rpc error: code = NotFound desc = failed to pull and unpack image "asdabacrmgmt.azurecr.io/apple-bean/app-api@sha256:e0a52450992a9b579c0a5674362d8eddc47d48e6c3e0ba989320626b0946b8c1": failed to unpack image on snapshotter overlayfs: unexpected media type application/octet-stream for sha256:e0a52450992a9b579c0a5674362d8eddc47d48e6c3e0ba989320626b0946b8c1: not found, failed to pull and unpack image "asdabacrmgmt.azurecr.io/apple-bean/app-api@sha256:e0a52450992a9b579c0a5674362d8eddc47d48e6c3e0ba989320626b0946b8c1": failed to resolve reference "asdabacrmgmt.azurecr.io/apple-bean/app-api@sha256:e0a52450992a9b579c0a5674362d8eddc47d48e6c3e0ba989320626b0946b8c1": failed to authorize: failed to fetch anonymous token: unexpected status from GET request to https://asdabacrmgmt.azurecr.io/apple-bean?scope=repository%3Asocial-logix%2Fapp-api%3Apull&service=asdabacrmgmt.azurecr.io:apple-bean]
and this:
argocd-image-updater test asdabacrmgmt.azurecr.io/apple-bean/app-api:dev --registries-conf-path /app/config/registries.conf --update-strategy digest
rgocd-image-updater test asdabacrmgmt.azurecr.io/apple-bean/app-api:dev --registries-conf-path /app/config/registries.conf --update-strategy
digest
INFO[0000] retrieving information about image image_alias= image_digest= image_name=asdabacrmgmt.azurecr.io/apple-bean/app-api image_tag=dev registry_url=asdabacrmgmt.azurecr.io
DEBU[0000] rate limiting is disabled prefix=asdabacrmgmt.azurecr.io registry="https://asdabacrmgmt.azurecr.io"
INFO[0000] Loaded 1 registry configurations from /app/config/registries.conf
INFO[0000] /scripts/login.sh dir= execID=bdb85
INFO[0000] Fetching available tags and metadata from registry application=test image_alias= image_digest= image_name=asdabacrmgmt.azurecr.io/apple-bean/app-api image_tag=dev registry_url=asdabacrmgmt.azurecr.io
FATA[0000] could not get tags: cannot use update strategy 'digest' for image 'asdabacrmgmt.azurecr.io/apple-bean/app-api:dev' without a version constraint application=test image_alias= image_digest= image_name=asdabacrmgmt.azurecr.io/apple-bean/app-api image_tag=dev registry_url=asdabacrmgmt.azurecr.io