--domain-filter functionality

[hfi-wiit]

Hi,

we currently use external-dns with its designate provider.
We have an openstack tenant that holds multiple zones, however each cluster only needs to access a specific domain/zone.
So we have access tokens that only have access to the zone in question and our external-dns is started with --domain-filter=mysub.my.domain.

Switching to your webhook, everything worked in a testing environment without limited access.
However in production, where access is limited, I get a lot of errors in logs like the following.

[pod/external-dns-895bb548b-zd9jq/webhook] time="2025-09-15T10:33:38Z" level=error msg="Failed to get Records: Successfully re-authenticated, but got error executing request: Expected HTTP response code [200 204 300] when accessing [GET https://.../v2/zones/d28f0ff7-.../recordsets], but got 401 instead: {\"error\": {\"code\": 401, \"title\": \"Unauthorized\", \"message\": \"The request you have made requires authentication.\"}}"

It actually seems to try to access all the zones, not only the one it is meant to.
Is there a way to limit access to the requested zone?

[hfi-wiit]

I guess the feature may be there, only not yet configurable?

external-dns-openstack-webhook/cmd/webhook/main.go

Line 58 in 99dde72

epf := endpoint.NewDomainFilter([]string{})

[hfi-wiit]

Assuming it is only the flag that is missing, I created a PR: #65

[frittentheke]

Thanks @hfi-wiit for the bug report and also for the PR you proposed.
The issue of some of the zones in Designate not being accessible to the webhook certainly seems valid.

But please let us gather all the details to not just patch some of the underlying issue, but to understand and fix it thoroughly.

1. The domain-filter in external DNS (https://github.com/kubernetes-sigs/external-dns/blob/ad3d958e4a19c3155ba15d937195344bcbbcae16/docs/flags.md?plain=1#L58) should actually already work to exclude certain unwanted records from being created. But I believe the error you see actually comes from the reconciliation / listing of all existing records (see https://github.com/kubernetes-sigs/external-dns/blob/ad3d958e4a19c3155ba15d937195344bcbbcae16/provider/webhook/webhook.go#L158). In the case of our webhook this reaches this method here

   external-dns-openstack-webhook/internal/designate/provider/provider.go

   Line 133 in 99dde72

   func (p designateProvider) Records(ctx context.Context) ([]*endpoint.Endpoint, error) {

   , which in turn just tries to find all records across all zones. This should be

   external-dns-openstack-webhook/internal/designate/provider/provider.go

   Line 135 in 99dde72

   managedZones, err := p.getZones(ctx)

   , the calling https://github.com/inovex/external-dns-openstack-webhook/blob/main/internal/designate/client/client.go#L93-L96 targeting "all zones", regardless if they are accessible.
2. Could you kindly provide a detailed description of how you configure your access tokens (I suppose you use OpenStack application credentials with access rules ? https://docs.openstack.org/keystone/latest/user/application_credentials.html#access-rules)? In the end the webhook provider does call https://docs.openstack.org/api-ref/dns/dns-api-v2-index.html#list-zones. So the zone you have issues with is actually in that list, but its RecordSets can then not be fetched so it seems.
3. Could you run the webhook with debug log level and maybe even provide the line of code in which the error actually occurs? Your output likely is "just" the error that external-dns received from calling the webhook.

Thanks again for your valuable feedback @hfi-wiit. Please just bare with me to figure out how we can best handle this case and apply the "filtering" of zones or whatever approach best. In any case I believe it makes sense to allow filtering of zones, I just want to make sure this is the cleanest approach also for your case of actually restricted access to the Designate API.

[hfi-wiit]

Hi

We already use the --domain-filter flag on external-dns and at least with its built-in designate provider it used to work as expected.
The error definitely comes from the webhook sidecar container and it is repeated for different zones, so it seems to access all the zones in the project (tries to get the recordsets from all zones).

We use the following access rules on application credentials:

resource "openstack_identity_application_credential_v3" "this" {
  for_each = local.zones

  name  = "external-dns@${each.key}"
  roles = ["member"]

  access_rules {
    path    = "/v2/zones"
    service = "dns"
    method  = "GET"
  }

  access_rules {
    path    = "/v2/zones/${openstack_dns_zone_v2.this[each.key].id}"
    service = "dns"
    method  = "GET"
  }

  access_rules {
    path    = "/v2/zones/${openstack_dns_zone_v2.this[each.key].id}/recordsets"
    service = "dns"
    method  = "GET"
  }

  access_rules {
    path    = "/v2/zones/${openstack_dns_zone_v2.this[each.key].id}/recordsets/*"
    service = "dns"
    method  = "GET"
  }

  access_rules {
    path    = "/v2/zones/${openstack_dns_zone_v2.this[each.key].id}/recordsets"
    service = "dns"
    method  = "POST"
  }

  access_rules {
    path    = "/v2/zones/${openstack_dns_zone_v2.this[each.key].id}/recordsets/*"
    service = "dns"
    method  = "PUT"
  }

  access_rules {
    path    = "/v2/zones/${openstack_dns_zone_v2.this[each.key].id}/recordsets/*"
    service = "dns"
    method  = "DELETE"
  }
}

Yes the error is coming from external-dns's webhook RecordsHandler: https://github.com/kubernetes-sigs/external-dns/blob/ad3d958e4a19c3155ba15d937195344bcbbcae16/provider/webhook/webhook.go#L179

Here is the full log from pod start of both, external-dns and webhook container.
It is from a test setup with 3 zones, one of which should be accessed by this external-dns.
It complains about e8e845dc-ca21-4a38-aeab-609a1514be0f & e8e845dc-ca21-4a38-aeab-609a1514be0f, which are the zones the credentials don't have access to. It does not complain about the third zone, that it is supposed to update, but it also doesn't update it (removed a service with external-dns.alpha.kubernetes.io/hostname: test.hfitest.de; this works if I use an unlimited app cred).

The extenal-dns container in this case has the following flags:

      - args:
        - --source=service
        - --source=ingress
        - --registry=txt
        - --provider=webhook
        - --domain-filter=hfitest.de
        - --txt-prefix=hfipref
        - --txt-owner-id=hfiedns

[pod/external-dns-86646b46ff-fvgb5/webhook] time="2025-09-16T08:20:15Z" level=debug msg="Starting status server on 0.0.0.0:8080"
[pod/external-dns-86646b46ff-fvgb5/webhook] time="2025-09-16T08:20:15Z" level=info msg="Using OpenStack Keystone at https://identity.openstack.my.replaced.domain/v3/"
[pod/external-dns-86646b46ff-fvgb5/webhook] time="2025-09-16T08:20:15Z" level=info msg="Found OpenStack Designate (DNS) service at https://dns.openstack.my.replaced.domain/"
[pod/external-dns-86646b46ff-fvgb5/webhook] time="2025-09-16T08:20:15Z" level=debug msg="Starting webhook server on 127.0.0.1:8888"
[pod/external-dns-86646b46ff-fvgb5/webhook] time="2025-09-16T08:20:16Z" level=error msg="Failed to get Records: Successfully re-authenticated, but got error executing request: Expected HTTP response code [200 204 300] when accessing [GET https://dns.openstack.my.replaced.domain/v2/zones/4c6998b9-2cac-4b57-b29e-bbfaa741ff24/recordsets], but got 401 instead: {\"error\": {\"code\": 401, \"title\": \"Unauthorized\", \"message\": \"The request you have made requires authentication.\"}}"
[pod/external-dns-86646b46ff-fvgb5/external-dns] time="2025-09-16T08:20:15Z" level=info msg="config: {APIServerURL: KubeConfig: RequestTimeout:30s DefaultTargets:[] GlooNamespaces:[gloo-system] SkipperRouteGroupVersion:zalando.org/v1 Sources:[service ingress] Namespace: AnnotationFilter: LabelFilter: IngressClassNames:[] FQDNTemplate: CombineFQDNAndAnnotation:false IgnoreHostnameAnnotation:false IgnoreNonHostNetworkPods:false IgnoreIngressTLSSpec:false IgnoreIngressRulesSpec:false ListenEndpointEvents:false ExposeInternalIPV6:false GatewayName: GatewayNamespace: GatewayLabelFilter: Compatibility: PodSourceDomain: PublishInternal:false PublishHostIP:false AlwaysPublishNotReadyAddresses:false ConnectorSourceServer:localhost:8080 Provider:webhook ProviderCacheTime:0s GoogleProject: GoogleBatchChangeSize:1000 GoogleBatchChangeInterval:1s GoogleZoneVisibility: DomainFilter:[hfitest.de] ExcludeDomains:[] RegexDomainFilter: RegexDomainExclusion: ZoneNameFilter:[] ZoneIDFilter:[] TargetNetFilter:[] ExcludeTargetNets:[] AlibabaCloudConfigFile:/etc/kubernetes/alibaba-cloud.json AlibabaCloudZoneType: AWSZoneType: AWSZoneTagFilter:[] AWSAssumeRole: AWSProfiles:[] AWSAssumeRoleExternalID: AWSBatchChangeSize:1000 AWSBatchChangeSizeBytes:32000 AWSBatchChangeSizeValues:1000 AWSBatchChangeInterval:1s AWSEvaluateTargetHealth:true AWSAPIRetries:3 AWSPreferCNAME:false AWSZoneCacheDuration:0s AWSSDServiceCleanup:false AWSSDCreateTag:map[] AWSZoneMatchParent:false AWSDynamoDBRegion: AWSDynamoDBTable:external-dns AzureConfigFile:/etc/kubernetes/azure.json AzureResourceGroup: AzureSubscriptionID: AzureUserAssignedIdentityClientID: AzureActiveDirectoryAuthorityHost: AzureZonesCacheDuration:0s CloudflareProxied:false CloudflareCustomHostnames:false CloudflareCustomHostnamesMinTLSVersion:1.0 CloudflareCustomHostnamesCertificateAuthority:google CloudflareDNSRecordsPerPage:100 CloudflareRegionKey: CoreDNSPrefix:/skydns/ AkamaiServiceConsumerDomain: AkamaiClientToken: AkamaiClientSecret: AkamaiAccessToken: AkamaiEdgercPath: AkamaiEdgercSection: OCIConfigFile:/etc/kubernetes/oci.yaml OCICompartmentOCID: OCIAuthInstancePrincipal:false OCIZoneScope:GLOBAL OCIZoneCacheDuration:0s InMemoryZones:[] OVHEndpoint:ovh-eu OVHApiRateLimit:20 OVHEnableCNAMERelative:false PDNSServer:http://localhost:8081 PDNSServerID:localhost PDNSAPIKey: PDNSSkipTLSVerify:false TLSCA: TLSClientCert: TLSClientCertKey: Policy:sync Registry:txt TXTOwnerID:hfiedns TXTPrefix:hfipref TXTSuffix: TXTEncryptEnabled:false TXTEncryptAESKey: TXTNewFormatOnly:false Interval:1m0s MinEventSyncInterval:5s Once:false DryRun:false UpdateEvents:false LogFormat:text MetricsAddress::7979 LogLevel:info TXTCacheInterval:0s TXTWildcardReplacement: ExoscaleEndpoint: ExoscaleAPIKey: ExoscaleAPISecret: ExoscaleAPIEnvironment:api ExoscaleAPIZone:ch-gva-2 CRDSourceAPIVersion:externaldns.k8s.io/v1alpha1 CRDSourceKind:DNSEndpoint ServiceTypeFilter:[] CFAPIEndpoint: CFUsername: CFPassword: ResolveServiceLoadBalancerHostname:false RFC2136Host:[] RFC2136Port:0 RFC2136Zone:[] RFC2136Insecure:false RFC2136GSSTSIG:false RFC2136CreatePTR:false RFC2136KerberosRealm: RFC2136KerberosUsername: RFC2136KerberosPassword: RFC2136TSIGKeyName: RFC2136TSIGSecret: RFC2136TSIGSecretAlg: RFC2136TAXFR:false RFC2136MinTTL:0s RFC2136LoadBalancingStrategy:disabled RFC2136BatchChangeSize:50 RFC2136UseTLS:false RFC2136SkipTLSVerify:false NS1Endpoint: NS1IgnoreSSL:false NS1MinTTLSeconds:0 TransIPAccountName: TransIPPrivateKeyFile: DigitalOceanAPIPageSize:50 ManagedDNSRecordTypes:[A AAAA CNAME] ExcludeDNSRecordTypes:[] GoDaddyAPIKey: GoDaddySecretKey: GoDaddyTTL:0 GoDaddyOTE:false OCPRouterName: IBMCloudProxied:false IBMCloudConfigFile:/etc/kubernetes/ibmcloud.json TencentCloudConfigFile:/etc/kubernetes/tencent-cloud.json TencentCloudZoneType: PiholeServer: PiholePassword: PiholeTLSInsecureSkipVerify:false PiholeApiVersion:5 PluralCluster: PluralProvider: WebhookProviderURL:http://localhost:8888 WebhookProviderReadTimeout:5s WebhookProviderWriteTimeout:10s WebhookServer:false TraefikDisableLegacy:false TraefikDisableNew:false NAT64Networks:[] ExcludeUnschedulable:true}"
[pod/external-dns-86646b46ff-fvgb5/external-dns] time="2025-09-16T08:20:15Z" level=info msg="GitCommitShort=unknown, GoVersion=go1.24.3, Platform=linux/amd64, UserAgent=ExternalDNS/v20250514-v0.17.0"
[pod/external-dns-86646b46ff-fvgb5/external-dns] time="2025-09-16T08:20:15Z" level=info msg="Instantiating new Kubernetes client"
[pod/external-dns-86646b46ff-fvgb5/external-dns] time="2025-09-16T08:20:15Z" level=info msg="Using inCluster-config based on serviceaccount-token"
[pod/external-dns-86646b46ff-fvgb5/external-dns] time="2025-09-16T08:20:15Z" level=info msg="Created Kubernetes client https://10.96.0.1:443"
[pod/external-dns-86646b46ff-fvgb5/external-dns] time="2025-09-16T08:20:16Z" level=error msg="Failed to do run once: soft error\nfailed to get records with code 500"
[pod/external-dns-86646b46ff-fvgb5/external-dns] time="2025-09-16T08:21:17Z" level=error msg="Failed to do run once: soft error\nfailed to get records with code 500"
[pod/external-dns-86646b46ff-fvgb5/webhook] time="2025-09-16T08:21:17Z" level=error msg="Failed to get Records: Successfully re-authenticated, but got error executing request: Expected HTTP response code [200 204 300] when accessing [GET https://dns.openstack.my.replaced.domain/v2/zones/4c6998b9-2cac-4b57-b29e-bbfaa741ff24/recordsets], but got 401 instead: {\"error\": {\"code\": 401, \"title\": \"Unauthorized\", \"message\": \"The request you have made requires authentication.\"}}"
[pod/external-dns-86646b46ff-fvgb5/webhook] time="2025-09-16T08:22:17Z" level=error msg="Failed to get Records: Successfully re-authenticated, but got error executing request: Expected HTTP response code [200 204 300] when accessing [GET https://dns.openstack.my.replaced.domain/v2/zones/e8e845dc-ca21-4a38-aeab-609a1514be0f/recordsets], but got 401 instead: {\"error\": {\"code\": 401, \"title\": \"Unauthorized\", \"message\": \"The request you have made requires authentication.\"}}"
[pod/external-dns-86646b46ff-fvgb5/external-dns] time="2025-09-16T08:22:17Z" level=error msg="Failed to do run once: soft error\nfailed to get records with code 500"
[pod/external-dns-86646b46ff-fvgb5/webhook] time="2025-09-16T08:23:19Z" level=error msg="Failed to get Records: Successfully re-authenticated, but got error executing request: Expected HTTP response code [200 204 300] when accessing [GET https://dns.openstack.my.replaced.domain/v2/zones/4c6998b9-2cac-4b57-b29e-bbfaa741ff24/recordsets], but got 401 instead: {\"error\": {\"code\": 401, \"title\": \"Unauthorized\", \"message\": \"The request you have made requires authentication.\"}}"
[pod/external-dns-86646b46ff-fvgb5/external-dns] time="2025-09-16T08:23:19Z" level=error msg="Failed to do run once: soft error\nfailed to get records with code 500"
[pod/external-dns-86646b46ff-fvgb5/external-dns] time="2025-09-16T08:24:18Z" level=error msg="Failed to do run once: soft error\nfailed to get records with code 500"
[pod/external-dns-86646b46ff-fvgb5/webhook] time="2025-09-16T08:24:18Z" level=error msg="Failed to get Records: Successfully re-authenticated, but got error executing request: Expected HTTP response code [200 204 300] when accessing [GET https://dns.openstack.my.replaced.domain/v2/zones/e8e845dc-ca21-4a38-aeab-609a1514be0f/recordsets], but got 401 instead: {\"error\": {\"code\": 401, \"title\": \"Unauthorized\", \"message\": \"The request you have made requires authentication.\"}}"
[pod/external-dns-86646b46ff-fvgb5/external-dns] time="2025-09-16T08:25:19Z" level=error msg="Failed to do run once: soft error\nfailed to get records with code 500"
[pod/external-dns-86646b46ff-fvgb5/webhook] time="2025-09-16T08:25:19Z" level=error msg="Failed to get Records: Successfully re-authenticated, but got error executing request: Expected HTTP response code [200 204 300] when accessing [GET https://dns.openstack.my.replaced.domain/v2/zones/e8e845dc-ca21-4a38-aeab-609a1514be0f/recordsets], but got 401 instead: {\"error\": {\"code\": 401, \"title\": \"Unauthorized\", \"message\": \"The request you have made requires authentication.\"}}"