Unable to test coreruleset/nextcloud-rule-exclusions-plugin in ingress-nginx

[cloudymax]

I am unable to get go-ftw to work using the official tests here: https://github.com/coreruleset/nextcloud-rule-exclusions-plugin/tree/v1.4.0/tests/regression/nextcloud-rule-exclusions-plugin for the following reasons:

1. When advised to test with go-ftw 0.6.4 in this comment: Perhaps need a rule exception for 911100 on /remote.php/dav/files/your_username_here/ uris nextcloud-rule-exclusions-plugin#134 (comment) however this version fails to start on any system I try it on (AMD64, ARM64, Mac)

   🛠️  Starting tests!
   🚀 Running go-ftw!
   panic: runtime error: invalid memory address or nil pointer dereference
   [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x8674a1]
   
   goroutine 1 [running]:
   github.com/coreruleset/go-ftw/runner.RunTest(0xc0000b8870, 0xc0001ba2a0)
           /home/runner/work/go-ftw/go-ftw/runner/run.go:81 +0xe1
   github.com/coreruleset/go-ftw/runner.Run(0xc0001136c0, {0xc0001e0000, 0x56, 0xc00018f7d0?}, {0x0, 0x0, 0x0, 0x0, {0x0, 0x0}, ...}, ...)
           /home/runner/work/go-ftw/go-ftw/runner/run.go:61 +0x4b3
   github.com/coreruleset/go-ftw/cmd.runE(0xc000004900, {0x9427db?, 0x4?, 0x9426c3?})
           /home/runner/work/go-ftw/go-ftw/cmd/run.go:157 +0xd18
   github.com/spf13/cobra.(*Command).execute(0xc000004900, {0xeb9e40, 0x0, 0x0})
           /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:983 +0xabc
   github.com/spf13/cobra.(*Command).ExecuteC(0xc000004300)
           /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1115 +0x3ff
   github.com/spf13/cobra.(*Command).Execute(...)
           /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1039
   github.com/spf13/cobra.(*Command).ExecuteContext(...)
           /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1032
   github.com/coreruleset/go-ftw/cmd.Execute({0xc00002c460, 0x6a})
           /home/runner/work/go-ftw/go-ftw/cmd/root.go:47 +0xff
   main.main()
           /home/runner/work/go-ftw/go-ftw/main.go:37 +0x19a
   

2. When using the latest version of go-ftw in default mode it is unable to find a log-line in either Native or JSON format. I am using the official tests from this org verbatim as they are published here: https://github.com/coreruleset/nextcloud-rule-exclusions-plugin/tree/v1.4.0/tests/regression/nextcloud-rule-exclusions-plugin

   Error:

   👉 executing tests in file
       running 9508111-1: Error: failed to find start marker: can't find log marker. Am I reading the correct log? Log file: /var/log/modsec_audit.log
   

   JSON example contents of /var/log/modsec_audit.log to validate that log file exists and is populated.

   {
     "transaction": {
       "client_ip": "127.0.0.1",
       "time_stamp": "Thu Sep  4 13:05:45 2025",
       "server_id": "52e876505bc18b06b6dda8c18a069e0f25a4d726",
       "client_port": 53402,
       "host_ip": "127.0.0.1",
       "host_port": 80,
       "unique_id": "175698394525.850314",
       "request": {
         "method": "DELETE",
         "http_version": 1.1,
         "uri": "/apps/mail/api/internalAddress/example.inc?type=domain",
         "body": "",
         "headers": {
           "Accept": "text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5",
           "Connection": "close",
           "Content-Length": "0",
           "Host": "localhost",
           "User-Agent": "OWASP CRS test agent"
         }
       },
       "response": {
         "http_code": 404,
         "headers": {
           "Server": "",
           "Date": "Thu, 04 Sep 2025 11:05:45 GMT",
           "Content-Length": "146",
           "Content-Type": "text/html",
           "Connection": "close"
         }
       },
       "producer": {
         "modsecurity": "ModSecurity v3.0.14 (Linux)",
         "connector": "ModSecurity-nginx v1.0.4",
         "secrules_engine": "DetectionOnly",
         "components": [
           "OWASP_CRS/4.15.0\""
         ]
       },
       "messages": [
         {
           "message": "URL file extension is restricted by policy",
           "details": {
             "match": "Matched \"Operator `Within' with parameter `.ani/ .asa/ .asax/ .ascx/ .backup/ .bak/ .bat/ .cdx/ .cer/.cfg/ .cmd/ .com/ .compositefont/ .config/ .conf/ .crt/ .cs/ .csproj/ .csr/ .dat/ .db/ .dbf/ .dist/ .dll/ .dos/ .dpkg-dist/ .drv/ .gadget/ .h (343 characters omitted)' against variable `TX:EXTENSION' (Value: `.inc/' )",
             "reference": "o7,4o8,3v38,11t:urlDecodeUnio234,5t:lowercase",
             "ruleId": "920440",
             "file": "/etc/nginx/owasp-modsecurity-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf",
             "lineNumber": "1045",
             "data": ".inc",
             "severity": "2",
             "ver": "OWASP_CRS/4.15.0",
             "rev": "",
             "tags": [
               "application-multi",
               "language-multi",
               "platform-multi",
               "attack-protocol",
               "paranoia-level/1",
               "OWASP_CRS",
               "OWASP_CRS/PROTOCOL-ENFORCEMENT",
               "capec/1000/210/272",
               "PCI/6.5.10"
             ],
             "maturity": "0",
             "accuracy": "0"
           }
         },
         {
           "message": "Inbound Anomaly Score Exceeded (Total Score: 5)",
           "details": {
             "match": "Matched \"Operator `Ge' with parameter `5' against variable `TX:BLOCKING_INBOUND_ANOMALY_SCORE' (Value: `5' )",
             "reference": "",
             "ruleId": "949110",
             "file": "/etc/nginx/owasp-modsecurity-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf",
             "lineNumber": "222",
             "data": "",
             "severity": "0",
             "ver": "OWASP_CRS/4.15.0",
             "rev": "",
             "tags": [
               "anomaly-evaluation",
               "OWASP_CRS"
             ],
             "maturity": "0",
             "accuracy": "0"
           }
         }
       ]
     }
   }
   

3. When using cloud mode i run into the same error documented here: cloud mode test always success #467. All tests pass when they should not.

   👉 executing tests in file
           running 9508991-1: ✔ passed in 2.123826ms (RTT 51.978912ms)
           running 9508991-2: ✔ passed in 1.973332ms (RTT 51.854586ms)
           running 9508991-3: ✔ passed in 1.875197ms (RTT 51.771367ms)
           running 9508991-4: ✔ passed in 1.99529ms (RTT 51.846872ms)
           running 9508991-5: ✔ passed in 1.696723ms (RTT 51.574524ms)
           running 9508991-6: ✔ passed in 2.590194ms (RTT 52.470048ms)
           running 9508991-7: ✔ passed in 6.490357ms (RTT 56.153181ms)
           running 9508991-8: ✔ passed in 4.627064ms (RTT 54.319902ms)
           running 9508991-9: ✔ passed in 3.38364ms (RTT 53.205622ms)
           running 9508991-10: ✔ passed in 2.668685ms (RTT 52.41029ms)
           running 9508991-11: ✔ passed in 2.154914ms (RTT 52.017349ms)
           running 9508991-12: ✔ passed in 2.217447ms (RTT 52.100699ms)
           running 9508991-13: ✔ passed in 2.489357ms (RTT 52.326855ms)
           running 9508991-14: ✔ passed in 1.98747ms (RTT 51.866342ms)
           running 9508991-15: ✔ passed in 2.814477ms (RTT 52.684819ms)
           running 9508991-16: ✔ passed in 5.087162ms (RTT 54.65355ms)
   ➕ run 307 total tests in 879.552759ms
   ⏭ skipped 0 tests
   🎉 All tests successful!
   

Testing method

I have created a Docker Container based off of the official ingress-nginx image on which i install go-ftw and the test files for nextcloud.

FROM registry.k8s.io/ingress-nginx/controller:v1.13.1@sha256:37e489b22ac77576576e52e474941cd7754238438847c1ee795ad6d59c02b12a

USER root
COPY tests tests
COPY ftw.yaml tests/.ftw.yaml
RUN apk add wget tar nano
RUN wget https://github.com/coreruleset/go-ftw/releases/download/v0.6.4/ftw_0.6.4_linux_amd64.tar.gz
RUN tar xvf ftw_0.6.4_linux_amd64.tar.gz
RUN mv ftw tests/ftw

This image is then used as the source for installing ingress-nginx on a kubernetes cluster with the following values:

global:
  image:
    # -- Registry host to pull images from.
    registry: docker.io
controller:
  image:
    image: deserializeme/ingress-nginx-go-ftw
    tag: "0.0.4"
    digest: sha256:3467ddcd8153349ce789bcecae2e11c76d7aa249eb35503dbbf4b0131f1f2187
    digestChroot: ""
  # Mutually exclusive with keda autoscaling
  autoscaling:
    enabled: true
    annotations: {}
    minReplicas: 1
    maxReplicas: 4
    targetCPUUtilizationPercentage: 500
    targetMemoryUtilizationPercentage: 500
    behavior:
      scaleDown:
        stabilizationWindowSeconds: 60
        policies:
          - type: Pods
            value: 1
            periodSeconds: 30
      scaleUp:
        stabilizationWindowSeconds: 60
        policies:
          - type: Pods
            value: 1
            periodSeconds: 30
    ## Define requests resources to avoid probe issues due to CPU utilization in busy nodes
    ## ref: https://github.com/kubernetes/ingress-nginx/issues/4735#issuecomment-551204903
    ## Ideally, there should be no limits.
    ## https://engineering.indeedblog.com/blog/2019/12/cpu-throttling-regression-fix/
    resources:
      limits:
        cpu: 1000m
        memory: 1024Mi
      requests:
        cpu: 100m
        memory: 128Mi
  nodeSelector:
    kubernetes.io/os: linux
    ingress-ready: "true"
  config:
    annotations-risk-level: Critical
    # set log format to json
    log-format-escape-json: "true"
    log-format-upstream: '{"time": "$time_iso8601", "x_forwarded_for": "$proxy_add_x_forwarded_for", "request_id": "$req_id", "bytes_sent": $bytes_sent, "request_time": $request_time, "status": $status, "vhost": "$host", "remote_user": "$remote_user", "request_proto": "$server_protocol", "path": "$uri", "request_query": "$args", "request_length": $request_length, "duration": $request_time,"method": "$request_method", "http_referrer": "$http_referer", "http_user_agent": "$http_user_agent" }'
    # always use 301 instead of a 308 redirect
    http-redirect-code: "301"
    # block all requests with no user-agent to kick out the annoying script kiddies
    block-user-agents: ""
    # show real ip in the logs
    enable-real-ip: "true"
    use-forwarded-headers: "true"
    proxy-real-ip-cidr: "0.0.0.0/0"
    # restrict old versions of tls, could break old browsers/phones
    ssl-protocols: "TLSv1.3"
    # Enables Online Certificate Status Protocol stapling (OCSP) support.
    enable-ocsp: "true"
    # reject SSL handshake to an unknown virtualhost. helps to
    # mitigate the fingerprinting using default certificate of ingress
    ssl-reject-handshake: "true"
    # Enable Modsecurity and the OWASP Core rule set
    enable-modsecurity: "true"
    # Update ModSecurity config and rules
    modsecurity-snippet: |
      # plugins for rule exclusions
      Include /etc/nginx/owasp-modsecurity-crs/plugins/argocd-rule-exclusions-before.conf
      Include /etc/nginx/owasp-modsecurity-crs/plugins/grafana-rule-exclusions-before.conf
      Include /etc/nginx/owasp-modsecurity-crs/plugins/home-assistant-rule-exclusions-before.conf
      Include /etc/nginx/owasp-modsecurity-crs/plugins/matrix-rule-exclusions-before.conf
      Include /etc/nginx/owasp-modsecurity-crs/plugins/postgres-s3-rule-exclusions-before.conf
      Include /etc/nginx/owasp-modsecurity-crs/plugins/zitadel-rule-exclusions-before.conf
      Include /etc/nginx/owasp-modsecurity-crs/plugins/activitypub-rule-exclusions-before.conf
      Include /etc/nginx/owasp-modsecurity-crs/plugins/loki-rule-exclusions-before.conf
      Include /etc/nginx/owasp-modsecurity-crs/plugins/postgres-rule-exclusions-before.conf
      Include /etc/nginx/owasp-modsecurity-crs/plugins/tempo-rule-exclusions-before.conf
      Include /etc/nginx/owasp-modsecurity-crs/plugins/mimir-rule-exclusions-before.conf

      # OWASP Core Rule Set
      Include /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf

      # Enable prevention mode. Can be any of: DetectionOnly,On,Off (default is DetectionOnly)
      SecRuleEngine DetectionOnly

      # Enable scanning of the request body
      SecRequestBodyAccess On

      # Enable XML parsing
      SecRule REQUEST_HEADERS:Content-Type "(?:text|application(?:/soap\+|/)|application/xml)/" \
        "id:200000,phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=XML"

      # Enable JSON parsing
      SecRule REQUEST_HEADERS:Content-Type "application/json" \
        "id:200001,phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=JSON"

      # Note NGINX Ingress has its own annotations, keep in sync!
      #
      # Max request sizes in bytes (with/without files)
      # 10Gb (default is 12.5Mb)
      SecRequestBodyLimit 10740000000

      # 10Gb (default is 128Kb)
      SecRequestBodyNoFilesLimit 10740000000

      # Reject if larger (we could also let it pass with ProcessPartial)
      SecRequestBodyLimitAction Reject

      # Send ModSecurity audit logs to the stdout (only for rejected requests)
      SecAuditLog /var/log/modsec_audit.log

      SecAuditLogFormat JSON

      # could be On/Off/RelevantOnly
      SecAuditEngine RelevantOnly
  # we set the timezone on the pod, to make logs easier to read
  extraEnvs:
    - name: TZ
      value: '{{ .global_time_zone }}'
  extraVolumes:
    - name: plugins
      configMap:
        name: modsecurity-plugins
  extraVolumeMounts:
    - name: plugins
      mountPath: /etc/nginx/owasp-modsecurity-crs/plugins
  # -- This configuration defines if Ingress Controller should allow users to set
  # their own *-snippet annotations, otherwise this is forbidden / dropped
  # when users add those annotations.
  # Global snippets in ConfigMap are still respected
  allowSnippetAnnotations: true
  # you need this after latest helm chart release to allow server snippets
  annotationsRiskLevel: Critical
  resources:
    requests:
      cpu: 100m
      memory: 90Mi
  service:
    enabled: true
    type: LoadBalancer
    externalTrafficPolicy: 'Local'
    external:
      enabled: true
  metrics:
    enabled: true
    serviceMonitor:
      enabled: true
    prometheusRule:
      enabled: false

Once the container is live I can enter it via k exec -it ingress-nginx-controller-85cff45b7-z545c -- /bin/sh then cd to the tests dir and execute ./ftw

[EsadCetiner]

@cloudymax

however this version fails to start on any system I try it on (AMD64, ARM64, Mac)

go-ftw should work for both architectures, can you share what command your running?

When using the latest version of go-ftw in default mode it is unable to find a log-line in either Native or JSON format.

The tests for the Nextcloud plugin v1.4.0 uses an older test format so you won't be able to run those tests with go-ftw 1.x versions. This won't be an issue with future releases of the plugin as all plugins have been migrated to use the new test format.

Also did you follow our documentation on how to run our tests? You need to temporarily add this config for ModSecurity:

SecAction \
    "id:900005,\
    phase:1,\
    nolog,\
    pass,\
    ctl:ruleEngine=DetectionOnly,\
    ctl:ruleRemoveById=910000,\
    setvar:tx.blocking_paranoia_level=4,\
    setvar:tx.crs_validate_utf8_encoding=1,\
    setvar:tx.arg_name_length=100,\
    setvar:tx.arg_length=400,\
    setvar:tx.total_arg_length=64000,\
    setvar:tx.max_num_args=255,\
    setvar:tx.max_file_size=64100,\
    setvar:tx.combined_file_sizes=65535"

SecRule REQUEST_HEADERS:X-CRS-Test "@rx ^.*$" \
    "id:999999,\
    phase:1,\
    pass,\
    t:none,\
    log,\
    msg:'%{MATCHED_VAR}'"

and the following for go-ftw with nginx

[fzipi]

CC: @EsadCetiner

[EsadCetiner]

@fzipi I'm waiting for a response from OP