Glue Table for WAF Access Logs is missing some of the log fields (e.g. labels)

[m-davison]

Is your feature request related to a problem? Please describe.
The Glue Table created for the GlueWafAccessLogsTable does not contain the newer fields for:

• labels
• captcharesponse
• challengeresponse
• ja3Fingerprint

These are useful fields for querying and debugging rules, especially the labels struct.

Describe the feature you'd like
Add the missing fields to the GlueWafAccessLogsTable resource in the aws-waf-security-automations-firehose-athena.template so we don't need to customize the solution in order to get the extra fields when debugging rules.

Additional context
See Creating a table for AWS WAF logs without partitioning for full set of fields

[WillAWS]

Hi @m-davison , thanks for your enhancement request. Could you provide the details of your use case and how you want to query those fields, etc.?

[m-davison]

The key field I'm after at the moment is labels. Whilst I don't need that it to be used by the lambdas, I do use the waf_access_logs table to debug rules I've setup or look for patterns during / after an attack to see how best to design a rule.

For example, if I have one rule setup to add a label to any web based UAs where the atp token is missing, and I want to see after a login issue which of the login requests had that label associated to it, or other labels e.g. standard ATP or BOT labels that I could potentially use.

Plus I think It would just be good to have access to all the data in the WAF access logs, rather than just a subset so I don't need to either customise the solution or create a second table for debugging purposes.

[WillAWS]

@m-davison Currently this is not in the scope of the out-of-box solution as the main purpose of the log analysis feature is to automate detecting and blocking common attacks via a built-in Lambda or Athena log parser using necessary fields instead of bringing the whole data. We can add your request to our backlog, but in general if you have specific needs like this, we recommend you to customize the source code to get fast resolution.

[m-davison]

No worries... Thanks @WillAWS ... I have customised it, just thought I'd raise it in case it could be of use to others too

[bios6]

Hi, Thanks for raising this issue, we will add this to our backlog and look into this issue