feat(results): improve human readability of semantic validation results

[ChristopherJHart]

Currently, if nac-validate receives one or more results from one or more rules, all of the results from a given rule are lumped into a single, long, comma-separated string. An example of this is shown below:

ERROR - Semantic error, rule 311: Verify TCP or UDP protocol is specified for non-well-known ports in filters (['apic.tenants[EXAMPLE].filters[BACKUP_PORTS].entries[TCP_8400] - Port fields contain non-well-known port(s) (destination_from_port=8400) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.', 'apic.tenants[EXAMPLE].filters[BACKUP_PORTS].entries[TCP_8401] - Port fields contain non-well-known port(s) (destination_from_port=8401) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.', 'apic.tenants[EXAMPLE].filters[BACKUP_PORTS].entries[TCP_8408] - Port fields contain non-well-known port(s) (destination_from_port=8408) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.', 'apic.tenants[EXAMPLE].filters[BACKUP_PORTS].entries[TCP_2987] - Port fields contain non-well-known port(s) (destination_from_port=2987) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.', 'apic.tenants[EXAMPLE].filters[BACKUP_PORTS].entries[TCP_8403] - Port fields contain non-well-known port(s) (destination_from_port=8403) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.', 'apic.tenants[EXAMPLE].filters[BACKUP_PORTS].entries[TCP_8405] - Port fields contain non-well-known port(s) (destination_from_port=8405) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.', 'apic.tenants[EXAMPLE].filters[SMB].entries[TCP_445] - Port fields contain non-well-known port(s) (destination_from_port=445) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.', 'apic.tenants[EXAMPLE].filters[SMB-S].entries[TCP_445] - Port fields contain non-well-known port(s) (source_from_port=445) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.', 'apic.tenants[EXAMPLE].filters[ESX_PORTS].entries[TCP_2049] - Port fields contain non-well-known port(s) (destination_from_port=2049) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.', 'apic.tenants[EXAMPLE].filters[ESX_PORTS].entries[TCP_902] - Port fields contain non-well-known port(s) (destination_from_port=902) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.', 'apic.tenants[EXAMPLE].filters[RDP-S].entries[TCP_3389] - Port fields contain non-well-known port(s) (source_from_port=3389) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.', 'apic.tenants[EXAMPLE].filters[NetBIOS].entries[TCP_139] - Port fields contain non-well-known port(s) (destination_from_port=139) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.', 'apic.tenants[EXAMPLE].filters[NetBIOS-S].entries[TCP_139] - Port fields contain non-well-known port(s) (source_from_port=445) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.', 'apic.tenants[EXAMPLE].filters[RDP].entries[RDP-TCP] - Port fields contain non-well-known port(s) (destination_from_port=3389) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.', 'apic.tenants[FILTER_TEST_FAIL].filters[CUSTOM_PORT_NO_PROTOCOL].entries[PORT_8080_NO_PROTO] - Port fields contain non-well-known port(s) (destination_from_port=8080) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.', 'apic.tenants[FILTER_TEST_FAIL].filters[SRC_CUSTOM_NO_PROTOCOL].entries[SRC_9000_NO_PROTO] - Port fields contain non-well-known port(s) (source_from_port=9000) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.', "apic.tenants[FILTER_TEST_FAIL].filters[CUSTOM_PORT_ICMP].entries[ICMP_WITH_PORT] - Port fields contain non-well-known port(s) (destination_from_port=8080) but protocol is 'icmp' which is not valid for port-based filtering. Protocol must be 'tcp' or 'udp' when port fields are used.", 'apic.tenants[FILTER_TEST_FAIL].filters[PORT_RANGE_NO_PROTOCOL].entries[RANGE_NO_PROTO] - Port fields contain non-well-known port(s) (destination_from_port=5000, destination_to_port=6000) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.', 'apic.tenants[FILTER_TEST_FAIL].filters[MIXED_PORTS_NO_PROTOCOL].entries[MIXED] - Port fields contain non-well-known port(s) (destination_from_port=9999) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.', "apic.tenants[FILTER_TEST_FAIL].filters[BOTH_RULES_FAIL].entries[BAD_FILTER] - Port fields contain non-well-known port(s) (destination_to_port=5000) but protocol is 'igmp' which is not valid for port-based filtering. Protocol must be 'tcp' or 'udp' when port fields are used.", 'apic.tenants[TEST].filters[PERMIT_TCP_81].entries[TCP-81] - Port fields contain non-well-known port(s) (destination_from_port=81) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.'])
ERROR - Semantic error, rule 310: Verify filter port ranges are valid (to_port requires from_port, from <= to) (['apic.tenants[FILTER_TEST_FAIL].filters[MISSING_DST_FROM_PORT].entries[TCP_TO_443] - destination_to_port is specified (443) but destination_from_port is missing. destination_from_port must be specified to define a valid port range.', 'apic.tenants[FILTER_TEST_FAIL].filters[MISSING_SRC_FROM_PORT].entries[TCP_TO_8080] - source_to_port is specified (8080) but source_from_port is missing. source_from_port must be specified to define a valid port range.', 'apic.tenants[FILTER_TEST_FAIL].filters[INVALID_DST_PORT_RANGE].entries[TCP_INVALID_RANGE] - destination_from_port (9000) is greater than destination_to_port (8000). From port must be less than or equal to to port.', 'apic.tenants[FILTER_TEST_FAIL].filters[INVALID_SRC_PORT_RANGE].entries[TCP_INVALID_SRC_RANGE] - source_from_port (65535) is greater than source_to_port (1024). From port must be less than or equal to to port.', 'apic.tenants[FILTER_TEST_FAIL].filters[BOTH_RULES_FAIL].entries[BAD_FILTER] - destination_to_port is specified (5000) but destination_from_port is missing. destination_from_port must be specified to define a valid port range.', 'apic.tenants[chart2].filters[chart2_test_https].entries[chart2_test_https_entry] - destination_to_port is specified (https) but destination_from_port is missing. destination_from_port must be specified to define a valid port range.', 'apic.tenants[chart2].filters[chart2_test_dns].entries[chart2_test_dns_entry] - source_to_port is specified (53) but source_from_port is missing. source_from_port must be specified to define a valid port range.'])

This can be difficult for a human to read and methodically take action on each violation, as it is not intuitive where one violation ends and the next begins.

This PR modifies the result formatting such that each result from a given rule is broken out into a more human-readable list, as shown below:

ERROR - Semantic error, rule 311: Verify TCP or UDP protocol is specified for non-well-known ports in filters:
    - apic.tenants[EXAMPLE].filters[BACKUP_PORTS].entries[TCP_8400] - Port fields contain non-well-known port(s) (destination_from_port=8400) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.
    - apic.tenants[EXAMPLE].filters[BACKUP_PORTS].entries[TCP_8401] - Port fields contain non-well-known port(s) (destination_from_port=8401) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.
    - apic.tenants[EXAMPLE].filters[BACKUP_PORTS].entries[TCP_8408] - Port fields contain non-well-known port(s) (destination_from_port=8408) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.
    - apic.tenants[EXAMPLE].filters[BACKUP_PORTS].entries[TCP_2987] - Port fields contain non-well-known port(s) (destination_from_port=2987) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.
    - apic.tenants[EXAMPLE].filters[BACKUP_PORTS].entries[TCP_8403] - Port fields contain non-well-known port(s) (destination_from_port=8403) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.
    - apic.tenants[EXAMPLE].filters[BACKUP_PORTS].entries[TCP_8405] - Port fields contain non-well-known port(s) (destination_from_port=8405) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.
    - apic.tenants[EXAMPLE].filters[SMB].entries[TCP_445] - Port fields contain non-well-known port(s) (destination_from_port=445) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.
    - apic.tenants[EXAMPLE].filters[SMB-S].entries[TCP_445] - Port fields contain non-well-known port(s) (source_from_port=445) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.
    - apic.tenants[EXAMPLE].filters[ESX_PORTS].entries[TCP_2049] - Port fields contain non-well-known port(s) (destination_from_port=2049) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.
    - apic.tenants[EXAMPLE].filters[ESX_PORTS].entries[TCP_902] - Port fields contain non-well-known port(s) (destination_from_port=902) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.
    - apic.tenants[EXAMPLE].filters[RDP-S].entries[TCP_3389] - Port fields contain non-well-known port(s) (source_from_port=3389) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.
    - apic.tenants[EXAMPLE].filters[NetBIOS].entries[TCP_139] - Port fields contain non-well-known port(s) (destination_from_port=139) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.
    - apic.tenants[EXAMPLE].filters[NetBIOS-S].entries[TCP_139] - Port fields contain non-well-known port(s) (source_from_port=445) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.
    - apic.tenants[EXAMPLE].filters[RDP].entries[RDP-TCP] - Port fields contain non-well-known port(s) (destination_from_port=3389) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.
    - apic.tenants[FILTER_TEST_FAIL].filters[CUSTOM_PORT_NO_PROTOCOL].entries[PORT_8080_NO_PROTO] - Port fields contain non-well-known port(s) (destination_from_port=8080) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.
    - apic.tenants[FILTER_TEST_FAIL].filters[SRC_CUSTOM_NO_PROTOCOL].entries[SRC_9000_NO_PROTO] - Port fields contain non-well-known port(s) (source_from_port=9000) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.
    - apic.tenants[FILTER_TEST_FAIL].filters[CUSTOM_PORT_ICMP].entries[ICMP_WITH_PORT] - Port fields contain non-well-known port(s) (destination_from_port=8080) but protocol is 'icmp' which is not valid for port-based filtering. Protocol must be 'tcp' or 'udp' when port fields are used.
    - apic.tenants[FILTER_TEST_FAIL].filters[PORT_RANGE_NO_PROTOCOL].entries[RANGE_NO_PROTO] - Port fields contain non-well-known port(s) (destination_from_port=5000, destination_to_port=6000) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.
    - apic.tenants[FILTER_TEST_FAIL].filters[MIXED_PORTS_NO_PROTOCOL].entries[MIXED] - Port fields contain non-well-known port(s) (destination_from_port=9999) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.
    - apic.tenants[FILTER_TEST_FAIL].filters[BOTH_RULES_FAIL].entries[BAD_FILTER] - Port fields contain non-well-known port(s) (destination_to_port=5000) but protocol is 'igmp' which is not valid for port-based filtering. Protocol must be 'tcp' or 'udp' when port fields are used.
    - apic.tenants[TEST].filters[PERMIT_TCP_81].entries[TCP-81] - Port fields contain non-well-known port(s) (destination_from_port=81) but protocol is not specified. Best practice dictates that protocol (tcp/udp) be explicitly defined when using non-well-known port numbers.
ERROR - Semantic error, rule 310: Verify filter port ranges are valid (to_port requires from_port, from <= to):
    - apic.tenants[FILTER_TEST_FAIL].filters[MISSING_DST_FROM_PORT].entries[TCP_TO_443] - destination_to_port is specified (443) but destination_from_port is missing. destination_from_port must be specified to define a valid port range.
    - apic.tenants[FILTER_TEST_FAIL].filters[MISSING_SRC_FROM_PORT].entries[TCP_TO_8080] - source_to_port is specified (8080) but source_from_port is missing. source_from_port must be specified to define a valid port range.
    - apic.tenants[FILTER_TEST_FAIL].filters[INVALID_DST_PORT_RANGE].entries[TCP_INVALID_RANGE] - destination_from_port (9000) is greater than destination_to_port (8000). From port must be less than or equal to to port.
    - apic.tenants[FILTER_TEST_FAIL].filters[INVALID_SRC_PORT_RANGE].entries[TCP_INVALID_SRC_RANGE] - source_from_port (65535) is greater than source_to_port (1024). From port must be less than or equal to to port.
    - apic.tenants[FILTER_TEST_FAIL].filters[BOTH_RULES_FAIL].entries[BAD_FILTER] - destination_to_port is specified (5000) but destination_from_port is missing. destination_from_port must be specified to define a valid port range.
    - apic.tenants[chart2].filters[chart2_test_https].entries[chart2_test_https_entry] - destination_to_port is specified (https) but destination_from_port is missing. destination_from_port must be specified to define a valid port range.
    - apic.tenants[chart2].filters[chart2_test_dns].entries[chart2_test_dns_entry] - source_to_port is specified (53) but source_from_port is missing. source_from_port must be specified to define a valid port range.

[aitestino]

Hi @ChristopherJHart,

I've opened a draft PR (#363) that addresses the readability improvements you proposed here. The implementation uses a pattern-based approach that automatically detects and colorizes different output elements (headers, separators, bullet points, etc.) without hardcoding rule-specific logic.

This allows rules to define their own rich output formats with sections like "WHY THIS MATTERS" and "RECOMMENDED FIX" while getting consistent terminal colorization.

Would love your feedback on the approach. Screenshots will be added to the PR shortly.

[ChristopherJHart]

Closing this PR as its being superseded by #363