sFlow v5 header ethertype encoding issues on Cisco ASR-9903 with IOS-XR 7.9.2 #1002
Description
Hello!
We got very curious sFlow packet dump from customer which FastNetMon was not able to parse.
Platform was following: Cisco ASR-9903 with IOS-XR 7.9.2
Configuration:
flow exporter-map SFLOW-EXPORTER-MAP-IPV4
dfbit set
packet-length 1400
version sflow v5
!
transport udp 6343
source-address y.y.y.y
source Loopback0
destination x.x.x.x
!
flow monitor-map SFLOW-MONITOR-MAP
record sflow
sflow options
input ifindex physical
output ifindex physical
if-counters polling-interval 20
extended-router
extended-gateway
!
exporter SFLOW-EXPORTER-MAP-IPV4
!
sampler-map USS-1000
random 1 out-of 1000
!
FastNetMon's parser error was following:
2024-02-02 12:39:13,491 [DEBUG] sflow: Cannot parse nested packet using ng parser: unknown_ethertype
2024-02-02 12:39:13,491 [DEBUG] sflow: Cannot parse nested packet using ng parser: unknown_ethertype
2024-02-02 12:39:13,491 [DEBUG] sflow: Cannot parse nested packet using ng parser: unknown_ethertype
In details it was more curious that ethertype was different all the time and we found 1170 unique ethertypes:
./fastnetmon_pcap_reader sflow ~/sflow_lol_xr_data.pcap |grep Ethertype|awk '{print $2}'| head -n 10
613c
217d
197d
2178
1138
2179
213e
11fb
213d
113c
Total number:
./fastnetmon_pcap_reader sflow ~/sflow_lol_xr_data.pcap |grep Ethertype|awk '{print $2}'|sort -g | uniq |wc -l
1170
How it looks in Wireshark:
What even more curious that we noticed that right after this obviously incorrect ethertype proper IPv4 header follows.
We slightly altered our parser to print frist byte which follow after ethertype and got following results:
./fastnetmon_pcap_reader sflow ~/sflow_lol_xr_data.pcap |grep Version|sort | uniq -c
214835 Version: 4
26 Version: 6
So it clearly confirms that remaining part of packet header is intact and our parser was able to parse it with ethertype value set to IPv4 manually:
{"agent_address":"0.0.0.0","destination_asn":0,"destination_country":"","destination_ip":"xx","destination_mac":"f4:xx:91","destination_port":53,"forwarding_status":"unknown","fragmentation":false,"input_interface":123,"ip_length":66,"ip_version":"ipv4","length":0,"output_interface":116,"packets":1,"protocol":"udp","sample_ratio":1000,"source_asn":0,"source_country":"","source_ip":"xx","source_mac":"44:xx:de","source_port":16681,"timestamp":0,"ttl":252}
Finally: Cisco XR has some bug which corrupts byte which stores ethertype and replaces it by some other value which appears to be random.