Skip to content

test/entropy: Update checks to include sid #2588

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

test/entropy: Update checks to include sid #2588

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion tests/entropy/entropy-01/test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
requires:
min-version: 8
version: 8.0.0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this change ? Should it not work in 8.0.1 ?


checks:
- filter:
Expand Down
2 changes: 2 additions & 0 deletions tests/entropy/entropy-02/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
This test checks the entropy keyword with a comparison against HTTP file data and ensures
that the logging name is composed of the sticky buffer and sid.
14 changes: 14 additions & 0 deletions tests/entropy/entropy-02/test.rules
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# The entropy value is 4.137370175000773
alert http any any -> any any (msg:"entropy simple test"; file.data; entropy: value > 4; sid:1;)
alert http any any -> any any (msg:"entropy simple test"; file.data; entropy: value >= 4; sid:2;)
alert http any any -> any any (msg:"entropy simple test"; file.data; entropy: value >= 5; sid:3;)
alert http any any -> any any (msg:"entropy simple test"; file.data; entropy: value 4-5; sid:4;)
alert http any any -> any any (msg:"entropy simple test"; file.data; entropy: value !4-5; sid:5;)
alert http any any -> any any (msg:"entropy simple test"; file.data; entropy: value < 4; sid:6;)
alert http any any -> any any (msg:"entropy simple test"; file.data; entropy: value != 4; sid:7;)
alert http any any -> any any (msg:"entropy simple test"; file.data; entropy: value = 4; sid:8;)
# The entropy value is 4.150007324019584
alert http any any -> any any (msg:"entropy simple test"; file.data; entropy: offset 10, value > 4.14; sid:10;)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I understand the ticket here, but

  • a flow may have multiple txs
  • a tx may have multiple files (for example I could write a signature to detect a HTTP POST with 2 files : one having a low entropy, and the other file having a high entropy)

So, is it good to combine sid and sticky buffer name ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The entropy value is saved as a flow-related variable -- using VarNameStorageRegister -- into a global namespace. The sid is being added to "scope" the name to a rule.

I think you're saying that there needs to be further scoping when entropy is evaluated over multiple transactions?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I fear flow+sticky buffer+sid is not unique enough

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great analysis.
I suppose we could use a combination of file.md5 and the entropy to make it unique?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I am not sure I totally get the point : do you just want to have logged with an alert the entropy(s) that matched ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes -- the idea is to log the calculated entropy value -- not only in the alert but in all places where the flow is logged.

Consider a rule with mult. entropy statements --- we'd like to know the calculated entropy values of each so they can be adjusted if needed

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not only in the alert but in all places where the flow is logged.

It does not make much sense to me to log in a flow event, some value that belongs to a transaction

And even then, each transaction may have multi-buffers (like dns requests) and you would want to log these different values...

I feel this needs more design, instead of reusing what is available in the engine even if that is not fit...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Considering this affects output, I tend to agree with Jeff. Could we maybe take some time in the next dev meeting for this, or open a thread to discuss and propose?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated the entropy PR to use <sticky_buffer> where tracks the entropy keyword usage; where 1 is for the first occurrence, 2 for the 2nd, and so on.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it work for a packet having multiple transactions ?


# Multiple entropy usages
alert http any any -> any any (msg:"entropy simple test"; http.response_body;entropy: value < 5; file.data; entropy: offset 10, value > 4.14; sid:11;)
89 changes: 89 additions & 0 deletions tests/entropy/entropy-02/test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
requires:
min-version: 8.0.1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs update... 9 ?


pcap: ../entropy-01/input.pcap

args:
- --set logging.entropy.make-unique=on

checks:
- filter:
count: 1
match:
event_type: alert
alert.signature_id: 1
metadata.entropy.sid:1;buffer:file_data;instance:1: 4.137370175000773
- filter:
count: 1
match:
event_type: alert
alert.signature_id: 2
metadata.entropy.sid:2;buffer:file_data;instance:1: 4.137370175000773
- filter:
count: 0
match:
event_type: alert
alert.signature_id: 3
- filter:
count: 1
match:
event_type: alert
alert.signature_id: 4
metadata.entropy.sid:4;buffer:file_data;instance:1: 4.137370175000773
- filter:
count: 0
match:
event_type: alert
alert.signature_id: 5
- filter:
count: 0
match:
event_type: alert
alert.signature_id: 6
- filter:
count: 1
match:
event_type: alert
alert.signature_id: 7
metadata.entropy.sid:7;buffer:file_data;instance:1: 4.137370175000773
- filter:
count: 0
match:
event_type: alert
alert.signature_id: 8
- filter:
count: 1
match:
event_type: alert
alert.signature_id: 10
metadata.entropy.sid:10;buffer:file_data;instance:1: 4.150007324019584
- filter:
count: 1
match:
event_type: alert
alert.signature_id: 10
metadata.entropy.sid:10;buffer:file_data;instance:1: 4.150007324019584
- filter:
count: 1
match:
event_type: alert
alert.signature_id: 11
metadata.entropy.sid:11;buffer:file_data;instance:1: 4.137370175000773
metadata.entropy.sid:11;buffer:file_data;instance:2: 4.150007324019584
- filter:
count: 1
match:
event_type: flow
src_ip: 10.92.95.2
dest_ip: 10.92.67.138
flow.pkts_toserver: 5
flow.pkts_toclient: 5
metadata.entropy.sid:1;buffer:file_data;instance:1: 4.137370175000773
metadata.entropy.sid:2;buffer:file_data;instance:1: 4.137370175000773
metadata.entropy.sid:3;buffer:file_data;instance:1: 4.137370175000773
metadata.entropy.sid:4;buffer:file_data;instance:1: 4.137370175000773
metadata.entropy.sid:5;buffer:file_data;instance:1: 4.137370175000773
metadata.entropy.sid:6;buffer:file_data;instance:1: 4.137370175000773
metadata.entropy.sid:7;buffer:file_data;instance:1: 4.137370175000773
metadata.entropy.sid:8;buffer:file_data;instance:1: 4.137370175000773
metadata.entropy.sid:10;buffer:file_data;instance:1: 4.150007324019584
Loading