Mime type reload v2.1 #13689
Closed
Mime type reload v2.1 #13689
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Following commit will add a new value that will overflow the actual size. Ticket: 7816
File magic is known to have performance issue. When looking for an alternative, I've been pointed to the tree_magic_mini crate that output the mime type of a binary stream. This is different from magic but it has multiple advantages as it is a standard so it can be correlated with other tools. So instead of replacing magic, this patch adds a new mime type output to fileinfo events. This patch also adds the `file.mimetype` sticky buffer. Mime type has 2 advantages over file magic. First it is really faster and second, the result are easier to use as the MIME type are well defined. But it provides less information than magic for example with regards to the size of images. Ticket: 7816
This patch adds a flag `--enable-gpl-mimetype` to force the usage of `tree_magic_db` that is GPL license and will not allow double licensing. This will allow the built Suricata to have mime type identification not dependant of the operating system database. Ticket: 7816
This can be used with the requires keyword to avoid signatures load failure. Ticket: 7816
As file data may have not been inspected yet, this is going to miss in the event even if it could be available. Let's compute it before logging the file. Ticket: 7816
Only build the tree_magic_mini crate if needed.
regit
requested review from
jasonish,
jufajardini,
victorjulien and
a team
as code owners
|
I'm missing a clear explanation here and in the docs of what the practical effect of with and w/o the gpl data means for the matching capabilities and the output. |
It makes sense. I will document in the user doc how to use it without embedded data. |
jasonish
reviewed
Aug 5, 2025
| - run: tar xf prep/suricata-verify.tar.gz | ||
| - run: ./autogen.sh | ||
| - run: ./configure --enable-unittests --enable-coccinelle | ||
| - run: ./configure --enable-unittests --enable-coccinelle --enable-mimetype --enable-bundled-gpl-mimetype |
There was a problem hiding this comment.
--enable-bundled-gpl-mimetype doesn't exist in this PR.
jasonish
reviewed
Aug 5, 2025
| - run: ./scripts/bundle.sh | ||
| - run: ./autogen.sh | ||
| - run: ./configure --enable-warnings | ||
| - run: ./configure --enable-warnings --enable-mimetype --enable-bundled-gpl-mimetype |
There was a problem hiding this comment.
--enable-bundled-gpl-mimetype doesn't exist in this PR.
jasonish
reviewed
Aug 5, 2025
|
|
||
| static int DetectFileMimetypeSetupNoSupport(DetectEngineCtx *de_ctx, Signature *s, const char *str) | ||
| { | ||
| SCLogError("no libmagic support built in, needed for filemagic keyword"); |
There was a problem hiding this comment.
libmagic -> mimetype? Or something like that.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Update of #13647 addressing comments
Contribution style:
https://docs.suricata.io/en/latest/devguide/contributing/contribution-process.html
Our Contribution agreements:
https://suricata.io/about/contribution-agreement/ (note: this is only required once)
Changes (if applicable):
(including schema descriptions)
https://redmine.openinfosecfoundation.org/projects/suricata/issues
Link to ticket: https://redmine.openinfosecfoundation.org/issues/7816
Describe changes:
SV_BRANCH=OISF/suricata-verify#2606