Skip to content

Using SIMD in a Fluent Bit Container #10215

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
jinyongchoi opened this issue Apr 15, 2025 · 7 comments · May be fixed by #10225
Open

Using SIMD in a Fluent Bit Container #10215

jinyongchoi opened this issue Apr 15, 2025 · 7 comments · May be fixed by #10225

Comments

@jinyongchoi
Copy link
Contributor

jinyongchoi commented Apr 15, 2025 •
edited
Loading

Hello,
According to the documentation below, SIMD is supposed to be enabled by default in the container.
However, in my testing, it appears that SIMD is not actually enabled.
I'm curious to know under what conditions SIMD gets enabled in a container environment.
Below are the results from testing with a Docker container image.

Thanks!

Image

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 165
model name	: Intel(R) Core(TM) i5-10400 CPU @ 2.90GHz
stepping	: 5
microcode	: 0xfc
cpu MHz		: 4000.011
cache size	: 12288 KB
physical id	: 0
siblings	: 12
core id		: 5
cpu cores	: 6
apicid		: 11
initial apicid	: 11
fpu		: yes
fpu_exception	: yes
cpuid level	: 22
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts vnmi pku ospke md_clear flush_l1d arch_capabilities
vmx flags	: vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple shadow_vmcs pml ept_mode_based_exec
bugs		: spectre_v1 spectre_v2 spec_store_bypass swapgs itlb_multihit srbds mmio_stale_data retbleed eibrs_pbrsb gds bhi
bogomips	: 5799.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 39 bits physical, 48 bits virtual

> docker run -p 127.0.0.1:24224:24224 fluent/fluent-bit:4.0.0 /fluent-bit/bin/fluent-bit -i forward -o stdout -p format=json_lines -f 1
Fluent Bit v4.0.0
* Copyright (C) 2015-2025 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

______ _                  _    ______ _ _             ___  _____ 
|  ___| |                | |   | ___ (_) |           /   ||  _  |
| |_  | |_   _  ___ _ __ | |_  | |_/ /_| |_  __   __/ /| || |/' |
|  _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / /_| ||  /| |
| |   | | |_| |  __/ | | | |_  | |_/ / | |_   \ V /\___  |\ |_/ /
\_|   |_|\__,_|\___|_| |_|\__| \____/|_|\__|   \_/     |_(_)___/ 


[2025/04/15 01:45:59] [ info] [fluent bit] version=4.0.0, commit=712b48d7c3, pid=1
[2025/04/15 01:45:59] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2025/04/15 01:45:59] [ info] [simd    ] disabled
[2025/04/15 01:45:59] [ info] [cmetrics] version=0.9.9
[2025/04/15 01:45:59] [ info] [ctraces ] version=0.6.2

> docker run -p 127.0.0.1:24224:24224 fluent/fluent-bit:4.0.0-amd64 /fluent-bit/bin/fluent-bit -i forward -o stdout -p format=json_lines -f 1
Fluent Bit v4.0.0
* Copyright (C) 2015-2025 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

______ _                  _    ______ _ _             ___  _____ 
|  ___| |                | |   | ___ (_) |           /   ||  _  |
| |_  | |_   _  ___ _ __ | |_  | |_/ /_| |_  __   __/ /| || |/' |
|  _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / /_| ||  /| |
| |   | | |_| |  __/ | | | |_  | |_/ / | |_   \ V /\___  |\ |_/ /
\_|   |_|\__,_|\___|_| |_|\__| \____/|_|\__|   \_/     |_(_)___/ 


[2025/04/15 01:50:23] [ info] [fluent bit] version=4.0.0, commit=712b48d7c3, pid=1
[2025/04/15 01:50:23] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2025/04/15 01:50:23] [ info] [simd    ] disabled
[2025/04/15 01:50:23] [ info] [cmetrics] version=0.9.9
[2025/04/15 01:50:23] [ info] [ctraces ] version=0.6.2


> docker run -p 127.0.0.1:24224:24224 fluent/fluent-bit:3.2.0-amd64 /fluent-bit/bin/fluent-bit -i forward -o stdout -p format=json_lines -f 1
Fluent Bit v3.2.0
* Copyright (C) 2015-2024 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

______ _                  _    ______ _ _           _____  _____ 
|  ___| |                | |   | ___ (_) |         |____ |/ __  \
| |_  | |_   _  ___ _ __ | |_  | |_/ /_| |_  __   __   / /`' / /'
|  _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / /   \ \  / /  
| |   | | |_| |  __/ | | | |_  | |_/ / | |_   \ V /.___/ /./ /___
\_|   |_|\__,_|\___|_| |_|\__| \____/|_|\__|   \_/ \____(_)_____/


[2025/04/15 01:58:38] [ info] [fluent bit] version=3.2.0, commit=50512ed56a, pid=1
[2025/04/15 01:58:38] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2025/04/15 01:58:38] [ info] [simd    ] disabled
[2025/04/15 01:58:38] [ info] [cmetrics] version=0.9.9
[2025/04/15 01:58:38] [ info] [ctraces ] version=0.5.7
@patrick-stephens
Copy link
Contributor

It doesn't appear to actually be enabled in the container definition, FLB_SIMD needs to be turned on as it is disabled by default.

@jinyongchoi
Copy link
Contributor Author

@patrick-stephens
The documentation I referred to is below:
https://docs.fluentbit.io/manual/administration/performance#enabling-simd-support

Is there any plan to enable this by default in future container releases?

@patrick-stephens
Copy link
Contributor

It "just" needs enabling via a PR I think but will defer to @leonardo-albertovich or @edsiper if there is anything else required

@jinyongchoi
Copy link
Contributor Author

Got it!.
I'll wait for @leonardo-albertovich or @edsiper to weigh in.
Thanks!

@patrick-stephens
Copy link
Contributor

Or send a PR to test it?

@jinyongchoi
Copy link
Contributor Author

Sure, I will try it.

jinyongchoi added a commit to jinyongchoi/fluent-bit that referenced this issue Apr 17, 2025
@jinyongchoi
dockerfile: Enable default SIMD support in Docker (fluent#10215)
d3103fb 
According to the documentation,
SIMD is supposed to be enabled by default in Fluent Bit container images.

This commit updates the Dockerfile to reflect that, enabling SIMD by default.
@jinyongchoi jinyongchoi linked a pull request Apr 17, 2025 that will close this issue
Open
jinyongchoi added a commit to jinyongchoi/fluent-bit that referenced this issue Apr 17, 2025
@jinyongchoi
dockerfile: Enable default SIMD support in Docker (fluent#10215)
12620cb 
According to the documentation,
SIMD is supposed to be enabled by default in Fluent Bit container images.

This commit updates the Dockerfile to reflect that, enabling SIMD by default.

Signed-off-by: jinyong.choi <inimax801@gmail.com>
jinyongchoi added a commit to jinyongchoi/fluent-bit that referenced this issue Apr 17, 2025
@jinyongchoi
dockerfile: Enable default SIMD support in Docker (fluent#10215)
06d8967 
According to the documentation,
SIMD is supposed to be enabled by default in Fluent Bit container images.

This commit updates the Dockerfile to reflect that, enabling SIMD by default.

Signed-off-by: jinyong.choi <inimax801@gmail.com>
@jinyongchoi
Copy link
Contributor Author

I think it makes sense to wait until this ticket (#10178) is completed before proceeding.
I’ll hold off for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

dockerfile: Enable default SIMD support in Docker (#10215) jinyongchoi/fluent-bit
2 participants
@patrick-stephens @jinyongchoi