Skip to content

Regression: Run as user nobody:nogroup was broken in logstash:7.17.8 (logstash:7.17.7 works as expected) #14836

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
lioncubs opened this issue Jan 10, 2023 · 4 comments · May be fixed by #17546
Open

Regression: Run as user nobody:nogroup was broken in logstash:7.17.8 (logstash:7.17.7 works as expected) #14836

lioncubs opened this issue Jan 10, 2023 · 4 comments · May be fixed by #17546
Labels
bug status:confirmed Team:Logstash

Comments

@lioncubs
Copy link

Logstash information:

  1. docker.elastic.co/logstash/logstash:7.17.8
  2. Run directly from the default docker containers
  3. Run in Kubernetes with appropriate config - can reproduce with no config as well
root@kind:~# docker run -u nobody:nogroup -v $PWD/data:/usr/share/logstash/data --rm -ti docker.elastic.co/logstash/logstash:7.17.8 bash
nobody@5255f955baef:/usr/share/logstash$ bin/logstash --version
Using bundled JDK: /usr/share/logstash/jdk
logstash 7.17.8

root@kind:~# docker run -u nobody:nogroup -v $PWD/data:/usr/share/logstash/data --rm -ti docker.elastic.co/logstash/logstash:7.17.7 bash
nobody@4f6e2456591f:/usr/share/logstash$ bin/logstash --version
Using bundled JDK: /usr/share/logstash/jdk
logstash 7.17.7

Plugins installed: (bin/logstash-plugin list --verbose)

AS in from docker image:

root@kind:~# docker run -u nobody:nogroup -v $PWD/data:/usr/share/logstash/data --rm -ti docker.elastic.co/logstash/logstash:7.17.7 bash
nobody@d919a69b950a:/usr/share/logstash$ bin/logstash-plugin list --verbose
Using bundled JDK: /usr/share/logstash/jdk
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
logstash-codec-avro (3.3.1)
logstash-codec-cef (6.2.5)
logstash-codec-collectd (3.1.0)
logstash-codec-dots (3.0.6)
logstash-codec-edn (3.1.0)
logstash-codec-edn_lines (3.1.0)
logstash-codec-es_bulk (3.1.0)
logstash-codec-fluent (3.4.1)
logstash-codec-graphite (3.0.6)
logstash-codec-json (3.1.1)
logstash-codec-json_lines (3.1.0)
logstash-codec-line (3.1.1)
logstash-codec-msgpack (3.1.0)
logstash-codec-multiline (3.1.1)
logstash-codec-netflow (4.2.2)
logstash-codec-plain (3.1.0)
logstash-codec-rubydebug (3.1.0)
logstash-filter-aggregate (2.10.0)
logstash-filter-anonymize (3.0.6)
logstash-filter-cidr (3.1.3)
logstash-filter-clone (4.2.0)
logstash-filter-csv (3.1.1)
logstash-filter-date (3.1.15)
logstash-filter-de_dot (1.0.4)
logstash-filter-dissect (1.2.5)
logstash-filter-dns (3.1.5)
logstash-filter-drop (3.0.5)
logstash-filter-elasticsearch (3.11.1)
logstash-filter-fingerprint (3.3.2)
logstash-filter-geoip (7.2.12)
logstash-filter-grok (4.4.2)
logstash-filter-http (1.2.1)
logstash-filter-json (3.2.0)
logstash-filter-kv (4.5.0)
logstash-filter-memcached (1.1.0)
logstash-filter-metrics (4.0.7)
logstash-filter-mutate (3.5.6)
logstash-filter-prune (3.0.4)
logstash-filter-ruby (3.1.8)
logstash-filter-sleep (3.0.7)
logstash-filter-split (3.1.8)
logstash-filter-syslog_pri (3.1.1)
logstash-filter-throttle (4.0.4)
logstash-filter-translate (3.3.1)
logstash-filter-truncate (1.0.5)
logstash-filter-urldecode (3.0.6)
logstash-filter-useragent (3.3.3)
logstash-filter-uuid (3.0.5)
logstash-filter-xml (4.1.3)
logstash-input-azure_event_hubs (1.4.4)
logstash-input-beats (6.2.6)
└── logstash-input-elastic_agent (alias)
logstash-input-couchdb_changes (3.1.6)
logstash-input-dead_letter_queue (1.1.12)
logstash-input-elasticsearch (4.12.3)
logstash-input-exec (3.4.0)
logstash-input-file (4.4.4)
logstash-input-ganglia (3.1.4)
logstash-input-gelf (3.3.2)
logstash-input-generator (3.1.0)
logstash-input-graphite (3.0.6)
logstash-input-heartbeat (3.1.1)
logstash-input-http (3.4.5)
logstash-input-http_poller (5.1.0)
logstash-input-imap (3.2.0)
logstash-input-jms (3.2.2)
logstash-input-pipe (3.1.0)
logstash-input-redis (3.7.0)
logstash-input-s3 (3.8.3)
logstash-input-snmp (1.3.1)
logstash-input-snmptrap (3.1.0)
logstash-input-sqs (3.1.3)
logstash-input-stdin (3.4.0)
logstash-input-syslog (3.6.0)
logstash-input-tcp (6.2.7)
logstash-input-twitter (4.1.0)
logstash-input-udp (3.5.0)
logstash-input-unix (3.1.2)
logstash-integration-elastic_enterprise_search (2.1.2)
 ├── logstash-output-elastic_app_search
 └──  logstash-output-elastic_workplace_search
logstash-integration-jdbc (5.2.6)
 ├── logstash-input-jdbc
 ├── logstash-filter-jdbc_streaming
 └── logstash-filter-jdbc_static
logstash-integration-kafka (10.9.0)
 ├── logstash-input-kafka
 └── logstash-output-kafka
logstash-integration-rabbitmq (7.3.1)
 ├── logstash-input-rabbitmq
 └── logstash-output-rabbitmq
logstash-output-cloudwatch (3.0.10)
logstash-output-csv (3.0.8)
logstash-output-elasticsearch (11.4.1)
logstash-output-email (4.1.1)
logstash-output-file (4.3.0)
logstash-output-graphite (3.1.6)
logstash-output-http (5.2.5)
logstash-output-lumberjack (3.1.9)
logstash-output-nagios (3.0.6)
logstash-output-null (3.0.5)
logstash-output-pipe (3.0.6)
logstash-output-redis (5.0.0)
logstash-output-s3 (4.3.7)
logstash-output-sns (4.0.8)
logstash-output-sqs (6.0.0)
logstash-output-stdout (3.1.4)
logstash-output-tcp (6.0.2)
logstash-output-udp (3.2.0)
logstash-output-webhdfs (3.0.6)
logstash-patterns-core (4.3.4)


root@kind:~# docker run -u nobody:nogroup -v $PWD/data:/usr/share/logstash/data --rm -ti docker.elastic.co/logstash/logstash:7.17.8 bash
nobody@1db7966df365:/usr/share/logstash$ bin/logstash-plugin list --verbose
Using bundled JDK: /usr/share/logstash/jdk
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
logstash-codec-avro (3.3.1)
logstash-codec-cef (6.2.6)
logstash-codec-collectd (3.1.0)
logstash-codec-dots (3.0.6)
logstash-codec-edn (3.1.0)
logstash-codec-edn_lines (3.1.0)
logstash-codec-es_bulk (3.1.0)
logstash-codec-fluent (3.4.1)
logstash-codec-graphite (3.0.6)
logstash-codec-json (3.1.1)
logstash-codec-json_lines (3.1.0)
logstash-codec-line (3.1.1)
logstash-codec-msgpack (3.1.0)
logstash-codec-multiline (3.1.1)
logstash-codec-netflow (4.2.2)
logstash-codec-plain (3.1.0)
logstash-codec-rubydebug (3.1.0)
logstash-filter-aggregate (2.10.0)
logstash-filter-anonymize (3.0.6)
logstash-filter-cidr (3.1.3)
logstash-filter-clone (4.2.0)
logstash-filter-csv (3.1.1)
logstash-filter-date (3.1.15)
logstash-filter-de_dot (1.0.4)
logstash-filter-dissect (1.2.5)
logstash-filter-dns (3.1.5)
logstash-filter-drop (3.0.5)
logstash-filter-elasticsearch (3.11.1)
logstash-filter-fingerprint (3.3.2)
logstash-filter-geoip (7.2.12)
logstash-filter-grok (4.4.3)
logstash-filter-http (1.2.1)
logstash-filter-json (3.2.0)
logstash-filter-kv (4.5.0)
logstash-filter-memcached (1.1.0)
logstash-filter-metrics (4.0.7)
logstash-filter-mutate (3.5.6)
logstash-filter-prune (3.0.4)
logstash-filter-ruby (3.1.8)
logstash-filter-sleep (3.0.7)
logstash-filter-split (3.1.8)
logstash-filter-syslog_pri (3.1.1)
logstash-filter-throttle (4.0.4)
logstash-filter-translate (3.3.1)
logstash-filter-truncate (1.0.5)
logstash-filter-urldecode (3.0.6)
logstash-filter-useragent (3.3.3)
logstash-filter-uuid (3.0.5)
logstash-filter-xml (4.1.3)
logstash-input-azure_event_hubs (1.4.4)
logstash-input-beats (6.2.6)
└── logstash-input-elastic_agent (alias)
logstash-input-couchdb_changes (3.1.6)
logstash-input-dead_letter_queue (1.1.12)
logstash-input-elasticsearch (4.12.3)
logstash-input-exec (3.4.0)
logstash-input-file (4.4.4)
logstash-input-ganglia (3.1.4)
logstash-input-gelf (3.3.2)
logstash-input-generator (3.1.0)
logstash-input-graphite (3.0.6)
logstash-input-heartbeat (3.1.1)
logstash-input-http (3.4.5)
logstash-input-http_poller (5.1.0)
logstash-input-imap (3.2.0)
logstash-input-jms (3.2.2)
logstash-input-pipe (3.1.0)
logstash-input-redis (3.7.0)
logstash-input-s3 (3.8.3)
logstash-input-snmp (1.3.1)
logstash-input-snmptrap (3.1.0)
logstash-input-sqs (3.1.3)
logstash-input-stdin (3.4.0)
logstash-input-syslog (3.6.0)
logstash-input-tcp (6.2.7)
logstash-input-twitter (4.1.0)
logstash-input-udp (3.5.0)
logstash-input-unix (3.1.2)
logstash-integration-elastic_enterprise_search (2.1.2)
 ├── logstash-output-elastic_app_search
 └──  logstash-output-elastic_workplace_search
logstash-integration-jdbc (5.2.6)
 ├── logstash-input-jdbc
 ├── logstash-filter-jdbc_streaming
 └── logstash-filter-jdbc_static
logstash-integration-kafka (10.9.0)
 ├── logstash-input-kafka
 └── logstash-output-kafka
logstash-integration-rabbitmq (7.3.1)
 ├── logstash-input-rabbitmq
 └── logstash-output-rabbitmq
logstash-output-cloudwatch (3.0.10)
logstash-output-csv (3.0.8)
logstash-output-elasticsearch (11.4.1)
logstash-output-email (4.1.1)
logstash-output-file (4.3.0)
logstash-output-graphite (3.1.6)
logstash-output-http (5.2.5)
logstash-output-lumberjack (3.1.9)
logstash-output-nagios (3.0.6)
logstash-output-null (3.0.5)
logstash-output-pipe (3.0.6)
logstash-output-redis (5.0.0)
logstash-output-s3 (4.3.7)
logstash-output-sns (4.0.8)
logstash-output-sqs (6.0.0)
logstash-output-stdout (3.1.4)
logstash-output-tcp (6.0.3)
logstash-output-udp (3.2.0)
logstash-output-webhdfs (3.0.6)
logstash-patterns-core (4.3.4)

JVM (e.g. java -version):

  • Bundled!

OS version (uname -a if on a Unix-like system):

Any Linux version - tested on the following

root@kind:~# uname -a
Linux kind 5.15.0-33-generic #34~20.04.1-Ubuntu SMP Thu May 19 15:51:16 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Description of the problem including expected versus actual behavior:

  • Running logstash:7.17.8 with plain vanilla config SPECIFICALLY User nobody:nogroup (which is a security requirement for us) will cause logstash to crash - running in default (root) will run as expected.
  • Running logstash:7.17.7 as above runs in all scenarios as expected

Steps to reproduce:

Success with 7.17.7

  1. create an empty 'data' folder in your filesystem.
  2. chown nobody data
  3. chmod -R ugo+rw data
  4. docker run -u nobody:nogroup -v $PWD/data:/usr/share/logstash/data --rm -ti --entrypoint bash docker.elastic.co/logstash/logstash:7.17.7
  5. logstash -e 'input { stdin { } } output { stdout {} }'

Failure with 7.17.8

  1. create an empty 'data' folder in your filesystem.
  2. chown nobody data
  3. chmod -R ugo+rw data
  4. docker run -u nobody:nogroup -v $PWD/data:/usr/share/logstash/data --rm -ti --entrypoint bash docker.elastic.co/logstash/logstash:7.17.8
  5. logstash -e 'input { stdin { } } output { stdout {} }'
  6. Observe Failure

7.17.8 Failure - using nobody:nogroup

# 7.17.8 Failure - using nobody:nogroup
mkdir data.7.17.8
chown nobody data.7.17.8
chmod -R ugo+rw data.7.17.8
docker run -u nobody:nogroup -v $PWD/data.7.17.8:/usr/share/logstash/data --rm -ti  --entrypoint bash docker.elastic.co/logstash/logstash:7.17.8

nobody@0a35bff2d197:/usr/share/logstash$ logstash -e 'input { stdin { } } output { stdout {} }'
Using bundled JDK: /usr/share/logstash/jdk
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[FATAL] 2023-01-10 18:30:31.009 [main] Logstash - Logstash stopped processing because of an error: (LoadError) no such file to load -- logstash/build
org.jruby.exceptions.LoadError: (LoadError) no such file to load -- logstash/build
        at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:974) ~[jruby-complete-9.2.20.1.jar:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.polyglot_minus_0_dot_3_dot_5.lib.polyglot.require(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/polyglot-0.3.5/lib/polyglot.rb:65) ~[?:?]
        at RUBY.<main>(/usr/share/logstash/logstash-core/lib/logstash/api/commands/system/basicinfo_command.rb:20) ~[?:?]
        at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:974) ~[jruby-complete-9.2.20.1.jar:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.polyglot_minus_0_dot_3_dot_5.lib.polyglot.require(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/polyglot-0.3.5/lib/polyglot.rb:65) ~[?:?]
        at RUBY.<main>(/usr/share/logstash/logstash-core/lib/logstash/api/command_factory.rb:19) ~[?:?]
        at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:974) ~[jruby-complete-9.2.20.1.jar:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.polyglot_minus_0_dot_3_dot_5.lib.polyglot.require(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/polyglot-0.3.5/lib/polyglot.rb:65) ~[?:?]
        at RUBY.<main>(/usr/share/logstash/logstash-core/lib/logstash/api/modules/base.rb:19) ~[?:?]
        at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:974) ~[jruby-complete-9.2.20.1.jar:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.polyglot_minus_0_dot_3_dot_5.lib.polyglot.require(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/polyglot-0.3.5/lib/polyglot.rb:65) ~[?:?]
        at RUBY.<main>(/usr/share/logstash/logstash-core/lib/logstash/api/rack_app.rb:20) ~[?:?]
        at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:974) ~[jruby-complete-9.2.20.1.jar:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.polyglot_minus_0_dot_3_dot_5.lib.polyglot.require(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/polyglot-0.3.5/lib/polyglot.rb:65) ~[?:?]
        at RUBY.<main>(/usr/share/logstash/logstash-core/lib/logstash/webserver.rb:18) ~[?:?]
        at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:974) ~[jruby-complete-9.2.20.1.jar:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.polyglot_minus_0_dot_3_dot_5.lib.polyglot.require(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/polyglot-0.3.5/lib/polyglot.rb:65) ~[?:?]
        at RUBY.<main>(/usr/share/logstash/logstash-core/lib/logstash/agent.rb:23) ~[?:?]
        at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:974) ~[jruby-complete-9.2.20.1.jar:?]
        at RUBY.<main>(/usr/share/logstash/logstash-core/lib/logstash/runner.rb:44) ~[?:?]
        at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:974) ~[jruby-complete-9.2.20.1.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:92) ~[?:?]
nobody@0a35bff2d197:/usr/share/logstash$ exit
exit
root@kind:~#

Success with 7.17.7 - using nobody:nogroup - CTRL-C to stop

# Success with 7.17.7 - using nobody:nogroup - CTRL-C to stop
mkdir data.7.17.7
chown nobody data.7.17.7
chmod -R ugo+rw data.7.17.7
docker run -u nobody:nogroup -v $PWD/data.7.17.7:/usr/share/logstash/data --rm -ti  --entrypoint bash  docker.elastic.co/logstash/logstash:7.17.7

nobody@d47a3308a495:/usr/share/logstash$ logstash -e 'input { stdin { } } output { stdout {} }'
Using bundled JDK: /usr/share/logstash/jdk
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties
[2023-01-10T18:50:03,183][INFO ][logstash.runner          ] Log4j configuration path used is: /usr/share/logstash/config/log4j2.properties
[2023-01-10T18:50:03,200][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.17.7", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.16+8 on 11.0.16+8 +indy +jit [linux-x86_64]"}
[2023-01-10T18:50:03,204][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djdk.io.File.enableADS=true, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -Djruby.regexp.interruptible=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true]
[2023-01-10T18:50:03,254][INFO ][logstash.settings        ] Creating directory {:setting=>"path.queue", :path=>"/usr/share/logstash/data/queue"}
[2023-01-10T18:50:03,274][INFO ][logstash.settings        ] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/usr/share/logstash/data/dead_letter_queue"}
[2023-01-10T18:50:03,806][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2023-01-10T18:50:03,851][INFO ][logstash.agent           ] No persistent UUID file found. Generating new UUID {:uuid=>"6f060474-0c58-48ab-80dd-39a196200fc1", :path=>"/usr/share/logstash/data/uuid"}
[2023-01-10T18:50:05,320][WARN ][logstash.monitoringextension.pipelineregisterhook] xpack.monitoring.enabled has not been defined, but found elasticsearch configuration. Please explicitly set `xpack.monitoring.enabled: true` in logstash.yml
[2023-01-10T18:50:05,323][WARN ][deprecation.logstash.monitoringextension.pipelineregisterhook] Internal collectors option for Logstash monitoring is deprecated and may be removed in a future release.
Please configure Metricbeat to monitor Logstash. Documentation can be found at: 
https://www.elastic.co/guide/en/logstash/current/monitoring-with-metricbeat.html
[2023-01-10T18:50:05,746][WARN ][deprecation.logstash.codecs.plain] Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
[2023-01-10T18:50:05,813][WARN ][deprecation.logstash.outputs.elasticsearch] Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
[2023-01-10T18:50:06,128][INFO ][logstash.licensechecker.licensereader] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elasticsearch:9200/]}}
[2023-01-10T18:50:06,268][INFO ][logstash.licensechecker.licensereader] Failed to perform request {:message=>"elasticsearch: Temporary failure in name resolution", :exception=>Manticore::ResolutionFailure, :cause=>java.net.UnknownHostException: elasticsearch: Temporary failure in name resolution}
[2023-01-10T18:50:06,273][WARN ][logstash.licensechecker.licensereader] Attempted to resurrect connection to dead ES instance, but got an error {:url=>"http://elasticsearch:9200/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :message=>"Elasticsearch Unreachable: [http://elasticsearch:9200/][Manticore::ResolutionFailure] elasticsearch: Temporary failure in name resolution"}
[2023-01-10T18:50:06,308][INFO ][logstash.licensechecker.licensereader] Failed to perform request {:message=>"elasticsearch", :exception=>Manticore::ResolutionFailure, :cause=>java.net.UnknownHostException: elasticsearch}
[2023-01-10T18:50:06,313][WARN ][logstash.licensechecker.licensereader] Marking url as dead. Last error: [LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [http://elasticsearch:9200/_xpack][Manticore::ResolutionFailure] elasticsearch {:url=>http://elasticsearch:9200/, :error_message=>"Elasticsearch Unreachable: [http://elasticsearch:9200/_xpack][Manticore::ResolutionFailure] elasticsearch", :error_class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError"}
[2023-01-10T18:50:06,322][WARN ][logstash.licensechecker.licensereader] Attempt to validate Elasticsearch license failed. Sleeping for 0.02 {:fail_count=>1, :exception=>"Elasticsearch Unreachable: [http://elasticsearch:9200/_xpack][Manticore::ResolutionFailure] elasticsearch"}
[2023-01-10T18:50:06,348][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"No Available connections"}
[2023-01-10T18:50:06,385][ERROR][logstash.monitoring.internalpipelinesource] Failed to fetch X-Pack information from Elasticsearch. This is likely due to failure to reach a live Elasticsearch cluster.
[2023-01-10T18:50:06,651][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
^C[2023-01-10T18:50:07,948][WARN ][logstash.runner          ] SIGINT received. Shutting down.
[2023-01-10T18:50:08,748][INFO ][org.reflections.Reflections] Reflections took 158 ms to scan 1 urls, producing 119 keys and 419 values 
[2023-01-10T18:50:09,867][WARN ][deprecation.logstash.codecs.line] Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
[2023-01-10T18:50:09,902][WARN ][deprecation.logstash.inputs.stdin] Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
[2023-01-10T18:50:10,486][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>16, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>2000, "pipeline.sources"=>["config string"], :thread=>"#<Thread:0x2fd5fd2a run>"}
^C[2023-01-10T18:50:11,231][FATAL][logstash.runner          ] SIGINT received. Terminating immediately..
[2023-01-10T18:50:11,382][FATAL][org.logstash.Logstash    ] 
org.jruby.exceptions.ThreadKill: null
nobody@d47a3308a495:/usr/share/logstash$ exit
exit
root@kind:~#

Running 7.17.8 as root works - no need to show full output (same as with 7.17.7)

# Running 7.17.8 as root works - no need to show full output
mkdir data.7.17.8.root
chmod -R ugo+rw data.7.17.8.root
docker run -v $PWD/data.7.17.8.root:/usr/share/logstash/data --rm -ti  --entrypoint bash docker.elastic.co/logstash/logstash:7.17.8
logstash@d0cc92175a9e:~$ logstash -e 'input { stdin { } } output { stdout {} }'
Using bundled JDK: /usr/share/logstash/jdk
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties
...

Provide logs (if relevant):
@lioncubs lioncubs changed the title Regression 7.17.8 as opposed to 7.17.7 when running with user nobody:nogroup Regression: Run as user nobody:nogroup was broken in logstash:7.17.8 (logstash:7.17.7 works as expected) Jan 11, 2023
@lioncubs
Copy link
Author

Found the issue - in backtrace

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[FATAL] 2023-01-10 18:30:31.009 [main] Logstash - Logstash stopped processing because of an error: (LoadError) no such file to load -- logstash/build
org.jruby.exceptions.LoadError: (LoadError) no such file to load -- logstash/build

The file to load that fails logstash/build.rb not logstash/build

The Permissions for the following file had been changed from 7.17.7 -> 7.17.8

  • /usr/share/logstash/logstash-core/lib/logstash/build.rb
# logstash:7.17.7 - File is OK
root@kind:/home/radware/git/waas/waas_backend/docker-images/logstash# docker run -u nobody:nogroup -v $PWD/data.7.17.7:/usr/share/logstash/data --rm -ti  --entrypoint bash docker.elastic.co/logstash/logstash:7.17.7

nobody@4b9d4f571e35:/usr/share/logstash$ ls -al ./logstash-core/lib/logstash/build.rb 
-rw-rw-r-- 1 logstash root 156 Oct 13 13:24 ./logstash-core/lib/logstash/build.rb
nobody@4b9d4f571e35:/usr/share/logstash$ #Verify that this is the ONLY file that is not other+r
nobody@4b9d4f571e35:/usr/share/logstash$ find . ! -perm /o+r
nobody@4b9d4f571e35:/usr/share/logstash$ 

# logstash:7.17.8 - File is NOT OK
root@kind:/home/radware/git/waas/waas_backend/docker-images/logstash# docker run -u nobody:nogroup -v $PWD/data.7.17.8:/usr/share/logstash/data --rm -ti  --entrypoint bash docker.elastic.co/logstash/logstash:7.17.8

nobody@45c2639679b8:/usr/share/logstash$  ls -al ./logstash-core/lib/logstash/build.rb
-rw-rw---- 1 logstash root 156 Nov 30 16:11 ./logstash-core/lib/logstash/build.rb
nobody@45c2639679b8:/usr/share/logstash$ #Verify that this is the ONLY file that is not other+r
nobody@45c2639679b8:/usr/share/logstash$ find . ! -perm /o+r
./logstash-core/lib/logstash/build.rb
nobody@45c2639679b8:/usr/share/logstash$

Currently we have patched our docker image from docker.elastic.co/logstash/logstash:7.17.8 with the following changes, and all works - So there is your Fix :slight_smile:

Dockerfile

# Get Original Docker image from elastic.co
FROM docker.elastic.co/logstash/logstash:7.17.8

# fix permission issue on /usr/share/logstash/logstash-core/lib/logstash/build.rb in 7.17.8 (also exists in 8.6.0)
RUN chmod 0664 /usr/share/logstash/logstash-core/lib/logstash/build.rb

NOTE: I have also verified that this issue ALSO exists with 8.6.0 - So I assume any changes to 8.x since Nov 30 16:11 also has the same issue - Please make the same change in the 8.x Branch

@kaisecheng
Copy link
Contributor

The same issue was found in 8.5.1 but not in 8.5.0
diff of two versions

@kaisecheng
Copy link
Contributor

build.rb is created from temp file from rake artifact:generate_build_metadata. The temp file has permissions 0600. In the docker build process, chmod and chown change the permission of user and group. I do not see any change related to others. It is unclear to me why v8.5.0 has -rw-rw-r--, while v8.5.1 has -rw-rw----.

The fix will be update Dockerfile.j2 to add read permission to others

@kaisecheng
Copy link
Contributor

btw, I have tested locally with v8.5.0 and run the docker image build process, but the build.rb is missing read permission.

@bval bval linked a pull request Apr 10, 2025 that will close this issue
Open
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug status:confirmed Team:Logstash
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

chmod BUILD_METADATA_FILE to allow global read permissions bval/logstash
3 participants
@lioncubs @roaksoax @kaisecheng