Skip to content

Conversation

@mergify
Copy link
Contributor

@mergify mergify bot commented Oct 27, 2025

The queue factory setting "queueMaxBytes" used to be a long link:

long queueMaxBytes = RubyFixnum.num2long(args[6]);

But with the refactoring of #18180, it became an Integer by mistake:

.queueMaxBytes(getSetting(context, settings, QUEUE_MAX_BYTES).toJava(Integer.class))

Which prevents queues being bigger than 2^31-1, causing crashes to anyone setting their PQ to 2GB or more:

queue.max_bytes: 2147483647 # 2GB-1 OK
queue.max_bytes: 2147483648 # 2GB FAIL

This change makes queueMaxBytes a Long again, with a test confirming that queues larger than 2ˆ31-1 (max int) can be created.

Without this change the new test fails:

❯ bin/rspec logstash-core/spec/logstash/queue_factory_spec.rb
..F.

Failures:

  1) LogStash::QueueFactory when `queue.type` is `persisted` when queue.max_bytes is larger than Java int does not raise error
     Failure/Error: expect { queue = subject.create(settings) }.to_not raise_error

       expected no Exception, got #<RangeError: too big for int: 2147483648> with backtrace:
         # org/logstash/ackedqueue/QueueFactoryExt.java:97:in `create'
         # org/logstash/ackedqueue/QueueFactoryExt.java:88:in `create'
         # ./logstash-core/spec/logstash/queue_factory_spec.rb:85:in `block in <main>'
         # ./logstash-core/spec/logstash/queue_factory_spec.rb:85:in `block in <main>'
         # ./lib/bootstrap/rspec.rb:36:in `<main>'
     # ./logstash-core/spec/logstash/queue_factory_spec.rb:85:in `block in <main>'
     # ./lib/bootstrap/rspec.rb:36:in `<main>'

Finished in 0.03879 seconds (files took 0.03787 seconds to load)
4 examples, 1 failure

Failed examples:

rspec ./logstash-core/spec/logstash/queue_factory_spec.rb:84 # LogStash::QueueFactory when `queue.type` is `persisted` when queue.max_bytes is larger than Java int does not raise error

But with the change it passes:

❯ bin/rspec logstash-core/spec/logstash/queue_factory_spec.rb -fd

LogStash::QueueFactory
  when `queue.type` is `persisted`
    returns a `WrappedAckedQueue`
    per pipeline id subdirectory creation
      creates a queue directory based on the pipeline id
    when queue.max_bytes is larger than Java int
      does not raise error
  when `queue.type` is `memory`
    returns a `WrappedSynchronousQueue`

Finished in 0.03292 seconds (files took 0.04052 seconds to load)
4 examples, 0 failures
```<hr>This is an automatic backport of pull request #18366 done by [Mergify](https://mergify.com).

…8366)

The queue factory setting "queueMaxBytes" used to be a long [link](https://github.com/elastic/logstash/pull/18180/files#diff-a377851670e06cd113751aa73b726b446dfb721d03d6979b9195afff7facd9f5L60):

```ruby
long queueMaxBytes = RubyFixnum.num2long(args[6]);
```

But with the refactoring of #18180, it [became an Integer by mistake](https://github.com/elastic/logstash/pull/18180/files#diff-01eec670d2beabcd43041ff7fca8bc907eb227471dce886bb757b6414a655429R125):

```java
.queueMaxBytes(getSetting(context, settings, QUEUE_MAX_BYTES).toJava(Integer.class))
```
Which prevents queues being bigger than 2^31-1, causing crashes to anyone setting their PQ to 2GB or more:

```yaml
queue.max_bytes: 2147483647 # 2GB-1 OK
queue.max_bytes: 2147483648 # 2GB FAIL
```

This change makes queueMaxBytes a Long again, with a test confirming that queues larger than 2ˆ31-1 (max int) can be created.

Without this change the new test fails:

```
❯ bin/rspec logstash-core/spec/logstash/queue_factory_spec.rb
..F.

Failures:

  1) LogStash::QueueFactory when `queue.type` is `persisted` when queue.max_bytes is larger than Java int does not raise error
     Failure/Error: expect { queue = subject.create(settings) }.to_not raise_error

       expected no Exception, got #<RangeError: too big for int: 2147483648> with backtrace:
         # org/logstash/ackedqueue/QueueFactoryExt.java:97:in `create'
         # org/logstash/ackedqueue/QueueFactoryExt.java:88:in `create'
         # ./logstash-core/spec/logstash/queue_factory_spec.rb:85:in `block in <main>'
         # ./logstash-core/spec/logstash/queue_factory_spec.rb:85:in `block in <main>'
         # ./lib/bootstrap/rspec.rb:36:in `<main>'
     # ./logstash-core/spec/logstash/queue_factory_spec.rb:85:in `block in <main>'
     # ./lib/bootstrap/rspec.rb:36:in `<main>'

Finished in 0.03879 seconds (files took 0.03787 seconds to load)
4 examples, 1 failure

Failed examples:

rspec ./logstash-core/spec/logstash/queue_factory_spec.rb:84 # LogStash::QueueFactory when `queue.type` is `persisted` when queue.max_bytes is larger than Java int does not raise error
```

But with the change it passes:

```
❯ bin/rspec logstash-core/spec/logstash/queue_factory_spec.rb -fd

LogStash::QueueFactory
  when `queue.type` is `persisted`
    returns a `WrappedAckedQueue`
    per pipeline id subdirectory creation
      creates a queue directory based on the pipeline id
    when queue.max_bytes is larger than Java int
      does not raise error
  when `queue.type` is `memory`
    returns a `WrappedSynchronousQueue`

Finished in 0.03292 seconds (files took 0.04052 seconds to load)
4 examples, 0 failures
```

(cherry picked from commit 6c48e51)
@github-actions
Copy link
Contributor

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

@jsvd jsvd merged commit a0e5260 into 9.2 Oct 27, 2025
11 of 12 checks passed
@jsvd jsvd deleted the mergify/bp/9.2/pr-18366 branch October 27, 2025 11:07
@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

cc @jsvd

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants