Skip to content

Conversation

dmitry-ganyushin
Copy link
Collaborator

6060: Add load w/ compression parameters to AlignAndFocusPowderFromFiles

Description of work

Added an additional parameter AlignAndFocusPowderFromFiles to determine whether its call to LoadEventNexus should enable compressing events during load. The parameter is called "MinSizeCompressOnLoad" which is a floating point number of Gbytes.
.

Copy link
Member

@peterfpeterson peterfpeterson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This missed the mark. The request was "AlignAndFocusPowderFromFiles needs additional parameter(s) to determine whether its call to LoadEventNexus should enable compressing events during load." When it is determined that the file should be compressed on load, the parameters CompressTolerance and CompressBinningMode need to be supplied to LoadEventNexus. The __compressEvents() method is called by the __accummlate() method which is run after a chunk is processed. These methods should not be modified.

This is also missing release notes.

@dmitry-ganyushin dmitry-ganyushin self-assigned this Aug 19, 2024
Copy link
Member

@peterfpeterson peterfpeterson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried this in three modes:

  • without the CompressTolerance parameter - did not compress on load
  • with the CompressTolerance > filesize - did not compress on load
  • with the CompressTolerance < filesize - did compress on load

@AndreiSavici AndreiSavici merged commit 4b0abd2 into main Aug 21, 2024
@AndreiSavici AndreiSavici deleted the compression-options branch August 21, 2024 19:41
@dmitry-ganyushin dmitry-ganyushin restored the compression-options branch August 21, 2024 20:39
@dmitry-ganyushin dmitry-ganyushin deleted the compression-options branch August 21, 2024 23:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants