Skip to content

v1.4.0

Latest
Latest
Compare
Choose a tag to compare
@greensh16 greensh16 released this 18 Aug 08:00
v1.4.0
554b744

[1.4.0] - 2025-08-18

Major Features Added

Memory Limiting System

  • Memory usage limits with --memory-limit MB option for resource-constrained environments
  • Real-time memory monitoring using RSS (Resident Set Size) tracking
  • Graceful degradation - automatically disables caching when approaching 95% of memory limit
  • Early termination - stops scanning when memory limit is exceeded to prevent system issues
  • Platform-aware monitoring - bypasses limits gracefully on platforms without RSS support
  • Configurable check intervals with hidden --memory-check-interval-ms option for fine-tuning

HPC Cluster Support

  • Memory-conscious scanning designed for High-Performance Computing environments
  • Job scheduler integration with examples for SLURM, PBS/Torque, and LSF
  • Resource-constrained operation that respects allocated memory limits
  • Batch job compatibility with conservative memory usage patterns

Enhanced Features

Memory Management

  • Intelligent cache disabling when memory pressure is detected
  • Partial result handling when scans are terminated early due to memory limits
  • Memory status reporting in scan results with MemoryLimitStatus enum
  • Cross-platform compatibility with fallback behavior on unsupported systems

CLI Improvements

  • New --memory-limit option for setting memory usage limits in megabytes
  • Enhanced help text with clear memory limiting documentation
  • Memory status output showing when limits are approached or exceeded
  • Profile integration showing memory usage alongside performance metrics

Performance Improvements

Memory Efficiency

  • Reduced memory allocations when operating under memory constraints
  • Optimized data structures for memory-limited environments
  • Throttled memory checks to minimize monitoring overhead (default: 200ms intervals)
  • Smart caching decisions based on available memory headroom

Resource Management

  • Thread pool optimization when memory limits are active
  • Incremental scanning with memory-aware cache management
  • Early exit strategies to prevent resource exhaustion
  • Memory-conscious progress reporting with reduced overhead

Use Cases and Examples

HPC Integration

# SLURM job with 2GB memory allocation
#SBATCH --mem=2G
rudu /lustre/project --memory-limit 1800 --threads 4

# PBS job with conservative memory usage
#PBS -l mem=1gb
rudu /data --memory-limit 900 --no-cache

# Memory-constrained deep scan
rudu /filesystem --memory-limit 256 --depth 5 --profile

Memory Monitoring Behavior

Memory Usage System Behavior
< 95% limit Normal operation with all features enabled
95-100% limit Disables caching, reduces memory allocations
> 100% limit Terminates scan early, returns partial results
Platform unsupported Disables monitoring, continues normally

Documentation Updates

  • New "Memory Limiting for HPC Clusters" section in README
  • Comprehensive usage examples for different HPC schedulers
  • Best practices guide for memory-constrained environments
  • Platform compatibility matrix for memory monitoring support
  • Integration examples with SLURM, PBS, and LSF job schedulers

CLI Options

  • --memory-limit MB - Set memory usage limit in megabytes
  • --memory-check-interval-ms MS - Hidden option for tuning check frequency

Backward Compatibility

  • Full backward compatibility with all existing command-line options
  • No breaking changes to existing APIs or output formats
  • Optional memory limiting - all existing workflows continue to work unchanged
  • Graceful fallback on platforms without memory monitoring support

Bug Fixes and Stability Improvements

Caching System Fixes

  • Cache test reliability improvements with better test isolation and cleanup
  • Cache file handling robustness improvements for edge cases
  • Memory-mapped cache stability enhancements
  • Cache invalidation logic fixes for better reliability

Code Quality and Linting

  • Clippy warnings resolved - all code now passes strict linting requirements
  • Code formatting standardized across all modules and benchmarks
  • Benchmark consistency improvements across all performance tests
  • Example code cleaned up and validated

CI/CD Pipeline Improvements

  • GitHub Actions workflow optimization for faster CI runs
  • Test reliability improvements with better resource management
  • Build process streamlining and dependency management

Documentation and Examples

  • New tutorial documentation added in docs/basic-usage.md
  • Comprehensive exclusion guide in docs/exclude_tutorial.md with 490+ lines of examples
  • Memory monitor demo example showing practical memory limiting usage
  • Cache disable demo for testing memory-constrained environments
  • Enhanced benchmarking with new overhead benchmark suite
Assets 4
Loading