why microsteam occupy 58G main memory?

[useryq8]

Environment Details

MicroStream Version:08.01.00-MS-GA
JDK version:21.01
OS:Ubuntu 18.04.6 LTS, centos 7
Used frameworks: Spring boot 2.7.8

Describe the bug

it sees the directly buffer is not released for every
@fh-ms @fh-ms could you help see this problem? or there is some setting we can do limit the direct memory size. Thank you very much!

java.lang.OutOfMemoryError: Cannot reserve 4096 bytes of direct buffer memory (allocated: 25769800004, limit: 25769803776) at java.base/java.nio.Bits.reserveMemory(Bits.java:178) at java.base/java.nio.DirectByteBuffer.(DirectByteBuffer.java:127) at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:360) at one.microstream.memory.XMemory.allocateDirectNative(XMemory.java:1078) at one.microstream.memory.XMemory.allocateDirectNative(XMemory.java:1085) at one.microstream.persistence.binary.types.ChunksBuffer.(ChunksBuffer.java:92) at one.microstream.persistence.binary.types.ChunksBuffer.New(ChunksBuffer.java:56) at one.microstream.persistence.binary.types.BinaryStorer$Default.synchCreateStoringChunksBuffers(BinaryStorer.java:282) at one.microstream.persistence.binary.types.BinaryStorer$Default.internalInitialize(BinaryStorer.java:263) at one.microstream.persistence.binary.types.BinaryStorer$Default.defaultInitialize(BinaryStorer.java:250) at one.microstream.persistence.binary.types.BinaryStorer$Default.(BinaryStorer.java:161) at one.microstream.persistence.binary.types.BinaryStorer$Creator$Default.createLazyStorer(BinaryStorer.java:961) at one.microstream.persistence.binary.types.BinaryStorer$Creator.createStorer(BinaryStorer.java:875) at one.microstream.persistence.binary.types.BinaryStorer$Creator.createStorer(BinaryStorer.java:855) at one.microstream.persistence.types.PersistenceManager$Default.createStorer(PersistenceManager.java:253) at one.microstream.persistence.types.PersistenceManager$Default.store(PersistenceManager.java:300) at one.microstream.storage.types.StorageConnection.store(StorageConnection.java:401) at one.microstream.storage.types.Database$Default.store(Database.java:173)

To Reproduce

user LazyArrayList and write data, remove data by multi threads(200+), the machine has 220G main memory, the jvm heap is set 64G, no other program runs on this machine, the total count of the lazyArrayList item is about 30,000, the total size is about 300M

Expected behavior

A clear and concise description of what you expected to happen.

Screenshots

Additional context

Add any other context about the problem here.

[hg-ms]

At the first glance it seems that the housekeeping is not aggressive enough.

1. Check if items removed from the list are no more referenced so that they can be delete by the java GC. If not, the storage will keep the cached.
2. You can try to use the ‘adaptive’ StorageHousekeepingController:

final EmbeddedStorageConfigurationBuilder builder = EmbeddedStorageConfigurationBuilder.New();
	builder.setHousekeepingAdaptive(true);
	...
	final EmbeddedStorageFoundation<?> foundation = builder.createEmbeddedStorageFoundation();				
	final EmbeddedStorageManager storage = foundation.createEmbeddedStorageManager().start();

3. Try a manual house keeping configuration

• Decrease entity-cache-threshold
• Decrease entity-cache-timeout
• Increase housekeeping-time-budget
  See https://docs.microstream.one/manual/storage/configuration/housekeeping.html

[useryq8]

At the first glance it seems that the housekeeping is not aggressive enough.

1. Check if items removed from the list are no more referenced so that they can be delete by the java GC. If not, the storage will keep the cached.
2. You can try to use the ‘adaptive’ StorageHousekeepingController:

final EmbeddedStorageConfigurationBuilder builder = EmbeddedStorageConfigurationBuilder.New();
	builder.setHousekeepingAdaptive(true);
	...
	final EmbeddedStorageFoundation<?> foundation = builder.createEmbeddedStorageFoundation();				
	final EmbeddedStorageManager storage = foundation.createEmbeddedStorageManager().start();

3. Try a manual house keeping configuration

• Decrease entity-cache-threshold
• Decrease entity-cache-timeout
• Increase housekeeping-time-budget
  See https://docs.microstream.one/manual/storage/configuration/housekeeping.html
  @fh-ms Thanks your reply.

My current setting is like this , Could you give some suggestion？ I used eclipseStore 1.1.0 and java21

    var builder = org.eclipse.store.storage.embedded.configuration.types.EmbeddedStorageConfiguration.Builder();
    builder.setHousekeepingAdaptive(true);

    var foundation = builder
      .setStorageDirectory(graphDbPath)
      .setChannelCount(1)
      .setDataFileMaximumSize(org.eclipse.serializer.configuration.types.ByteSize.New(2,
                                                                                      org.eclipse.serializer.configuration.types.ByteUnit.GB))
      .setHousekeepingInterval(Duration.of(5, ChronoUnit.SECONDS))
      .setHousekeepingTimeBudget(Duration.of(3, ChronoUnit.SECONDS))
      .createEmbeddedStorageFoundation();

    foundation.onConnectionFoundation(org.eclipse.serializer.persistence.binary.jdk8.types.BinaryHandlersJDK8::registerJDK8TypeHandlers);

    var manager = foundation.createEmbeddedStorageManager().start();
    manager.persistenceManager().objectRegistry().ensureCapacity( 1024 * 1024 * 10);
    return manager;

[hg-ms]

I’d try to:
decrease the HouseKeepingIntervall, to run it more often (default is once per second)
increase the setHousekeepingTimeBudget carefully (default is 10ms)
decrease the EntityCacheTimeout (default is 1 Day)

.setHousekeepingInterval(Duration.of(100, ChronoUnit.MILLIS))
.setHousekeepingTimeBudget(Duration.of(50, ChronoUnit.MILLIS))
.setEntityCacheTimeout(Duration.of(1, ChronoUnit.MINUTES))

With this setup housekeeping will be done every 100ms for a 50 ms max and clear cache items that are older than one minute.