Memory leak using the brush tool

[AdanMora]

Hi,

In addition to #150, there is a significant performance issue when you use the brush tool, it takes all the computer's resources, I have 16GB RAM and when you use the tool, increase the usage up to 8GB and it continues increasing rapidly.

I found that the problem could be that there is no something like a garbage collector who deletes from memory all jobs related to the brush and the coco-annotator en general. When you works con coco-annotator, then you close it or just go to the main page, the memory is still on usage, then if you continue working it increases again the memory usage, it should be something to kill processes when you stop working on a image, like when you save an image or go to another page or image, etc. Also when you stop using coco fro an arbitrary time (like 5 minutes) it could clean the sockets automatically.

Thanks.

[koenvandesande]

My browser runs out of memory when I use the brush tool on a 1920x1080 image, and drag it around a lot. Like Adan says, the memory usage seems to increase over time. But I can get it to run out of memory by drawing with the brush in a single image.

[jsbroks]

The performance issue is caused by PaperJS, it is beyond the scope of this project. The Boolean operations can not process fast enough for large compounds.

[AdanMora]

I don't know exactly how did you implemented that part, but did you check if there is something like sockets or workers that remain on RAM when you stop using any PaperJS tool? That's the issue, when you stop using the tools, even when I go back to the main page, the memory usage should decrease.

[arnaudcareo]

Same issue here. Regarding the amount of memory it takes and the fact that it's never freed, it really looks like memory leak. At each stroke, more memory is eaten. Saving and reloading the page (with the same shapes drawn) fixes the issue until all the memory is eaten again.

[jsbroks]

Shapes are united using PaperJS

    unite(compound, simplify = true, undoable = true) {
      if (this.compoundPath == null) this.createCompoundPath();
      if (undoable) this.createUndoAction("Unite");
      
      let newCompound = this.compoundPath.unite(compound);
      newCompound.onDoubleClick = this.compoundPath.onDoubleClick;
      newCompound.onClick = this.compoundPath.onClick;

      this.compoundPath.remove();
      this.compoundPath = newCompound;
      //this.keypoints.bringToFront();

      if (simplify) this.simplifyPath();
    }

Found in the Annotation.vue file.
Not sure how the memory leak is happening.

[jsbroks]

I think I figured it out... the paperjs shape is getting united because it happens so fast vue is trying to observe the changes which is causing memory leaks.

[jsbroks]

https://stackoverflow.com/questions/45814507/how-to-set-a-component-non-reactive-data-in-vue-2

[koenvandesande]

You seem to know how those internals work. My suggestion would be that it could also be the undo system; if you set undoable to false does the memory leaking stop?

[jsbroks]

Not as much of a concern for the undo systems because no changes are made to the internals of the paperjs object when they are added to the arrays of undos.

Thus what happens it that Vue ends up binding ALL the internal properties of Paper.js Circle object!

This means that every time any internal property of our Circle object changes, Vue’s reactivity listener gets called. That might not sound that terrible, but consider this; our Circle represents a particular graphical object on the screen which is (by default) updated 60 times a second via browser’s own animation loop.

If the circle is constantly being animated (which it probably is… we are after all building a game), we end up calling Vue’s reactivity listener 60 times per second.

And that is for one object, and for its one property that is mutated somewhere deep down in the heart of PaperJS code.

Now imagine we have 100 Monster objects. That would cause 6000 totally unnecessary calls per second per property.

This article explains the issue very well.

[AdanMora]

You seem to know how those internals work. My suggestion would be that it could also be the undo system; if you set undoable to false does the memory leaking stop?

Actually, that system is trivial, it's just a flag on MongoDB, it has no repercussion on the memory. The problem has to be related to PaperJS and Vue, as @jsbroks said.

[jsbroks]

There are actually 2 undo systems. One is a simple flag in the database but theres annother one within the image annotator. The second system is alittle more complex, not persistant, and handled by vue and paperjs. It appends paperjs shapes so you can do quick undos of polygons. All polygons and orders are stored in the vuex store

[AdanMora]

Ohhh yes, I never use it so I didn't remember it.

Interesting, that takes memory of course, but I don't know if it affects the memory leak.