Loading kmodel on MaixPy crashing device

[THuffam]

Hi
When loading a kmodel file - either from SD or flash the device (MaixPy Bit - new version with mic) crashes (
disconnects from IDE and LCD goes blank). This is at line:
task = kpu.load(0x300000)

The kmodel was generated using axelerate running on Google Colab.
This was using firmware: maixpy_v0.6.2_78_g469c985e6_openmv_kmodel_v4_with_ide_support.bin

When I try without the IDE (firmware: maixpy_v0.6.2_78_g469c985e6_minimum_with_kmodel_v4_support.bin) , using the VS Code console and run the code line by line it shows the error (on kpu.load):
E (77045384089) SYSCALL: Out of memory

I am using the code from raccoon_detector.py and using my kmodel file.

My kmodel file is 1871kb in size. How to work around this?

[THuffam]

I noticed in the raccoon_detector.py that it says:
# needs firmware from my fork with yolov3 support, see
# https://github.com/sipeed/MaixPy/pull/451

How do I use that firmware? and how do I know if I'm using Yolo2 or Yolo3?
Thanks

[AIWintermuteAI]

1871kb should be fine. But yes, the model you trained is YOLOv3 - if you used master branch of axelerate, which you probably did. To train a model that can run on current, unmodified MaixPy you need to use legacy branch.
Alternatively, try using binary and example script from here:
https://drive.google.com/file/d/1q1BcWA8GiTQ_3Q9vYkSysRvGD62K2zh4/view?usp=sharing
that is for YOLOv3.

[THuffam]

Thank you very much for your response Dmitry.

I tried the binary you supplied but also got the out of memory error (E (68325830111) SYSCALL: Out of memory) when loading my model (but it did load your model ok - which is 300k smaller than mine).

Where can I get a legacy branch from - and would I need to build it myself?
or is there a way to convert the model to v3 so I can use the mimimum (latest) version (which throws the following error if I try to load my model: ValueError: [MAIXPY]kpu: load error:2002, ERR_KMODEL_VERSION: only support kmodel V3).

I am looking at resizing the GC as per this article - will this affect it at run time?

Also, in the example you sent (person_detector_yolo3.py) there are the following 2 lines:

a = kpu.set_outputs(task, 0, 10, 8, 21) #the actual shape needs to match the last layer shape of your model(before Reshape)
a = kpu.set_outputs(task, 1, 20, 16, 21) #the actual shape needs to match the last layer shape of your model(before Reshape)

Where do I get these values? Is this something I can see when running axelerate in colab?

Many thanks again.

[AIWintermuteAI]

Hi!
So I added an FAQ section to main readme
https://github.com/AIWintermuteAI/aXeleRate#question-faq
That should answer your question about legacy branch.

or is there a way to convert the model to v3 so I can use the mimimum (latest) version (which throws the following error if I try to load my model: ValueError: [MAIXPY]kpu: load error:2002, ERR_KMODEL_VERSION: only support kmodel V3).

You can use earlier version of nn-case and manually convert trained .tflite model from the project folder. Not sure what pitfalls there will be as I didn't use earlier versions of nncase for a while.

I am looking at resizing the GC as per this article - will this affect it at run time?

That can help you to get slightly more available RAM.

Where do I get these values? Is this something I can see when running axelerate in colab?

Yes, when you start training you can see the model summary. Check closed issues, there was a question about that.

Hope that was helpful. If everything works out for you, consider buying a beer/coffee for the project :)
https://www.buymeacoffee.com/hardwareai

[THuffam]

Thanks for this. Still struggling but I have been watching memory usage and noticed your kmodel file is only 874k whereas mine is 1871k... how did you get yours so small? Is there something I can do to make it smaller (I assume while training the model)?

I had to reduce the GC to 256k to be able to load the kmodel file.. but then I get out of heap memory trying to load an image (from SD card). It also runs out of memory when trying to use the camera.

Thanks again for your assistance.

[AIWintermuteAI]

I think I'm just using smaller feature extractor? MobileNet5_0 (this corresponds to mobilenet with alpha = 0.5).

[THuffam]

Have used MobileNet5_0 which generates a much smaller kmodel (870k) - thanks for that tip.
But testing it is proving difficult...
When I use the binary you provide (in your mask detector example - maixpy.bin), when I plug in the maixpy device the screen shows "SDcard not mount, use flash". So I cannot try the static image example (person_detector_yolov3.py).
When I try the camera example (person_detector_yolov3_cam.py), I get the following error:

code = kpu.run_yolo3(task, img)
[MAIXPY]kpu: img w=320,h=240, but model w=320,h=224
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: [MAIXPY]kpu: check img format err!

Any suggestions?

[AIWintermuteAI]

Yes. Exactly as it says in the error message, your model has input size of 224,320, but you want to feed it with the image from camera, that has size of 240,320.
You need to either resize the camera image before feeding it to NN or train the model to take 240,320 image as input.

[THuffam]

Oh, sorry I must have missed that. Have retrained and it now runs without throwing an exception!!
Now trying to get it to recognise the target objects is an issue.
When run using your demo code (person_detector_yolov3_cam.py) updated for my solution, it throws a lot of false positives and does not seem to recognise my target (a shark). So as your code says on the following line:

a = kpu.init_yolo3(task, 0.3, 0.3, 3, 2, anchor) #tweak the second parameter if you're getting too many false positives

I updated it to:

a = kpu.init_yolo3(task, 0.92, 0.3, 3, 2, anchor) #tweak the second parameter if you're getting too many false positives

This yeilds the least number of false positives but when it does show a result it is off - drawing the rectangle either above or below the object. Is there a reason for this or is it more likely that these are just false positives and it hasn't recognised the shark?
Will attach images..

[AIWintermuteAI]

Okay, so 0.3 confidence threshold is way too low for YOLOv3 model, that is correct.
Now about bounding boxes in the wrong place - that could be a) wrong anchors b) upside down / mirrored image from camera or something along these lines.
I recommend you do inference on static image with person_detector_yolov3.py and see if the result is right.
As a side note, the inference from camera script was tested with Maix Go camera - and the parameters (rotation, vflip) in the script are for that board.

[THuffam]

Thanks again.
I found turning the camera around 90 degrees improved the issue but the boxes were closer to the object, not surrounding it.
Same with static images - see below.
What do you recommend I do for changing the anchor parameters - I have just used the ones in your detector colab example (which are the same in your maixpy mask example)?

[AIWintermuteAI]

Okay. If the problem persists with static images, there must be something wrong with how it decodes the boxes.
If you run inference on the computer on the same images, how do the boxes look?

[chaojunchi]

HI，
I'm training a bike recognition model right now. More than 5,000 pictures of the bike were collected and trained, and the trained model was downloaded to the K210 Flash. Running effect is still good, can be very accurate identification of all kinds of bikes, but there are some wrong identification. I have tried many times to adjust coord_scale, object_scale and no_object_scale parameters for training, but it is still easy to misidentify.
I don't quite understand how to adjust these parameters. I just make repeated adjustments and observe the operation effect. I would like to know how these parameters should be adjusted, which parameters should be adjusted under what circumstances and how much is appropriate. Please help me, thank you!

[AIWintermuteAI]

@chaojunchi please create a new issue. You question has no relation to this issue.

[THuffam]

The inference run on Colab (after training) yielded correct results :


Have not tried it on my PC (don't have a linux environment).