Cannot connect to RT1062 with openMV IDE when script is running (Ubuntu)

[kdarras]

Describe the bug
Cannot connect to RT1062 with openMV IDE when script is running. Instead, "Busy... please wait... " is displayed indefinitely (I waited >5 min)

To Reproduce

• insert an SD card into the RT1062
• connect the RT1062 to the openMV IDE, open the example script for advanced FD on memory
• add this to the script for diagnostics

import os
from machine import LED
if not "triggered" in os.listdir():
    os.mkdir("triggered")  # Make a directory for triggered images

and at the end:

    if triggered:
        LED("LED_GREEN").on()
        img.save("triggered/img.bmp")
        LED("LED_GREEN").off()

• save script to openMV and disconnect the RT1062 from IDE, unplug
• plug the RT1062 back into the computer to power the board, wait for the script to run as indicated by green LED
• try to connect through the IDE → “Busy... please wait...”

Workarounds
-unplug and replug the board, and click "connect" in the IDE right after the connection button becomes available the second time (within a second or so on Ubuntu, right after the SD card is detected by the system)
-introduce a delay (e.g. time.sleep_ms(5000)) for the user to have time to connect the IDE to the board before the script proceeds

Expected behavior
The IDE interrupts the running script and connects to the board

Screenshots

Please complete the following information

• IDE version: 4.7.0
• Firmware version: 4.7.0
• Board: RT1062
• OS: Ubuntu

[kwagyeman]

Yeah, I followed your steps and the IDE can stop the script. After it's running. This works. However, there are some issues with saving the script that cause problems. On windows atleast, after saving the script to disk and resetting the camera I had to close the IDE as it locked up, then unplug and replug the camera. After doing that the camera then started up and ran the script and I was able to connect from the IDE and stop the script as normal. I could see the green LED blinking when I connected via the IDE.

The IDE already has code to forcibly call windows API to unmount the flash disk on script reset of the camera, however, these don't appear to work very well. I'll add an action item to improve this. #307.

Anyway, as for this issue, if you have the script as you described on the camera, when it's running, you can indeed connect to the camera from the IDE. Note that I used the latest dev release firmware under Tools->Install Latest Dev Release.

[kdarras]

Thanks for testing. I will try to reproduce this later today with the latest dev firmware.
Note I am using Linux.

[kdarras]

With the latest dev release installed today, the problem as reproduced above persists.

[kwagyeman]

Can you give me your exact script? I was not able to reproduce with the latest dev release.

[kdarras]

# This work is licensed under the MIT license.
# Copyright (c) 2013-2023 OpenMV LLC. All rights reserved.
# https://github.com/openmv/openmv/blob/master/LICENSE
#
# Advanced Frame Differencing Example
#
# This example demonstrates using frame differencing with your OpenMV Cam. This
# example is advanced because it performs a background update to deal with the
# background image changing overtime.

import sensor
import time
import os
from machine import LED
if not "triggered" in os.listdir():
    os.mkdir("triggered")  # Make a directory for triggered images

TRIGGER_THRESHOLD = 5

BG_UPDATE_FRAMES = 50  # How many frames before blending.
BG_UPDATE_BLEND = 128  # How much to blend by... ([0-255]==[0.0-1.0]).

sensor.reset()  # Initialize the camera sensor.
sensor.set_pixformat(sensor.RGB565)  # or sensor.RGB565
sensor.set_framesize(sensor.QVGA)  # or sensor.QQVGA (or others)
sensor.skip_frames(time=2000)  # Let new settings take affect.
sensor.set_auto_whitebal(False)  # Turn off white balance.
clock = time.clock()  # Tracks FPS.

# Take from the main frame buffer's RAM to allocate a second frame buffer.
# There's a lot more RAM in the frame buffer than in the MicroPython heap.
# However, after doing this you have a lot less RAM for some algorithms...
# So, be aware that it's a lot easier to get out of RAM issues now. However,
# frame differencing doesn't use a lot of the extra space in the frame buffer.
# But, things like AprilTags do and won't work if you do this...
extra_fb = sensor.alloc_extra_fb(sensor.width(), sensor.height(), sensor.RGB565)

print("About to save background image...")
sensor.skip_frames(time=2000)  # Give the user time to get ready.
extra_fb.replace(sensor.snapshot())
print("Saved background image - Now frame differencing!")

triggered = False

frame_count = 0
while True:
    clock.tick()  # Track elapsed milliseconds between snapshots().
    img = sensor.snapshot()  # Take a picture and return the image.

    frame_count += 1
    if frame_count > BG_UPDATE_FRAMES:
        frame_count = 0
        # Blend in new frame. We're doing 255-alpha here because we want to
        # blend the new frame into the background. Not the background into the
        # new frame which would be just alpha. Blend replaces each pixel by
        # ((NEW*(alpha))+(OLD*(255-alpha)))/255. So, a low alpha results in
        # low blending of the new image while a high alpha results in high
        # blending of the new image. We need to reverse that for this update.
        img.blend(extra_fb, alpha=(255 - BG_UPDATE_BLEND))
        extra_fb.replace(img)

    # Replace the image with the "abs(NEW-OLD)" frame difference.
    img.difference(extra_fb)

    hist = img.get_histogram()
    # This code below works by comparing the 99th percentile value (e.g. the
    # non-outlier max value against the 90th percentile value (e.g. a non-max
    # value. The difference between the two values will grow as the difference
    # image seems more pixels change.
    diff = hist.get_percentile(0.99).l_value() - hist.get_percentile(0.90).l_value()
    triggered = diff > TRIGGER_THRESHOLD
    if triggered:
        LED("LED_GREEN").on()
        img.save("triggered/img.bmp")
        LED("LED_GREEN").off()

    print(clock.fps(), triggered)  # Note: Your OpenMV Cam runs about half as fast while
    # connected to your computer. The FPS should increase once disconnected.

[kwagyeman]

@kdarras - Okay, I was able to reproduce. It appears to happen on linux but not windows. I can connect normally on windows.

Since it's the same firmware in either case, but, linux fails, then this is an IDE issue. I will transfer the bug there.

[kwagyeman]

The IDE doesn't have anything special with regards to connecting to the camera via linux/windows... It must be some unrelated API that's called during connect.

I'm not in an IDE bug fixing window right now, are you able to wait a few months on me fixing this? I plan to tackle all the other accumulated bugs with the IDE at the same time.

[kdarras]

Sure - I have a workaround. Just wanted to make sure it's logged. All the best