[Bug]: Submit to Frigate+ shows black frame

[tve]

Checklist

• I have updated to the latest available Frigate version.
• I have cleared the cache of my browser.
• I have tried a different browser to see if it is related to my browser.
• I have tried reproducing the issue in incognito mode to rule out problems with any third party extensions or plugins I have installed.

Describe the problem you are having

When clicking on "submit to frigate+" I sometimes get a black frame:

Hitting space two times very rapidly to advance to the next frame generally fixes it.

Steps to reproduce

1. View a detection clip
2. Stop at some frame
3. Click the frigate+ button
4. observe black frame (this is intermittent!)

Version

0.15.0 beta 4 - 0.15.0-87e7b62

In which browser(s) are you experiencing the issue with?

Floorp (based on firefox)

Frigate config file

mqtt:
  host: core.voneicken.com
  user: frigate
  password: 

timestamp_style:
  format: '%Y-%m-%d %H:%M:%S'

record:
  enabled: true
  retain:
    days: 1
    mode: motion
  detections:
    retain:
      days: 30
      mode: active_objects

snapshots:
  enabled: true
  timestamp: false
  retain:
    default: 200

birdseye:
  enabled: true
  mode: motion

environment_vars:
  LIBVA_DRIVER_NAME: i965 # required for vaapi to find the correct driver

ffmpeg:
  global_args: -hide_banner # -loglevel debug
  hwaccel_args: preset-vaapi
  # hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format
  #   yuv420p
  #output_args:
  #  record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an

detectors:
  coral1:
    type: edgetpu
    device: usb:0
  #cpu1:
    #type: cpu

go2rtc:
  streams:
    studio:
      - ffmpeg:http://esp32-cam2.voneicken.com:81#video=h264#hardware
    gate:
      - rtsp://frigate:@192.168.11.88:554/cam/realmonitor?channel=1&subtype=0
    patio:
      - rtsp://frigate:@192.168.11.91:554/cam/realmonitor?channel=1&subtype=0
    overview:
      - rtsp://frigate:@192.168.11.93:554/cam/realmonitor?channel=1&subtype=0

model:
  path: plus://db5e39d2f6b05cdfb7fefee0a328eaa2
objects:
  filters:
    dog:
      min_score: .7
      threshold: .9
    cat:
      min_score: .65
      threshold: .8
    cow:
      min_score: .65
      threshold: .85
    face:
      min_score: .7
    package:
      min_score: .65
      threshold: .9
    license_plate:
      min_score: .6
    amazon:
      min_score: .75
    ups:
      min_score: .75
    fedex:
      min_score: .75
    person:
      min_score: .65
      threshold: .85
    car:
      min_score: .65
      threshold: .85

telemetry:
  stats:
    network_bandwidth: true

cameras:

  # Dahua IPC-HFW5442T-ASE-NI
  gate:
    ui:
      order: 0
    birdseye:
      mode: continuous
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/gate
          roles: [record]
        - path: 
            rtsp://frigate:@192.168.11.88:554/cam/realmonitor?channel=1&subtype=1
          roles: [detect]
    zones:
      inside:
        coordinates: 0.654,1,0.668,0.673,0.373,0.683,0,0.481,0,1
        inertia: 2
        loitering_time: 0
      outside:
        coordinates: 0.374,0.184,0.713,0.2,0.64,0.559,0.346,0.587
        inertia: 2
        loitering_time: 0
    record:
      retain:
        days: 5
        mode: motion
    objects:
      track: [person, car, dog, cat, sheep, bicycle, motorcycle, truck, bus, horse,
        sheep, bear, face, license_plate, amazon, usps, fedex, ups, dhl, deer, raccoon,
        fox, squirrel, cow, goat, rabbit, waste_bin, package]
    review:
      alerts:
        required_zones: inside
    motion:
      mask:
        - 0.025,0.011,0.427,0.011,0.427,0.065,0.026,0.065
        - 0.687,0.53,0.66,0.997,0.737,0.992,0.751,0.524
    detect:
      annotation_offset: -1400
  overview:
    webui_url: http://frigate:@192.168.11.93/
    ui:
      order: 0
    birdseye:
      mode: continuous
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/overview
          roles: [record]
        - path: 
            rtsp://frigate:view2frigate!@192.168.11.93:554/cam/realmonitor?channel=1&subtype=1
          roles: [detect]
    record:
      retain:
        days: 5
        mode: motion
    objects:
      track: [person, car, dog, cat, sheep, bicycle, motorcycle, truck, bus, horse,
        sheep, bear, amazon, usps, fedex, ups, dhl, deer, raccoon, fox, squirrel,
        cow, goat, rabbit]
    detect:
      annotation_offset: -1400
    motion:
      mask: 0.01,0.005,0.43,0.006,0.427,0.059,0.012,0.059
    onvif:
      host: 192.168.11.93
      port: 80
      user: frigate
      password: 
      autotracking:
        enabled: true
        calibrate_on_startup: false
        required_zones:
          - full
        zooming: relative
        track: [person, car, dog, cat, sheep, bicycle, motorcycle, truck, bus, horse,
          sheep, bear, amazon, usps, fedex, ups, dhl, deer, raccoon, fox, cow, goat]
        return_preset: Overview
        movement_weights: 0.04, 1.0, 1.0367317199707031, 1.3062768443938224, 0.3844547048691781
    zones:
      full:
        coordinates: 0,0.062,1,0.065,1,1,0,1
        loitering_time: 0
  patio:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/patio
          roles: [record]
        - path: 
            rtsp://frigate:@192.168.11.91:554/cam/realmonitor?channel=1&subtype=1
          roles: [detect]
    record:
      retain:
        days: 2
    objects:
      track: [person, car, dog, cat, sheep, bicycle, motorcycle, truck, bus, horse,
        sheep, bear, face, license_plate, amazon, usps, fedex, ups, dhl, deer, raccoon,
        fox, squirrel, cow, goat, rabbit, waste_bin, package]
    motion:
      mask:
        - 0.02,0.054,0.399,0.054,0.399,0,0.021,0
        - 0.028,0.917,0.196,0.917,0.195,0.967,0.027,0.967
    zones:
      tracking_start:
        coordinates: 0.344,0.121,0.224,0.493,0.562,0.863,0.871,0.903,0.892,0.565,0.77,0.4,0.757,0.163
        loitering_time: 1
        inertia: 3
    onvif:
      host: 192.168.11.91
      port: 80
      user: frigate
      password: 
      autotracking:
        enabled: true
        calibrate_on_startup: false
        zooming: relative
        required_zones:
          - tracking_start
        track:
          - person
          - dog
          - cat
          - sheep
        return_preset: Driveway
        movement_weights: 0.33, 1.0, 1.0782661437988281, 1.2704954603666898, 0.3403729777182304
    review:
      alerts:
        required_zones: tracking_start
    detect:
      annotation_offset: -1800
  old-gate:
    ui:
      order: 1
    birdseye:
      mode: continuous
    ffmpeg:
      inputs:
        - path: rtsp://phycam.voneicken.com/live/1/h264.sdp
          roles: [detect, record]
      #input_args: [ -flags, low_delay, -strict, experimental ]
      input_args:
        - -avoid_negative_ts
        - make_zero
        - -use_wallclock_as_timestamps
        - '1'
        - -timeout
        - '5000000'
        - -fflags
        - nobuffer
        - -flags
        - low_delay
        - -strict
        - experimental
        - -fflags
        - +genpts+discardcorrupt
    detect:
      enabled: false
      width: 768
      height: 432
      fps: 5
    timestamp_style:
      position: bl
    motion:
      mask:
        - 768,0,712,0,620,205,605,432,768,432
    record:
      retain:
        days: 2
    zones:
      inside:
        coordinates: 570,432,615,292,372,247,0,292,0,432
      outside:
        coordinates: 388,44,663,93,636,266,333,221
    objects:
      track: [person, car, dog, cat, sheep, bicycle, motorcycle, truck, bus]

  # rpi-cam2 -- rPi3a+imx462-based camera 1920x1080
  birdbath:
    ui:
      order: 2
    ffmpeg:
      inputs:
        - path: rtsp://rpi-cam2.voneicken.com:8554/cam5fps
          roles: [detect, record]
    detect:
      width: 1920
      height: 1080
      fps: 5
    #mqtt:
    #  event_history: True
    timestamp_style:
      position: bl
    motion:
      #threshold: 20  # default is 25, pixel value delta to detect motion
      #delta_alpha: 0.15  # default is 0.2
      #frame_alpha: 0.15  # default is 0.2
      contour_area: 50  # default is 100, misses hummingbirds
      mask:
        - 1468,0,1920,0,1920,198,1468,172
    record:
      retain:
        days: 3
    objects:
      track: [bird, cat, dog, bear, sheep, deer, raccoon, fox, squirrel, goat, rabbit]

  studio:
    ui:
      order: 3
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/studio
          roles: [detect, record]
    objects:
      track: [person, bird, cat, dog, bear, sheep, deer, raccoon, fox, squirrel, goat,
        rabbit, horse]
    motion:
      threshold: 30
      contour_area: 10
      improve_contrast: false
version: 0.15-0

docker-compose file or Docker CLI command

ExecStart=/usr/bin/docker run \
  --name %n \
  --restart=unless-stopped \
  -e PLUS_API_KEY= \
  --device /dev/bus/usb:/dev/bus/usb \
  --device /dev/dri:/dev/dri \
  -v /media:/media/frigate \
  -v /home/frigate/config:/config \
  -v /etc/localtime:/etc/localtime:ro \
  --mount type=tmpfs,target=/tmp/cache,tmpfs-size=1000000000 \
  --shm-size=256m \
  -p 5000:5000 \
  -p 1935:1935 \
  -p 1984:1984 \
  -p 8971:8971 \
  -p 8554:8554 \
  -p 8555:8555/tcp \
  -p 8555:8555/udp \
  --privileged \
  ghcr.io/blakeblackshear/frigate:0.15.0-beta4

Relevant Frigate log output

There is no log message of any sort when this happens

Relevant go2rtc log output

There is no log message of any sort when this happens.

Operating system

Debian

Install method

Docker CLI

Network connection

Wired

Camera make and model

Dahua IPC-HFW5442T-ASE-NI

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

I'm getting this black frame maybe 10%-20% of the time, not only for this camera (the others are Dahua too). It's annoying when I managed to stop on exactly the right frame that I loose it...

[hawkeye217]

I can't reproduce this at all. Can you try another browser like Chrome or Edge and see if have the same behavior there?

[hawkeye217]

Could it be that this offset is applied incorrectly in this use-case?

No. There is nothing involved with the backend here. All Frigate is doing is using a canvas HTML element to grab the pixels being rendered by the HTML video player.

The only thing I can think of is that the video decoder in your browser is processing frames ahead of what's being displayed. If it works fine with a Chromium-based browser, that would certainly point to the Firefox-based browser's renderer.

Can you share your browser name and version, screen resolution, and the video codec your camera is using?

[tve]

The black frame only happens with floorp, the time offset between what I see in the video and what shows up submitted in frigate+ is the same in floorp and chromium.

• Floorp 11.28.0 (64-bit) for Arch Linux
• Chromium Version 138.0.7204.92 (Official Build) Arch Linux (64-bit)
• Both on Wayland with Hyprland compositor
• Since you mention screen resolutions.... I tried chromium on my 2nd display, which is 2560x1440 rotated 90 degrees and 0.9x DPI scaling as well as on my primary display, which is 3840x2160 scaled 1.2x; in both cases the time offset is the same
• Camera is a Dahua IPC-HFW5442T with the following streams:

• I'm using go2rtc: rtsp://frigate:gate2frigate!@192.168.0.0:554/cam/realmonitor?channel=1&subtype=0 and the ffmpeg config for this camera is:

    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/gate
          roles: [record]
        - path: 
            rtsp://frigate:gate2frigate!@192.168.0.0:554/cam/realmonitor?channel=1&subtype=1
          roles: [detect]

[hawkeye217]

We've had no other reports of this, so it's difficult to pinpoint what might be the cause. Can you reproduce this on a Mac or Windows browser?

[tve]

I pulled out my old Windows 10 laptop and Google Chrome 136.0.7103.48 and I get exactly the same result. When you try to repro this are you using footage from a fast moving car or similar so you can tell if there is a 0.5 second - 1 second offset?
Edit: maybe i need to be clear about what I'm doing. I'm looking at "detections" and I click on a video. The video plays and I hit the spacebar right when the headlights of the car disappear off-frame causing the video to show the roof of the car as opposed to the blinding headlights. I then click on the frigate+ button in the stopped video frame and switch over to the frigate+ images review page and see that what appears there is a frame taken about one second prior, which is obvious 'cause of where the car is and the blinding headlights.

[hawkeye217]

Thanks for clarifying! There's a few things going on here:

1. Your original bug report about the black frame seems to be an issue specific to the Floorp browser. We've had no other reports of that, and the fact that it works fine on other browsers seems to support that conclusion.

2. Your offset frame is related to your camera settings and the way that ffmpeg grabs frames from your recording stream for upload to Frigate+. Frigate does not decode the stream with the record role and merely copies it directly to your storage. When you request a frame from it, Frigate passes your browser player's frame time to ffmpeg, which will seek to the frame time and try to grab a frame. Here's the relevant code:

   frigate/frigate/util/image.py

   Line 941 in ee1db24

   def get_image_from_recording(

   Videos are made up of keyframes (I-frames) and dependent frames (P-frames, B-frames). Keyframes are the "anchor points" in a video stream that allow for efficient seeking. The more frequent the keyframes, the more precise ffmpeg can be in its seeking without having to decode a large number of frames leading up to the desired time.

   While ffmpeg will try to get as close as possible to your specified time, the actual frame extracted will depend on the proximity of keyframes. If there's a long gap between keyframes, the frame returned might be further away from the exact relative_frame_time.

   With that said, your stream is only recording at 10fps. If you're trying to capture a precise moment of a fast-moving car from your 10fps stream with a 10-frame I-frame interval, you'll likely experience inaccuracy. While ffmpeg will try to get close to your requested time by seeking to the nearest keyframe and then decoding, the inherent limitations of the video's temporal resolution (10 frames per second) and the car's speed mean the frame you get might not perfectly represent that fleeting instant you're aiming for.

   You can improve this by increasing your frame rate and keeping the I-frame interval matched to the frame rate.