[HW Accel Support]: High CPU usage

[toperichvania]

Describe the problem you are having

I am struggling to understand how the streams hardware acceleration works and why I still have high CPU usage. My current setup is that I use Intel Quick sync for hardware acceleration and I use nvidia rtx4070 with 12G of RAM for inference. When I use Intel for hardware acceleration, I still have high CPU usage by ffmpeg process. I suspect that this is due to detections. But when I use NVIDIA I don't seem to see that same high CPU usage.
Further more, I have disabled the detection for all cameras and CPU is still very high as you can see in the image below

Configuration attached is a sample of config, there are more cameras on the system (60)

Some more questions:

• if I would use multiple NVIDIA cards for hardware acceleration as I have 60 cameras so that would not be enough RAM for single card, how could I specify this in config?
• using Intel iGPU seems to not have same effect on RAM like using NVIDIA card. It seems that ffmpeg uses less of RAM in that case? I wonder why? if I would use intel Arc card, would it behave same as NVIDIA or iGPU in terms of used RAM?

Version

0.15

Frigate config file

mqtt:
  enabled: true
  host: 192.168.1.71
  user: '{FRIGATE_MQTT_USER}'
  password: '{FRIGATE_MQTT_PASSWORD}'

ffmpeg:
  hwaccel_args: preset-intel-qsv-h264
  # hwaccel_args: preset-nvidia

go2rtc:
  rtsp:
    username: "admin"
    password: "linc2025"
  webrtc:
    ice_servers:
    - urls: [stun:192.168.1.71:3478]
    - urls: [stun:stun1.l.google.com:19302]
    candidates:
      - 192.168.0.252:8555
      - stun:8555
      - 10.120.0.7:8555
  streams:
    ptz_main:
      - ffmpeg:rtsp://@192.168.1.88:554/Streaming/Channels/101?transportmode=unicast&profile=Profile_1
    ptz_sub:
      - ffmpeg:rtsp://@192.168.1.88:554/Streaming/Channels/102?transportmode=unicast&profile=Profile_2
    i_pro:
      - ffmpeg:rtsp://admin:Linc2024!@192.168.1.159/ONVIF/MediaInput?profile=def_profile1

cameras:
  PTZ:
    enabled: true
    objects:
      track:
        - license_plate
        - face
        - person
        - car

    mqtt:
      enabled: true
      height: 720
      quality: 100
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/ptz_main
          input_args: preset-rtsp-restream
          roles:
            - record
            - detect
        # - path: rtsp://127.0.0.1:8554/ptz_sub
        #   input_args: preset-rtsp-restream
        #   roles:
        #     - detect
    onvif:
      host: 192.168.1.88
      port: 80
      user: admin
      password: Ftek3835!
    detect:
      enabled: true
      # width: 1280
      # height: 720
    live:
      stream_name: ptz_main

  iPro:
    enabled: true
    objects:
      track:
        - face
        - license_plate
        - person
        - car
    mqtt:
      enabled: true
      height: 720
      quality: 100
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/i_pro
          input_args: preset-rtsp-restream
          roles:
            - record
            - detect
    onvif:
      host: 192.168.1.159
      port: 80
      user: admin
      password: Linc2024!
    detect:
      enabled: true
    live:
      stream_name: i_pro

record:
  enabled: true
  retain:
    days: 1
    mode: all
  alerts:
    retain:
      days: 1
      mode: motion
  detections:
    retain:
      days: 1
      mode: motion

snapshots:
  enabled: true
  clean_copy: true
  timestamp: true
  bounding_box: true
  crop: true
  height: 720
  quality: 100
  retain:
    default: 1

detect:
  fps: 5
  enabled: true

detectors:
  tensorrt:
    type: tensorrt
      #    device: 0

model:
  path: /config/model_cache/tensorrt/yolov7-320.trt
  input_tensor: nchw
  input_pixel_format: rgb
  width: 320
  height: 320

      #model:
      #  path: plus://8cb3aa750bc072279dfb22d0a537434c

semantic_search:
  enabled: true
  reindex: false
  model_size: small

objects:
  filters:
    face:
      min_score: .7
    license_plate:
      min_score: .6
    person:
      min_score: .65
      threshold: .85

docker-compose file or Docker CLI command

services:
  frigate:
    privileged: true
    container_name: aware-vms
    restart: unless-stopped
      #image: ghcr.io/blakeblackshear/frigate:0.15.0-tensorrt
      #image: frigate:latest-tensorrt
    image: europe-west1-docker.pkg.dev/aware-vms/aware-vms/linc-security-systems/aware-vms:latest-tensorrt 
    shm_size: "8gb"
    devices:
      - /dev/dri/renderD128
    runtime: nvidia
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./config:/config
      - /shared/recordings:/media/frigate
      - type: tmpfs 
        target: /tmp/cache
        tmpfs:
          size: 2000000000
    ports:
      - "1984:1984"
      - "8971:8971"
      - "5000:5000" # Internal unauthenticated access. Expose carefully.
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_RTSP_PASSWORD: "password"
      FRIGATE_MQTT_USER: "dev"
      FRIGATE_MQTT_PASSWORD: "password"
        #      YOLO_MODELS: "yolov7-320"
      PLUS_API_KEY: "8da6d579-21c3-455a-ba9b-a3e5915232fa:cb72ac743fea1d2b19bbd0d10cdbee64121ba849"

Relevant Frigate log output

-

Relevant go2rtc log output

-

FFprobe output from your camera

Input #0, rtsp, from 'rtsp://admin:Ftek3835!@192.168.1.88:554/Streaming/Channels/101?transportmode=unicast&profile=Profile_1':
  Metadata:
    title           : Media Presentation
  Duration: N/A, start: 0.040000, bitrate: N/A
  Stream #0:0: Video: h264 (High), yuvj420p(pc, bt709, progressive), 2560x1440, 25 fps, 25 tbr, 90k tbn

Install method

Proxmox via Docker

Object Detector

TensorRT

Network connection

Wired

Camera make and model

iPro

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

No response

[hawkeye217]

Hardware acceleration improves ffmpeg decoding CPU usage, which doesn't look overly excessive to me. This could potentially be lowered even more depending on your detect resolution settings.

The frigate.output process seems reasonable to be using ~30% especially if you're running the system with 60 cameras.

if I would use multiple NVIDIA cards for hardware acceleration as I have 60 cameras so that would not be enough RAM for single card, how could I specify this in config?

You'd need to use some manual ffmpeg arguments for each camera to change the hwaccel_device. For example:

  global_args: -hide_banner -loglevel warning -threads 2 -hwaccel_device 1

using Intel iGPU seems to not have same effect on RAM like using NVIDIA card. It seems that ffmpeg uses less of RAM in that case? I wonder why? if I would use intel Arc card, would it behave same as NVIDIA or iGPU in terms of used RAM?

ffmpeg's difference in RAM usage is likely just due to the way it utilizes the different GPUs. I don't recall users mentioning RAM usage differences in the past but you could try searching the past discussions.

[toperichvania]

Thanks! So is it possible to turn off decoding for some cameras where I don't need detection on? If I remove -detect state from the stream does it help. I have set the detect to false on those but I don't think that helped

…

On 17 Jul 2025, at 13:13, Josh Hawkins ***@***.***> wrote: Hardware acceleration improves ffmpeg decoding CPU usage, which doesn't look overly excessive to me. This could potentially be lowered even more depending on your detect resolution settings <https://docs.frigate.video/frigate/camera_setup/#choosing-a-detect-resolution>. The frigate.output process seems reasonable to be using ~30% especially if you're running the system with 60 cameras. if I would use multiple NVIDIA cards for hardware acceleration as I have 60 cameras so that would not be enough RAM for single card, how could I specify this in config? You'd need to use some manual ffmpeg arguments for each camera to change the hwaccel_device. For example: global_args: -hide_banner -loglevel warning -threads 2 -hwaccel_device 1 using Intel iGPU seems to not have same effect on RAM like using NVIDIA card. It seems that ffmpeg uses less of RAM in that case? I wonder why? if I would use intel Arc card, would it behave same as NVIDIA or iGPU in terms of used RAM? ffmpeg's difference in RAM usage is likely just due to the way it utilizes the different GPUs. I don't recall users mentioning RAM usage differences in the past but you could try searching the past discussions. — Reply to this email directly, view it on GitHub <#19183 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BIJSLASYAN7P4GSDVIW7BZ33I6AL3AVCNFSM6AAAAACBXPFKICVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTGNZZGAZTQMA>. You are receiving this because you authored the thread.

[NickM-27]

No, decoding is required for multiple parts of the UI along with features like birdseye

[toperichvania]

OK. Thanks. Problem I have is hardware related and I am trying to minimise requirement of GPU for streams. Intel stopped E24 series processors that had iGPU integrated and I am left with Nvidia expensive cards that are not as efficient as Intel.

…

On 17 Jul 2025, at 13:13, Josh Hawkins ***@***.***> wrote: Hardware acceleration improves ffmpeg decoding CPU usage, which doesn't look overly excessive to me. This could potentially be lowered even more depending on your detect resolution settings <https://docs.frigate.video/frigate/camera_setup/#choosing-a-detect-resolution>. The frigate.output process seems reasonable to be using ~30% especially if you're running the system with 60 cameras. if I would use multiple NVIDIA cards for hardware acceleration as I have 60 cameras so that would not be enough RAM for single card, how could I specify this in config? You'd need to use some manual ffmpeg arguments for each camera to change the hwaccel_device. For example: global_args: -hide_banner -loglevel warning -threads 2 -hwaccel_device 1 using Intel iGPU seems to not have same effect on RAM like using NVIDIA card. It seems that ffmpeg uses less of RAM in that case? I wonder why? if I would use intel Arc card, would it behave same as NVIDIA or iGPU in terms of used RAM? ffmpeg's difference in RAM usage is likely just due to the way it utilizes the different GPUs. I don't recall users mentioning RAM usage differences in the past but you could try searching the past discussions. — Reply to this email directly, view it on GitHub <#19183 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BIJSLASYAN7P4GSDVIW7BZ33I6AL3AVCNFSM6AAAAACBXPFKICVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTGNZZGAZTQMA>. You are receiving this because you authored the thread.

[hawkeye217]

I'd reduce your detect resolution as I originally suggested and see if that improves things first.

[toperichvania]

Yes, I will test that this week and let you know. Thanks!

…

On 17 Jul 2025, at 21:07, Josh Hawkins ***@***.***> wrote: I'd reduce your detect resolution as I originally suggested and see if that improves things first. — Reply to this email directly, view it on GitHub <#19183 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BIJSLATSP4LFN23QDTU7NTL3I7YAFAVCNFSM6AAAAACBXPFKICVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTGNZZGYYTKNI>. You are receiving this because you authored the thread.

[Cold-Lemonade]

@toperichvania I think you inadvertently exposed your Frigate-Plus API key. You might need to request a new one from @blakeblackshear

[blakeblackshear]

You can just regenerate it in the Frigate+ UI. That will invalidate the old one.

[toperichvania]

Ah, yes, thanks. done..

…

On 17 Jul 2025, at 22:31, Blake Blackshear ***@***.***> wrote: You can just regenerate it in the Frigate+ UI. That will invalidate the old one. — Reply to this email directly, view it on GitHub <#19183 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BIJSLASJJAJOTTG22E4CRBT3JABYNAVCNFSM6AAAAACBXPFKICVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTGNZZG42DKOA>. You are receiving this because you were mentioned.

[toperichvania]

Out of curiosity, what other UI parts are needed decoding?

…

On 17 Jul 2025, at 20:55, Nicolas Mowen ***@***.***> wrote: No, decoding is required for multiple parts of the UI along with features like birdseye — Reply to this email directly, view it on GitHub <#19183 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BIJSLARDZK3HK5IVRHCT6XT3I7WTRAVCNFSM6AAAAACBXPFKICVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTGNZZGU4TMMQ>. You are receiving this because you authored the thread.

[hawkeye217]

The decoded stream is used any time a camera is displayed in the UI except History view (which plays the record stream) and the Object Lifecycle pane (which uses frame grabs from the record stream).