Skip to content

WebRTC send-receive audio creates High CPU Utilization #401

@BajaMexico

Description

@BajaMexico

If you run the code from https://fastrtc.org/userguide/gradio/ for the WebRTC Component it creates a 100% utilization of the CPU. This happens as soon as you press "Record" on the UI, there doesn't need to be any audio to actually call the ReplyOnPause method. The fans start to really go fast to keep the CPU cool. And while the example is fairly trivial if you do anything in your ReplyOnPause method (I am using Meta Seamless) the actual audio response starts to talk really slow. I assume this is due to the high CPU utilization-> high fans-> thermal throttling of CPU, Am I doing something wrong, is there a way to keep the CPU utilization to a lower value, is this just the nature of monitoring for audio input?

This is with:
fastrtc 0.0.33
fastrtc-moonshine-onnx 20241016

on a Windows 11 WSL2 stock linux system

Here is the code from the webpage example above with additions of the line
import numpy as np

and line change of
demo.launch(server_name="127.0.0.1", server_port = 49000)

import gradio as gr
import numpy as np
from fastrtc import WebRTC, ReplyOnPause

def response(audio: tuple[int, np.ndarray]):
"""This function must yield audio frames"""
...
yield audio

with gr.Blocks() as demo:
gr.HTML(
"""


Chat (Powered by WebRTC ⚡️)


"""
)
with gr.Column():
with gr.Group():
audio = WebRTC(
mode="send-receive",
modality="audio",
)
audio.stream(fn=ReplyOnPause(response),
inputs=[audio], outputs=[audio],
time_limit=60)
demo.launch(server_name="127.0.0.1", server_port = 49000)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions