Mute / ignore user inputs during yield

[AndreasKarasenko]

In my current setup I have a longer startup audio playing with can_interrupt=False.
However, while the audio can not be interrupted, users can still interact with the system.
That means it handles the audio input, calls the handler, produces audio output and queus the yield.

Is there a way to suppress this? E.g. by muting the user microphone during yields.

[freddyaboulton]

Yea the microphone should be muted if can_interrupt=False!

[AndreasKarasenko]

Currently I seem to run into a bug then, I can queue up multiple interactions even if can_interrupt=False while the output of either the startup or the handler is still playing.

[freddyaboulton]

I mean that we should modify the code so that the mic is muted if can_interrupt=False. Will hopefully fix this soon for you.

[AndreasKarasenko]

Hey, sorry to be pushy but is there an update on this?

[AndreasKarasenko]

@freddyaboulton I managed to create a workaround for my issue. Some background first though:
I noticed is that fastrtc (at least in my usage) generates the TTS audio chunks in one spike, which I see first in my GPU usage for the LLM interaction and then in my CPU usage for the TTS. E.g. 2s usage spike while the generated audio is about 10s long.

During this yield ReplyOnPause can be triggered by the same user session again, lead to an interaction with self.fn, and queue the response although the old audio has not finished yielding.
ReplyOnPause has a blocking mechanism in ReplyOnPause.receive by checking self.state.responding, however: from what I gathered self.state.responding is reset with self.reset() before the audio yield has finished. --> ReplyOnPause does not seem to be retriggered if speaking DURING the (e.g.) 2s spike. self.reset() sets self.state.responding back to False. Or it bypasses it somehow (tbh I'm not that deep in the source code).

What I did to work around it is to set the current time, estimate the expected duration of the audio yield, and set a manual block in ReplyOnPause that checks if enough time has elapsed. It works well enough and does not seem to break multi-user sessions (I guess because we get an individual handler per webrtc token?).

Feel free to correct any misconceptions!