Update screencast_thread.rs for support fps limit #152
Conversation
src/screencast_thread.rs
Outdated
| None => return Err(anyhow::anyhow!("failed to use shm capture to get size")), | ||
| }; | ||
|
|
||
| let framerate = 60; // Default framerate. XXX is there a better way? |
There was a problem hiding this comment.
Yeah, we simply shouldn't wait by default. The compositor won't do unlimited frame rates, but synchronize captures to the screen's frame rate.
This is also why I am not sure this approach in general is such a good idea, as the compositor could potentially introduce another delay, if we send the capture request earlier in the redraw cycle, after we already waited to hit the target frame rate.
Worse, if we wait the full duration, it might happen that the request reaches a busy compositor to late to be effective for the current draw cycle, potentially cutting our frame rate in half.
So a good implementation of this should imo, track the current frame rate and only if we are ahead of the target, delay the next frame by a full frame minus some safety margin.
There was a problem hiding this comment.
I noticed something that seems strange. When I capture a 60Hz screen, if the screen is mostly empty, the captured FPS is approximately 47 FPS. However, if I move the mouse, the FPS immediately drops significantly. This effect is even more noticeable when watching a video (whether it's 24 FPS or 60 FPS), as the captured FPS remains around 23-24 FPS. I don't understand what is causing this behavior.
There was a problem hiding this comment.
This effect is even more noticeable when watching a video (whether it's 24 FPS or 60 FPS), as the captured FPS remains around 23-24 FPS. I don't understand what is causing this behavior.
Cosmic-comp doesn't draw at a constant frame rate, when nothing on the screen changes. So sampling from previous frame times doesn't work. You also shouldn't assume the refresh rate of the modes given by wl_output actually match the targeted framerate, as VRR and other things can mess with it.
The frame rate hint should purely be used to trottle further requests, should you receive frames too quickly.
There was a problem hiding this comment.
I assume that only throttling when we receive frames too quickly would be fine if the framerate is at a sufficiently high level (the higher the framerate, the better). However, suppose we have a very low framerate, such as 2 frames per second. We would quickly reach the limit of 2 frames, and then the next frame would have to wait for a certain period until the start of the next cycle.
As a result, we would encounter a situation where frames are distributed unevenly within a capture cycle. For example, if the compositor draws 60 different frames within one capture cycle, and we only take ~100ms to capture the first 2 frames, we would then have to wait ~900ms until the next capture cycle, causing all frames drawn during this period to be skipped.
I want a more even distribution of captured frames within a cycle. So, is there a threshold value to consider as a low frequency, where we would wait for the appropriate moment before requesting a frame capture? In other cases, we would only throttle if we receive frames too quickly.
There was a problem hiding this comment.
I want a more even distribution of captured frames within a cycle.
Looking at cosmic-comp's code again, frame pacing should be possible. A new capture request will queue a redraw, even if that happens to be empty (no changes since the last draw), it will update captures.
So worse case the request will still have to wait until the next redraw, which might be a full vblank cycle.
But I guess delaying by targeted frame-rate minus some time to allow processing in the compositor should work fine. If nothing else, you could wait half the targeted frame rate, submit and then make up for the rest of the time by throttling. The code should just avoid to unconditionally wait the full frame time, as that will guarantee you don't hit the frame rate in the first place.
But the code shouldn't assume the compositor's refresh rate as there is really no stable source for that kind of information.
- Read the refresh rate of the captured screen and assign it to the `framerate` of `StreamData`. - Optimize the waiting mechanism for the next frame based on @Drakulix's advice [advice](pop-os#152 (comment)) by prioritizing frame capture and sending it to PipeWire first. Then, calculate the time taken to capture that frame, and if we are ahead of the next frame's capture time, wait for approximately `delay_ns = target_ns - frame_took_ns`.
Balancing Frame Pacing and Frame Throttling Implement flexible frame throttling by: 1. Waiting for half of the target frame duration (`delay_til_next_frame_ns`). 2. Calculating the elapsed time for a frame capture: - If we are ahead of the next frame capture time, wait for `delay_til_next_frame_ns`. - If we are behind the next frame capture time, accumulate the delay into `accumulated_frame_debt_ns`. 3. Using `accumulated_frame_debt_ns` to reduce waiting time in `delay_before_capture_frame_ns` and `delay_til_next_frame_ns`, helping to reach the target frame rate.
#149
Needs testing. I am using std::thread::sleep to block the thread until the next frame capture time. I'm not sure if it will cause any issues.