Skip to content

fix(session-replay): revert max key-frame interval to once per video segment #5156

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 2, 2025

Conversation

philprime
Copy link
Contributor

📜 Description

Changes the session replay video segment keyframe interval to 1.

💡 Motivation and Context

H.264 uses different types of frames:

  • I‑frames are the least compressible but don't require other video frames to decode.
  • P‑frames can use data from previous frames to decompress and are more compressible than I‑frames.

In #5134 we introduced a change in the video segment configuration to only use I-Frame by setting AVVideoMaxKeyFrameIntervalKey to increase (browser) compatibility.

Further testing surfaced now that the browser compatibility is not affected by this option, but it increases the file size of video segments:

$ ffprobe -select_streams v:0 -show_frames -show_entries frame=pict_type,coded_picture_number -of csv -hide_banner pre-5134.mp4
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'older.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 1
    compatible_brands: isommp41mp42
    creation_time   : 2025-04-24T14:50:59.000000Z
  Duration: 00:00:05.00, start: 0.000000, bitrate: 14 kb/s
  Stream #0:0[0x1](und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p(tv, progressive), 393x852 [SAR 1:1 DAR 131:284], 13 kb/s, 1 fps, 1 tbr, 600 tbn (default)
      Metadata:
        creation_time   : 2025-04-24T14:50:59.000000Z
        handler_name    : Core Media Video
        vendor_id       : [0][0][0][0]
frame,I,H.26[45] User Data Unregistered SEI message
frame,P,H.26[45] User Data Unregistered SEI message
frame,P,H.26[45] User Data Unregistered SEI message
frame,P,H.26[45] User Data Unregistered SEI message
frame,P,H.26[45] User Data Unregistered SEI message
$ ffprobe -select_streams v:0 -show_frames -show_entries frame=pict_type,coded_picture_number -of csv -hide_banner post-5134.mp4
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'newer.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 1
    compatible_brands: isommp41mp42
    creation_time   : 2025-04-24T14:49:05.000000Z
  Duration: 00:00:05.00, start: 0.000000, bitrate: 31 kb/s
  Stream #0:0[0x1](und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 393x852 [SAR 1:1 DAR 131:284], 29 kb/s, 1 fps, 1 tbr, 600 tbn (default)
      Metadata:
        creation_time   : 2025-04-24T14:49:05.000000Z
        handler_name    : Core Media Video
        vendor_id       : [0][0][0][0]
frame,I,H.26[45] User Data Unregistered SEI message
frame,I,H.26[45] User Data Unregistered SEI message
frame,I,H.26[45] User Data Unregistered SEI message
frame,I,H.26[45] User Data Unregistered SEI message
frame,I,H.26[45] User Data Unregistered SEI message

The actual difference in size varies depending on the amount of changes in the video, but we could see up to 100% larger sizes.

Using the latest changes on main this replay works in Chrome, Firefox and Safari.

💚 How did you test it?

For this replay I changed I-Frames interval to 6 (1 I-Frame per 5 second video segment at 1 FPS, the others are P-Frames) which reduced the size from 19KB to 6KB for the first segment, while it still works in all browsers.

📝 Checklist

You have to check all boxes before merging:

  • I added tests to verify the changes.
  • No new PII added or SDK only sends newly added PII if sendDefaultPII is enabled.
  • I updated the docs if needed.
  • I updated the wizard if needed.
  • Review from the native team if needed.
  • No breaking change or entry added to the changelog.
  • No breaking change for hybrid SDKs or communicated to hybrid SDKs.

@philprime philprime self-assigned this Apr 30, 2025
@philprime philprime marked this pull request as ready for review April 30, 2025 10:04
Copy link

codecov bot commented Apr 30, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.782%. Comparing base (c40037e) to head (08dfa4f).
Report is 5 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@              Coverage Diff              @@
##              main     #5156       +/-   ##
=============================================
+ Coverage   92.774%   92.782%   +0.007%     
=============================================
  Files          677       677               
  Lines        84275     84270        -5     
  Branches     30658     30645       -13     
=============================================
+ Hits         78186     78188        +2     
+ Misses        5987      5980        -7     
  Partials       102       102               
Files with missing lines Coverage Δ
...egrations/SessionReplay/SentryOnDemandReplay.swift 93.538% <100.000%> (+0.019%) ⬆️
...ions/SessionReplay/SentryOnDemandReplayTests.swift 92.792% <100.000%> (ø)

... and 16 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c40037e...08dfa4f. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Contributor

Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1233.62 ms 1251.70 ms 18.08 ms
Size 22.30 KiB 850.93 KiB 828.62 KiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
c6504da 1232.06 ms 1243.28 ms 11.22 ms
c6b920a 1239.53 ms 1256.53 ms 17.00 ms
318a891 1245.14 ms 1259.02 ms 13.88 ms
5e769dd 1216.24 ms 1245.74 ms 29.50 ms
7cd187e 1243.04 ms 1244.79 ms 1.75 ms
4329cdb 1206.96 ms 1227.74 ms 20.78 ms
6813f7c 1232.23 ms 1251.47 ms 19.24 ms
4350d44 1228.75 ms 1246.75 ms 18.00 ms
a176fc4 1226.24 ms 1247.50 ms 21.26 ms
25bcc50 1237.69 ms 1258.40 ms 20.71 ms

App size

Revision Plain With Sentry Diff
c6504da 20.76 KiB 414.44 KiB 393.69 KiB
c6b920a 21.58 KiB 631.19 KiB 609.61 KiB
318a891 22.30 KiB 750.02 KiB 727.72 KiB
5e769dd 21.58 KiB 572.21 KiB 550.62 KiB
7cd187e 20.76 KiB 401.65 KiB 380.89 KiB
4329cdb 22.30 KiB 833.10 KiB 810.80 KiB
6813f7c 21.58 KiB 614.86 KiB 593.28 KiB
4350d44 21.58 KiB 629.82 KiB 608.24 KiB
a176fc4 22.84 KiB 403.24 KiB 380.39 KiB
25bcc50 20.76 KiB 427.23 KiB 406.46 KiB

@philprime philprime requested a review from romtsn April 30, 2025 13:34
@philprime philprime merged commit a528625 into main May 2, 2025
78 checks passed
@philprime philprime deleted the philprime/session-replay-iframe branch May 2, 2025 07:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants