fix: screenshare performance with adaptive constraints #1202
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issues:
#629
#525
Improve screenshare performance by using adaptive constraints
Problem
When sharing games or other applications via screenshare, users experience significant performance degradation in the shared application and/or the stream itself. This is particularly noticeable when using the "Prefer Clarity" (detail) content mode.
The issue stems from the use of
minconstraints and theadvancedconstraint array. According to the W3C Media Capture and Streams specification:Additionally, bare values in the
advancedarray are treated asexactconstraints, which creates rigid requirements that the capture system must work hard to maintain.When the system cannot maintain mandatory minimums, it will:
Solution
This PR introduces content-aware adaptive constraints that match the user's selected content hint:
"motion" mode (Prefer Smoothness):
minconstraint for frame rate to ensure smooth playbackidealconstraint for resolution to allow adaptive scaling"detail" mode (Prefer Clarity):
idealconstraints for both frame rate and resolutionBoth modes:
advancedconstraint array that was forcing exact resolution matchesWith
ideal-only constraints, the browser targets the specified values but can drop below them without penalty, removing the performance overhead on the shared application.Technical Details
The key insight is that
idealvalues are treated as optimization targets, not mandatory requirements. From the spec:This allows the capture system to balance quality vs. performance dynamically based on what the shared application can actually provide.
Testing
Tested with game streaming on Linux. Performance improvements are noticeable in "detail" mode with no regression in "motion" mode behavior.