Skip to content

Conversation

@BenHodgsonR2R
Copy link

@BenHodgsonR2R BenHodgsonR2R commented Nov 15, 2025

Issues:
#629
#525

Improve screenshare performance by using adaptive constraints

Problem

When sharing games or other applications via screenshare, users experience significant performance degradation in the shared application and/or the stream itself. This is particularly noticeable when using the "Prefer Clarity" (detail) content mode.

The issue stems from the use of min constraints and the advanced constraint array. According to the W3C Media Capture and Streams specification:

Constraints which are specified using any or all of max, min, or exact are always treated as mandatory. If any constraint which uses one or more of those can't be met when calling applyConstraints(), the promise will be rejected.

Additionally, bare values in the advanced array are treated as exact constraints, which creates rigid requirements that the capture system must work hard to maintain.

When the system cannot maintain mandatory minimums, it will:

  • Duplicate frames to hit minimum frame rate requirements
  • Block/wait for frames from the capture source
  • Apply frame rate conversion with CPU overhead
  • Put additional load on the shared application's GPU/CPU

Solution

This PR introduces content-aware adaptive constraints that match the user's selected content hint:

"motion" mode (Prefer Smoothness):

  • Uses min constraint for frame rate to ensure smooth playback
  • Uses ideal constraint for resolution to allow adaptive scaling

"detail" mode (Prefer Clarity):

  • Uses ideal constraints for both frame rate and resolution
  • Allows the capture system to gracefully adapt when the shared application is under load
  • No mandatory minimums that force frame duplication or blocking

Both modes:

  • Removed the advanced constraint array that was forcing exact resolution matches
  • Lets basic constraints handle adaptation naturally per W3C spec guidance

With ideal-only constraints, the browser targets the specified values but can drop below them without penalty, removing the performance overhead on the shared application.

Technical Details

The key insight is that ideal values are treated as optimization targets, not mandatory requirements. From the spec:

ideal - A decimal number specifying an ideal value for the property. If possible, this value will be used, but if it's not possible, the user agent will use the closest possible match.

This allows the capture system to balance quality vs. performance dynamically based on what the shared application can actually provide.

Testing

Tested with game streaming on Linux. Performance improvements are noticeable in "detail" mode with no regression in "motion" mode behavior.

@BenHodgsonR2R BenHodgsonR2R marked this pull request as ready for review November 15, 2025 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant