Closed Bug 1528042 Opened 5 years ago Closed 10 months ago

Make enumerateDevices() more private ahead of getUserMedia() grant befind pref

Categories

(Core :: WebRTC: Audio/Video, enhancement, P2)

enhancement

Tracking

()

RESOLVED FIXED
115 Branch
Tracking Status
firefox67 --- wontfix
firefox115 --- fixed

People

(Reporter: jib, Assigned: jib)

References

(Depends on 1 open bug, Blocks 2 open bugs)

Details

Attachments

(12 files)

20.74 KB, image/png
Details
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review

Firefox already has a "privacy.resistFingerprinting" pref (bug 1372073) to reduce the fingerprinting surface of enumerateDevices().

Unfortunately, it renders site-driven device selection impossible, even after getUserMedia() access has been granted, making it nonviable as default behavior, for web compat reasons.

However, time has revealed that most WebRTC web sites implement device selection menus only after initial gUM grant, often with previews. The lack of labels in Firefox this early compounds this. [1]

So there's room to explore making the privacy.resistFingerprinting behavior [3] the default to sites that have NEVER been granted getUserMedia permission by the user, i.e. "the drive-by web", and then flip to today's behavior the first time the end user grants access to a site.

This could be done web compatibly by simulating the user inserting any additional devices found to be present at that point, firing the devicechange event, and updating the device list etc.

I believe Safari is exploring similar behavior [2], modulo some differences on repeat visits.

[1] https://webrtc.github.io/samples/src/content/devices/input-output/
[2] https://github.com/w3c/mediacapture-main/issues/559

[3] One question is what to do on systems without both a camera and microphone.
privacy.resistFingerprinting invents the missing (dummy) devices in that case.
We could consider keeping that behavior, perhaps with a caveat that once a
site has attempted getUserMedia() and succeeded or gotten NotFoundError, we'd
remove the invented devices, simulating their removal. The reasoning is that
most fingerprinting libraries won't risk triggering an actual user prompt, and
the information has already been revealed at this point (through NotFoundError).

Depends on: 1528056

Can confirm that this is breaking device selection in riot.im with resistFingerprint on. There is no way to select any camera apart from the default one.

See Also: → 1179084

The spec has finally stabilized here and Google Chrome has implemented the more private Safari model. https://github.com/w3c/mediacapture-main/pull/632

We should as well. Up-prioritizing this work.

Severity: normal → S3
Summary: Consider making enumerateDevices() more private ahead of initial getUserMedia() grant. → Make enumerateDevices() more private ahead of getUserMedia() grant.

Note we risk taking a web compat hit here by adding this until crbug 1101860 is fixed.

For context: labels are now limited to active capture, persistent permission is insufficient.

Part of this work is to update devicechange to fire less often to match the reduced enumerateDevices exposure.

Sites like Discord appear to already assume devicechange will only fire during active microphone use. The telltale sign this is firing when it shouldn't is "Device 2" since it doesn't have permission to show microphone labels.

Workaround: Discord could work around this Firefox bug by only showing the prompt if they're currently using a mic.

See Also: → 1223773
Attachment #9194508 - Attachment description: Bug 1528042 - Enumerate microphones, then cameras, then speakers (spec order). → Bug 1528042 - Enumerate microphones, then cameras, then speakers (spec order). r?karlt
Attachment #9194509 - Attachment description: Bug 1528042 - Limit exposure to 1 mic + 1 cam + 0 speakers wo/label, deviceId or groupId, unless DeviceInformationCanBeExposed. → Bug 1528042 - Limit enumerateDevices() fingerprinting vector ahead of getUserMedia success, to spec. r?karlt
Attachment #9194509 - Attachment description: Bug 1528042 - Limit enumerateDevices() fingerprinting vector ahead of getUserMedia success, to spec. r?karlt → Bug 1528042 - Limit enumerateDevices() fingerprinting vector ahead of (and after) getUserMedia success, to spec. r?karlt

These patches would bring enumerateDevices() up to spec, and close the fingerprinting leak that some sites have been using to work around bug 1609427 comment 11, so we may need an urgent solution there.

Depends on: 1609427
Blocks: 1528056
No longer depends on: 1528056
Attachment #9289344 - Attachment description: Bug 1528042 - Remove the now-unnecessary "Test must leave no active gUM streams behind" infra check. r?karlt → Bug 1528042 - Reimplement noGum() infra check for no active gUM streams left behind to no longer rely on enumerateDevices. r?karlt
Attachment #9291245 - Attachment description: Bug 1528042 - Fix broken WPT MediaDevices-enumerateDevices-persistent-permission.https.html r?karlt → Bug 1528042 - Fix broken WPT MediaDevices-enumerateDevices-persistent-permission.https.html by changing iframe enumeration to be without capture r?karlt
Attachment #9331177 - Attachment description: Bug 1528042 - Don't expose number of devices of a kind unless information can be exposed for that kind. r?karlt → Bug 1528042 - Don't expose number of devices of a kind unless information can be exposed for that kind per https://github.com/w3c/mediacapture-main/pull/900. r?karlt
Attachment #9333221 - Attachment description: Bug 1528042 - Test legacy enumerateDevices behavior. r?karlt → Bug 1528042 - Test legacy exposure of camera and microphone info from enumerateDevices() ahead of successful getUserMedia() calls. r?karlt
Attachment #9289343 - Attachment description: Bug 1528042 - Update some WPT and mochitests on groupId and output devices. r?karlt → Bug 1528042 - Add successful gUM calls to existing WPT and mochitests to let them continue to test full device information exposure in enumerateDevices(). r?karlt
Summary: Make enumerateDevices() more private ahead of getUserMedia() grant. → Make enumerateDevices() more private ahead of getUserMedia() grant befind pref

This is landing behind pref for now.

Pushed by jbruaroey@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/0ac40a8f40a0
Enumerate microphones, then cameras, then speakers (spec order). r=karlt
https://hg.mozilla.org/integration/autoland/rev/cf689822deaf
Limit enumerateDevices() fingerprinting vector ahead of (and after) getUserMedia success, to spec. r=karlt
https://hg.mozilla.org/integration/autoland/rev/c0373264c89b
Add media.devices.enumerate.legacy.enabled pref. r=karlt
https://hg.mozilla.org/integration/autoland/rev/e864d7f810b3
Fix navigator.mozGetUserMedia to also trigger device exposure. r=karlt
https://hg.mozilla.org/integration/autoland/rev/59b9147c85f0
Add successful gUM calls to existing WPT and mochitests to let them continue to test full device information exposure in enumerateDevices(). r=karlt
https://hg.mozilla.org/integration/autoland/rev/d0bfd185f0e7
Reimplement noGum() infra check for no active gUM streams left behind to no longer rely on enumerateDevices. r=karlt
https://hg.mozilla.org/integration/autoland/rev/cb2b433b47bd
Fix broken WPT MediaDevices-enumerateDevices-persistent-permission.https.html by changing iframe enumeration to be without capture r=karlt
https://hg.mozilla.org/integration/autoland/rev/94066f37c5a0
Test correct device list order in enumerateDevices(). r=karlt
https://hg.mozilla.org/integration/autoland/rev/a20768227ca6
Don't expose number of devices of a kind unless information can be exposed for that kind per https://github.com/w3c/mediacapture-main/pull/900. r=karlt
https://hg.mozilla.org/integration/autoland/rev/04d9fa0993ab
Test legacy exposure of camera and microphone info from enumerateDevices() ahead of successful getUserMedia() calls. r=karlt
Failed to create upstream wpt PR due to merge conflicts. This requires fixup from a wpt sync admin.

Backed out for causing build bustages on MediaDevices.cpp.

[task 2023-05-16T06:08:52.771Z] 06:08:52     INFO -  gmake[4]: Entering directory '/builds/worker/workspace/obj-build/dom/media'
[task 2023-05-16T06:08:52.777Z] 06:08:52     INFO -  /builds/worker/fetches/sccache/sccache /builds/worker/fetches/gcc/bin/g++ --sysroot /builds/worker/fetches/sysroot-x86_64-linux-gnu -std=gnu++17 -isystem /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/include/c++/7.5.0 -isystem /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/include/x86_64-linux-gnu/c++/7.5.0 -isystem /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/include/x86_64-linux-gnu -isystem /builds/worker/fetches/sysroot-x86_64-linux-gnu/usr/include -o Unified_cpp_dom_media2.o -c  -I/builds/worker/workspace/obj-build/dist/stl_wrappers -I/builds/worker/workspace/obj-build/dist/system_wrappers -include /builds/worker/checkouts/gecko/config/gcc_hidden.h -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -DDEBUG=1 -DHAVE_UINT64_T -DWEBRTC_MOZILLA_BUILD -DRTC_ENABLE_VP9 -DWEBRTC_POSIX -DWEBRTC_BUILD_LIBEVENT -DWEBRTC_LINUX -DWEBRTC_USE_X11 -DWEBRTC_USE_PIPEWIRE -DMOZILLA_INTERNAL_API -DOS_POSIX=1 -DOS_LINUX=1 -DMOZ_HAS_MOZGLUE -DMOZILLA_INTERNAL_API -DIMPL_LIBXUL -DSTATIC_EXPORTABLE_JS_API -I/builds/worker/checkouts/gecko/dom/media -I/builds/worker/workspace/obj-build/dom/media -I/builds/worker/checkouts/gecko/caps -I/builds/worker/checkouts/gecko/docshell/base -I/builds/worker/checkouts/gecko/dom/base -I/builds/worker/checkouts/gecko/dom/media/webrtc -I/builds/worker/checkouts/gecko/layout/generic -I/builds/worker/checkouts/gecko/layout/xul -I/builds/worker/checkouts/gecko/media/libyuv/libyuv/include -I/builds/worker/checkouts/gecko/netwerk/base -I/builds/worker/checkouts/gecko/toolkit/content/tests/browser -I/builds/worker/checkouts/gecko/dom/media/webrtc/common -I/builds/worker/checkouts/gecko/third_party/libwebrtc -I/builds/worker/checkouts/gecko/third_party/libwebrtc/third_party/abseil-cpp -I/builds/worker/workspace/obj-build/ipc/ipdl/_ipdlheaders -I/builds/worker/checkouts/gecko/ipc/chromium/src -I/builds/worker/workspace/obj-build/dist/include -I/builds/worker/workspace/obj-build/dist/include/nspr -I/builds/worker/workspace/obj-build/dist/include/nss -DMOZILLA_CLIENT -include /builds/worker/workspace/obj-build/mozilla-config.h -D_GLIBCXX_USE_CXX11_ABI=0 -fno-sized-deallocation -fno-aligned-new -fno-exceptions -fPIC -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pthread -pipe -gdwarf-4 -freorder-blocks -Os -fno-omit-frame-pointer -funwind-tables -Werror -Wall -Wempty-body -Wignored-qualifiers -Wpointer-arith -Wsign-compare -Wtype-limits -Wunreachable-code -Wno-invalid-offsetof -Wno-error=deprecated -Wduplicated-cond -Wimplicit-fallthrough -Wlogical-op -Wno-error=maybe-uninitialized -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=free-nonheap-object -Wformat -Wformat-overflow=2 -Wno-psabi -Wno-error=attributes -Werror=switch -fno-strict-aliasing -ffp-contract=off  -MD -MP -MF .deps/Unified_cpp_dom_media2.o.pp   Unified_cpp_dom_media2.cpp
[task 2023-05-16T06:08:52.777Z] 06:08:52     INFO -  In file included from Unified_cpp_dom_media2.cpp:110:0:
[task 2023-05-16T06:08:52.777Z] 06:08:52     INFO -  /builds/worker/checkouts/gecko/dom/media/MediaDevices.cpp: In member function 'bool mozilla::dom::MediaDevices::CanExposeInfo(mozilla::dom::MediaDeviceKind) const':
[task 2023-05-16T06:08:52.777Z] 06:08:52    ERROR -  /builds/worker/checkouts/gecko/dom/media/MediaDevices.cpp:343:1: error: control reaches end of non-void function [-Werror=return-type]
[task 2023-05-16T06:08:52.777Z] 06:08:52     INFO -   };
[task 2023-05-16T06:08:52.777Z] 06:08:52     INFO -   ^
[task 2023-05-16T06:08:52.777Z] 06:08:52     INFO -  cc1plus: all warnings being treated as errors
[task 2023-05-16T06:08:52.777Z] 06:08:52    ERROR -  gmake[4]: *** [/builds/worker/checkouts/gecko/config/rules.mk:668: Unified_cpp_dom_media2.o] Error 1
[task 2023-05-16T06:08:52.777Z] 06:08:52     INFO -  gmake[4]: Leaving directory '/builds/worker/workspace/obj-build/dom/media'
[task 2023-05-16T06:08:52.777Z] 06:08:52     INFO -  gmake[4]: Target 'target-objects' not remade because of errors.
[task 2023-05-16T06:08:52.777Z] 06:08:52    ERROR -  gmake[3]: *** [/builds/worker/checkouts/gecko/config/recurse.mk:72: dom/media/target-objects] Error 2
[task 2023-05-16T06:08:52.777Z] 06:08:52     INFO -  gmake[4]: Entering directory '/builds/worker/workspace/obj-build/layout/ipc'
Flags: needinfo?(jib)
Depends on: 1835146
Attachment #9291245 - Attachment description: Bug 1528042 - Fix broken WPT MediaDevices-enumerateDevices-persistent-permission.https.html by changing iframe enumeration to be without capture r?karlt → Bug 1528042 - Update expectations file for WPT MediaDevices-enumerateDevices-persistent-permission.https.html r?karlt
Pushed by jbruaroey@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/ea975297396d
Enumerate microphones, then cameras, then speakers (spec order). r=karlt
https://hg.mozilla.org/integration/autoland/rev/55c6b85b37c1
Limit enumerateDevices() fingerprinting vector ahead of (and after) getUserMedia success, to spec. r=karlt
https://hg.mozilla.org/integration/autoland/rev/b47fdabb6f86
Add media.devices.enumerate.legacy.enabled pref. r=karlt
https://hg.mozilla.org/integration/autoland/rev/8c51701fb02a
Fix navigator.mozGetUserMedia to also trigger device exposure. r=karlt
https://hg.mozilla.org/integration/autoland/rev/173f133fb868
Add successful gUM calls to existing WPT and mochitests to let them continue to test full device information exposure in enumerateDevices(). r=karlt
https://hg.mozilla.org/integration/autoland/rev/016bced62d48
Reimplement noGum() infra check for no active gUM streams left behind to no longer rely on enumerateDevices. r=karlt
https://hg.mozilla.org/integration/autoland/rev/375bade19a5c
Update expectations file for WPT MediaDevices-enumerateDevices-persistent-permission.https.html r=karlt
https://hg.mozilla.org/integration/autoland/rev/6c8c76da9ab7
Test correct device list order in enumerateDevices(). r=karlt
https://hg.mozilla.org/integration/autoland/rev/364f837033b8
Don't expose number of devices of a kind unless information can be exposed for that kind per https://github.com/w3c/mediacapture-main/pull/900. r=karlt
https://hg.mozilla.org/integration/autoland/rev/a809892ffbc5
Test legacy exposure of camera and microphone info from enumerateDevices() ahead of successful getUserMedia() calls. r=karlt
Flags: needinfo?(jib)
Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/40263 for changes under testing/web-platform/tests
Upstream PR merged by moz-wptsync-bot
Regressions: 1836377
Blocks: 1843434
See Also: 13979781732410
See Also: 1223773
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: