Open Bug 1610199 (egl-linux-vaapi) Opened 4 years ago Updated 1 month ago

[meta][Linux/EGL] Implement ffmpeg/VAAPI video playback

Categories

(Core :: Audio/Video: Playback, enhancement, P3)

enhancement

Tracking

()

People

(Reporter: stransky, Assigned: stransky)

References

(Depends on 28 open bugs)

Details

(Keywords: meta)

Attachments

(1 file, 4 obsolete files)

With dmabuf support already implemented it should be feasible to implement VAAPI playback by ffmpeg.

I already checked that the playback works fine on Wayland/Intel on mpv player which also uses ffmpeg to decode frames and vaExportSurfaceHandle() to convert AVFrame to dmabuf memory layout.

Some integration work needs to be done to carry dmabuf buffer by MediaDataDecoder::DecodedData to correct TextureClient to map it as EGL texture.

playback works fine on Wayland/Intel on mpv player

This require intel-vaapi-driver 2.4.0+, correct? At least according to https://github.com/intel/intel-vaapi-driver/issues/449

I use Fedora 31 as a reference distro with latest updates (libva-2.6.0).

Do you also plan to implement video encoding hardware acceleration through VAAPI? It could help quite a lot for video conferencing scenarios.

And a bit of a side question, in case of Vulkan + Wayland, would it use something else rather than EGL?

(In reply to Shmerl from comment #3)

Do you also plan to implement video encoding hardware acceleration through VAAPI? It could help quite a lot for video conferencing scenarios.

No, that's implemented by WebRTC project (https://webrtc.org/) so it needs to be implemented there. This work also does not help with video played via WebRTC like video conferencing.

You mean that Firefox uses WebRTC project code, and I should check there for VAAPI support?

WebRTC on Linux a major problem for clients like Cisco Webex, which constantly chokes in Firefox, complaining that "CPU is overutilized". I thought may be hardware acceleration can help it.

(In reply to Shmerl from comment #6)

You mean that Firefox uses WebRTC project code, and I should check there for VAAPI support?

Yes, exactly. Firefox uses WebRTC for video conferences which also includes video encode/decode there.

Hm, according to https://webrtc.org/web-apis/firefox/

For specific issues with the FIrefox WebRTC code, please use our bug tracker

And it points back to bugzilla here. I can open another bug, for WebRTC component.

(In reply to Shmerl from comment #4)

And a bit of a side question, in case of Vulkan + Wayland, would it use something else rather than EGL?

It will use vulkan extentions instead of EGL.
Importing VAAPI decoded frames to vulkan surface is not yet supported. There is a similar concept implemented on mpv though which fails on my configuration(amdgpu).

Pardon me if this is off-topic, but is this bug tracking all implementations of VA-API, or just the Intel one for now?

(In reply to Sandi Vujaković from comment #10)

Pardon me if this is off-topic, but is this bug tracking all implementations of VA-API, or just the Intel one for now?

This bug is tracking a playback by ffmpeg (image decode) and libva (decoded image transfer to dmabuf).

Just for the reference, if anyone will be looking. I opened WebRTC bug for VAAPI support: https://bugs.chromium.org/p/webrtc/issues/detail?id=11296

(In reply to Martin Stránský [:stransky] from comment #7)

Yes, exactly. Firefox uses WebRTC for video conferences which also includes video encode/decode there.

Really unfortunate that we're using the reference/Chrome implementation (libwebrtc). In GStreamer, this would be easy to do >_<

But anyway, if anyone would be interested in modifying libwebrtc to do this, seems like there was an attempt to do a similar thing for Android: bug 969395.

Is Firefox on Android not using libwebrtc? Otherwise it's upstream issue like above.

Attached patch WIP (obsolete) — Splinter Review

WIP patch. Video has wrong colors (perhaps due to wrong YUV color profile) and frames are shuttered. But decodes frames on gpu and pass them by dmabuf to compositor as gl textures.

Attachment #9122629 - Attachment is patch: true

(In reply to Martin Stránský [:stransky] from comment #15)

Created attachment 9122629 [details] [diff] [review]
WIP

WIP patch. Video has wrong colors (perhaps due to wrong YUV color profile) and frames are shuttered. But decodes frames on gpu and pass them by dmabuf to compositor as gl textures.

I remember having problems like this when trying to get VAAPI/DRI3 to work with Chromium/X11 under Wayland. It worked fine with Weston, but under GNOME it would have the wrong colour profile, due to necessary colourspace conversions not being fully supported in mutter. I don't know if the situation is still the same, but I wouldn't be all that surprised if it exercised similar code paths with wayland-egl-dmabuf. Maybe this is a Red Herring, but I thought I'd mention it.

Thanks, I already solved the wrong colors with moving to WebRender which supports NV12 format correctly. Last issue remaining is frame scattering, seems to be caused by conversion from DRI prime to GL texture - at least ffmpeg internal conversion works fine.

Is this hooked up to the "Media Capabilities API" so powerEfficient return true if VAAPI is available?

Depends on: 1613358
No longer blocks: egl-linux-dmabuf
Depends on: 1613364

(In reply to Kristian Klausen from comment #19)

Is this hooked up to the "Media Capabilities API" so powerEfficient return true if VAAPI is available?

This doesn't exist on any OS - see bug 1569686 to track this.

Martin,
In the WIP patch you attached earlier, you call av_hwdevice_ctx_create(). It's possible to build ffmpeg without CONFIG_LIBDRM, and in that case it will call vaGetDisplay() and not vaGetDisplayDRM() expected by your code. To ensure X11 version is never called, you can specify "connection_type" in options:

bool FFmpegDataDecoder<LIBAV_VER>::CreateVAAPIDeviceContext() {
  AVDictionary *opts = nullptr;
  mLib->av_dict_set(&opts, "connection_type", "drm", 0);
  if (mLib->av_hwdevice_ctx_create(&mVAAPIDeviceContext, AV_HWDEVICE_TYPE_VAAPI,
                                   NULL, opts, 0) < 0) {
    mLib->av_dict_free(&opts);
    …

Thanks, I'll update the patch.

Hi, just a small question if I'm not bothering you. When will the initial patches land in nightly? I would like to participate with testing this, but my PC dislikes compiling from source because my CPU is too weak.

Depends on: 1615604

(In reply to Marko from comment #23)

Hi, just a small question if I'm not bothering you. When will the initial patches land in nightly? I would like to participate with testing this, but my PC dislikes compiling from source because my CPU is too weak.

I have no idea.

Attached patch vaapi v1 (obsolete) — Splinter Review

First version of the patch for latest trunk, seems to be working somehow.

Attachment #9122629 - Attachment is obsolete: true
Depends on: 1616128
Depends on: 1616129
Depends on: 1616185
Depends on: 1616590
Depends on: 1616680
Depends on: 1616892

All needed patches are submitted in the tracked bugs here so we'll have working VAAPI on Wayland when these ones hit mozilla-central.
My performance measures are here - https://bugzilla.mozilla.org/show_bug.cgi?id=1616185#c7

(In reply to Martin Stránský [:stransky] from comment #26)

All needed patches are submitted in the tracked bugs here so we'll have working VAAPI on Wayland when these ones hit mozilla-central.
My performance measures are here - https://bugzilla.mozilla.org/show_bug.cgi?id=1616185#c7

What preferences are needed to enable all of it?

(In reply to Martin Stránský [:stransky] from comment #26)

All needed patches are submitted in the tracked bugs here so we'll have working VAAPI on Wayland when these ones hit mozilla-central.
My performance measures are here - https://bugzilla.mozilla.org/show_bug.cgi?id=1616185#c7

Awesome! To bad there are no reaction possibilities in bugzilla, this post deserves a thousand thumbs up and the likes.

Looks like you could rightfully claim https://www.bountysource.com/issues/55506502-add-va-api-hardware-decoding-support-on-linux now :)

If you can document all the required preferences, I'll enable it for testing.

Flags: needinfo?(stransky)

It doesn't work yet, Bug 1616129 and Bug 1616185 need to be fixed first.

Flags: needinfo?(stransky)
Depends on: 1618367
Attachment #9126905 - Attachment is obsolete: true
Depends on: 1619258

As Bug 1616185 landed you should be able to run va-api accelerated video playback on Wayland with latest nightly. I just tried 20200302212732 build and it works for me on Fedora 31 / Gnome / Mesa DRI Intel(R) UHD Graphics 630.

In order to run it you need to:

  • run Firefox under wayland compositor with MOZ_ENABLE_WAYLAND=1
  • set widget.wayland-dmabuf-vaapi.enabled to true at about:config
  • enable appropriate HW acceleration for Firefox (GL compositor or WebRender), verify that in about:support
  • have libva installed (I have libva-2.6.0-0.1.fc31.x86_64 on Fedora)

You can verify that VA-API is enabled by running Firefox with MOZ_LOG="PlatformDecoderModule:5" env variable and check in the log output that VA-API is enabled and used.

Attached file log (obsolete) —
Doesn't work on amdgpu with webrender. 

```

Confirmed. vaapi doesn't work with gl either. Aidan too reported the same error.

Blocks: 1210726
No longer blocks: 1210727

I see the same thing on Arch Linux with Intel iGPU:

[Child 14174: Main Thread]: D/PlatformDecoderModule Couldn't load function avcodec_get_hw_config
[Child 14174: Main Thread]: D/PlatformDecoderModule Couldn't load function av_hwdevice_ctx_create
[Child 14174: Main Thread]: D/PlatformDecoderModule Couldn't load function av_hwframe_transfer_get_formats
[Child 14174: Main Thread]: D/PlatformDecoderModule Couldn't load function av_hwdevice_ctx_create_derived
[Child 14174: Main Thread]: D/PlatformDecoderModule Sandbox decoder rejects requested type

I verified that libva.so.2.600.0, libva-drm.so.2.600.0 and libva-x11.so.2.600.0 are loaded in the content process. I also attached gdb to the content process and checked that disas can find the four functions above.

(In reply to Akarshan Biswas from comment #32)

Doesn't work on amdgpu with webrender.

amdgpu should be supported, It worked for me when I tested it some time ago. If there's any bug I'll fix that.

from the log:
[Child 2692: MediaPDecoder #1]: D/PlatformDecoderModule VA-API FFmpeg is disabled by platform

that means:

  • you're not running on Wayland or dmabuf is not supported
  • it's disabled by a preference or GL/Webrender is disabled

Please file a new bug for it and attach about:support page and also firefox log obtained by running:

MOZ_ENABLE_WAYLAND=1 MOZ_LOG="WidgetWayland:5" ./firefox

Thanks.

Guys, please file a new bug for it against Core / Audio,Video playback component (or just clone this one) and drop a link here.
Thanks.

(In reply to Martin Stránský [:stransky] from comment #35)

[Child 2692: MediaPDecoder #1]: D/PlatformDecoderModule VA-API FFmpeg is disabled by platform
I don't think the error is related to this. The error is on bug id 1616128 I think (loading library symbols from ffmpeg libraries )

But anyway, I will open a new bug report with all the logs.

Depends on: 1619530
Depends on: 1619543
Depends on: 1619544
Alias: wayland-vaapi
Depends on: 1619585
No longer depends on: 1619530
Depends on: 1619882
Depends on: 1620821

Video flickering on BBC video player when controls appear and disappear:
https://bugzilla.mozilla.org/show_bug.cgi?id=1621951

Depends on: 1622132
Depends on: 1622425

https://crash-stats.mozilla.org/report/index/87c543ef-efc7-4da8-abeb-243a70200318

This crash happens to me on Gnome shell 3.36.0 / Mutter 3.36.0 on Fedora 32 beta. It worked on Gnome Shell 3.34.3 / Mutter 3.34.3,

Is this bug already documented?

Depends on: 1622729

Hm why is it marked as fixed if it still occurs?

It was fixed 2 hours ago. Your build id is 20200318041411 (2020-03-18 04:14:11). Now we have to wait for the next update of Nightly.
Edit: You could also try out Linux x64 shippable opt build from bug 1622729 comment 4. If that crash still occurs, please comment in bug 1622729.

(In reply to Jan Andre Ikenmeyer [:darkspirit] from comment #42)

It was fixed 2 hours ago. Your build id is 20200318041411 (2020-03-18 04:14:11). Now we have to wait for the next update of Nightly.
Edit: You could also try out https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/A_UE9lP6T4aNqonpQDe-eQ/runs/0/artifacts/public/build/target.tar.bz2 (Linux x64 shippable opt). If it doesn't work, please comment in bug 1622729, not here.

Ooooooh such a coincidence that it got fixed just today :D I've been eyeing this bug for days now.

https://i.imgur.com/1jxhIe1.png

Why doesn't it work for 4k 60fps videos on youtube yet? The CPU usage goes bonkers and videos stutter like crazy. I have intel UHD 620 and my vainfo shows all H264, VP9 and HEVC decoding supported.

You might not be using VAAPI at all because of the sandbox (bug 1619585). Stuttering especially in case of overlays is known (bug 1619882) - or you might be decoding on the CPU. If you have multiple GPUs, it's bug 1622132. Only use https://nightly.mozilla.org for VAAPI testing.
Run $ GDK_BACKEND=wayland MOZ_LOG="PlatformDecoderModule:5" path/to/firefox to find out what's going on. Please create an own ticket and attach the log to it if you can't figure out what's the cause. If it's the sandbox, try to use env var LIBVA_DRIVER_NAME=i965 as well.

(In reply to Marko from comment #47)

https://gitlab.freedesktop.org/snippets/922
Sry here is the proper link for the log.

Marko, please open individual issues. This is a general bug for the implementation with lots of people subscribed that are not necessarily interested why things don't work on individual setups. You can then mark your new bugs as blockers for this one. Thx.

Depends on: 1623584
Depends on: 1624103
Depends on: 1624441
Depends on: 1625431

I wander if Bug 869813 "Option to set WebM or H.264 as preferred" could help Bug 1625431 and/or Bug 1624103
Not sure.
Thanks for your work.

Depends on: 1628690
Depends on: 1628746
Depends on: 1630754
Depends on: 1632059
Depends on: 1632095
Depends on: 1632456
Depends on: 1633273

Did anyone notice weird stuttering/shaking issues with 60 FPS H264 videos? They seem to happen exactly when the Youtube control appear and disappear. I'm on Arch Linux with Firefox 75 Wayland and intel-media-driver (LIBVA_DRIVER_NAME=iHD), with Intel UHD Graphics 630 integrated graphic card. No problems at all with 30 fps videos. Mpv seem to work fine with vaapi and 60 FPS H264 videos.

@michelesr

This is Bug 1619882

Depends on: 1639115
No longer depends on: 1639115

Just for the reference, since libwebrtc support came up before, I commented here with some details from libwebrtc developers: https://bugzilla.mozilla.org/show_bug.cgi?id=969395#c101

who basically recommend for Firefox to implement VAAPI support outside of libwebrtc anyway. So may be this effort can be somehow shared with the one for regular video handling.

Depends on: 1641603
Depends on: 1643855
Depends on: 1644546
Depends on: 1645671
Depends on: 1645673
Depends on: 1645678
Depends on: 1645672
Depends on: 1646051
Depends on: 1646329

I just updated nightly and while now dmabuf.video-textures is not needed, I get some green frames when GPU decoding. I don't see it mentioned somewhere which is why I comment. Maybe related to #1645673 ?

Alias: wayland-vaapi → egl-linux-vaapi
Depends on: 1619523
Summary: [Wayland] Implement ffmpeg/VAAPI video playback → [Linux/EGL] Implement ffmpeg/VAAPI video playback

With Bug 1619523 added (vaapi via dmabuf/x11) let's track all linux va-api issues as the X11/Wayland va-api&dmabuf code is almost identical.

(In reply to gsf.greece from comment #53)

I just updated nightly and while now dmabuf.video-textures is not needed, I get some green frames when GPU decoding. I don't see it mentioned somewhere which is why I comment. Maybe related to #1645673 ?

Let's finish X11/dmabuf/vaapi implementation first. I'll look at those minor ones after that as I expect they affect both X11 and Wayland backends.

Martin, are you going to make vaapi on X11 too? I understood that you could not and you left this work to other person. Or are you helping him? It's curiosity to know if this work is going to some place by you or by other that I have not seen XD.

And about the post in the other site, okey, didn't know it. But nevermind, thank to you for this.

(In reply to albertogomezmarin from comment #56)

Martin, are you going to make vaapi on X11 too?

Yes, a basic implementation is waiting at Bug 1619523.

Depends on: 1588904

but anyone are(In reply to Martin Stránský [:stransky] from comment #57)

(In reply to albertogomezmarin from comment #56)

Martin, are you going to make vaapi on X11 too?

Yes, a basic implementation is waiting at Bug 1619523.

but anybody has yet be assigned to this work. so until anybody go ahead you cant do nothing. right?

Blocks: 1210727
Depends on: 1647869
Depends on: 1650520

I have tried EGL on my computer with amdgpu drivers. The only thing I noticed is that the close windows is transprent like in the nvidia driver bug. But I think is another thing in WR.

Apart from this, I have tried to use vaapi and it doesn't work. I have seen VCN block not being used when playing a video in youtube so..
The video was in VP9 format that my ryzen is capable to decode and I have tested in chromium so...
maybe someone can help me a little bit?

Firefox-nightly with EGL and dmabuff and vaapi enabled things in about:config

(In reply to albertogomezmarin from comment #59)

I have tried EGL on my computer with amdgpu drivers. The only thing I noticed is that the close windows is transprent like in the nvidia driver bug. But I think is another thing in WR.

Apart from this, I have tried to use vaapi and it doesn't work. I have seen VCN block not being used when playing a video in youtube so..
The video was in VP9 format that my ryzen is capable to decode and I have tested in chromium so...
maybe someone can help me a little bit?

Firefox-nightly with EGL and dmabuff and vaapi enabled things in about:config

For VP9 videos you'll currently also need to disable media.ffvpx.enabled.

For further debugging question please also refer to https://mastransky.wordpress.com/2020/06/03/firefox-on-fedora-finally-gets-va-api-on-wayland/ and try running with MOZ_LOG="PlatformDecoderModule:5" (explained in the blog post).

(In reply to Robert Mader [:rmader] from comment #60)

(In reply to albertogomezmarin from comment #59)

I have tried EGL on my computer with amdgpu drivers. The only thing I noticed is that the close windows is transprent like in the nvidia driver bug. But I think is another thing in WR.

Apart from this, I have tried to use vaapi and it doesn't work. I have seen VCN block not being used when playing a video in youtube so..
The video was in VP9 format that my ryzen is capable to decode and I have tested in chromium so...
maybe someone can help me a little bit?

Firefox-nightly with EGL and dmabuff and vaapi enabled things in about:config

For VP9 videos you'll currently also need to disable media.ffvpx.enabled.

For further debugging question please also refer to https://mastransky.wordpress.com/2020/06/03/firefox-on-fedora-finally-gets-va-api-on-wayland/ and try running with MOZ_LOG="PlatformDecoderModule:5" (explained in the blog post).

done. I have been testing and firefox works with vaapi in a x11 .
I have been testing and this was my benchmarking:
cpu usage:
firefox stable with webrender: 26%
firefox vaapi: 14%
chromium vaapi: 6%
all was done in a fast way with VP9 and 1080p and youtube

But at least we have at the end vaapi

Thanks Alberto. To really measure the effect you'd need to compare the same FF version with and without VAAPI however, as a lot of work goes into optimizing WR atm.

For further optimizations AFAIK it's worth to keep an eye on frame building (bug 1596042 - bug 1650378 sound especially relevant here) and partial damage (bug 1625070 etc.)

(In reply to Robert Mader [:rmader] from comment #62)

Thanks Alberto. To really measure the effect you'd need to compare the same FF version with and without VAAPI however, as a lot of work goes into optimizing WR atm.

For further optimizations AFAIK it's worth to keep an eye on frame building (bug 1596042 - bug 1650378 sound especially relevant here) and partial damage (bug 1625070 etc.)

I am on exams right now so I cant do more tests.
But in the future I will. I hope this can improve the laptop experience in linux. And I expect firefox to be more power efficience than chromium that is the goal with this I think. I will do power tests in my laptop in the future so you can have the numbers

Depends on: 1650748
Depends on: 1652249
Depends on: 1652958
See Also: → egl-linux-dmabuf
Depends on: 1655794
Depends on: 1656436
Depends on: 1658035
Attached file about support info (txt) (obsolete) —
On a `Mac Mini 2009` with Nvidia onboard GPU NVIDIA Corporation C79 [GeForce 9400] (rev b1) with 256MB RAM and nouveau 1.0.16 drivers. I get accelerated (?) but corrupt video. Attaching the video screenshot. It looks similar to the wayland issue reported on the other `bug`.
 I can reproduce logs etc but I am not sure if they are needed.

On a Mac Mini 2009 with Nvidia onboard GPU NVIDIA Corporation C79 [GeForce 9400] (rev b1) with 256MB RAM and nouveau 1.0.16 drivers. I get accelerated (?) but corrupt video. Attaching the video screenshot. It looks similar to the wayland issue reported on the other bug.
I can reproduce logs etc but I am not sure if they are needed.

(In reply to ilgaz from comment #65)

On a Mac Mini 2009 with Nvidia onboard GPU NVIDIA Corporation C79 [GeForce 9400] (rev b1) with 256MB RAM and nouveau 1.0.16 drivers. I get accelerated (?) but corrupt video. Attaching the video screenshot. It looks similar to the wayland issue reported on the other bug.
I can reproduce logs etc but I am not sure if they are needed.

Please file a new bug for it.
Thanks.

Depends on: 1658772
Depends on: 1658900
Depends on: 1660192
Depends on: 1660195
Depends on: 1660336
Depends on: 1661292
Depends on: 1661441
Depends on: 1661669

I'm having trouble getting this working with proprietary nvidia-drivers on Linux with patched firefox 80 (vaapi fix backported from firefox 81). The log consistently complains about lack of webrender/dmabuf despite doing all the usual (webrender enabled, EGL enabled via environmental variable, relevant config options set).

I was going to find a bug, but I realized it might be not working because proprietary nvidia-drivers can't do dma-buf? There seems to be a lot of confusion about this around the net and I was hoping someone could clarify if it should be able to work so I can file a bug (or not).

(In reply to Gregory from comment #67)

I'm having trouble getting this working with proprietary nvidia-drivers on Linux with patched firefox 80 (vaapi fix backported from firefox 81). The log consistently complains about lack of webrender/dmabuf despite doing all the usual (webrender enabled, EGL enabled via environmental variable, relevant config options set).

I was going to find a bug, but I realized it might be not working because proprietary nvidia-drivers can't do dma-buf? There seems to be a lot of confusion about this around the net and I was hoping someone could clarify if it should be able to work so I can file a bug (or not).

Note this is on x11.

Proprietary Nvidia doesn't support GPL-licensed DMABUF Linux kernel API. There is no confusion. They would need to provide the required patches/firmware to the Mesa/Nouveau project. Please try to use your integrated graphics instead. bug 1658772 is about Mesa/Nouveau.

(In reply to Gregory from comment #67)

I was going to find a bug

There is already a bug about VDPAU/NVDEC support: bug 1210729.
VDPAU is not restricted to NVIDIA only, it's available for Gallium-based Mesa drivers too. But since that API is one of NVIDIA's native APIs, bug 1210729 is the most relevant tracking bug.

(In reply to Darkspirit, Servo QA from comment #69)

Proprietary Nvidia doesn't support GPL-licensed DMABUF Linux kernel API. There is no confusion.

Apparently there is for “normal” users. Could the logging be improved? Separating the message about DMABUF and WebRender for example, and explicitly mentioning the driver as a cause of errors in the logs?

[…]

Depends on: 1667537
Depends on: 1666099
Depends on: 1668735
Depends on: 1671673
Depends on: 1671652
Depends on: 1673184
Depends on: 1676115
Depends on: 1677124
Depends on: 1678486
Depends on: 1669189
Blocks: 1680205
Depends on: 1679454
Depends on: 1682471
Depends on: 1683808
Depends on: 1683556
Depends on: 1678806

What happened to the vaapi compatibility in latest nightly? It went from constantly crashing but working to not working at all even with all the flags set correctly? Any additional flags added to make VP8/9 vaapi work?

(In reply to Marko from comment #72)

What happened to the vaapi compatibility in latest nightly? It went from constantly crashing but working to not working at all even with all the flags set correctly? Any additional flags added to make VP8/9 vaapi work?

Can you file a new bug and attach your about:support there?
Thanks.

(In reply to Martin Stránský [:stransky] from comment #73)

(In reply to Marko from comment #72)

What happened to the vaapi compatibility in latest nightly? It went from constantly crashing but working to not working at all even with all the flags set correctly? Any additional flags added to make VP8/9 vaapi work?

Can you file a new bug and attach your about:support there?
Thanks.

https://bugzilla.mozilla.org/show_bug.cgi?id=1688627

Depends on: 1688627
Depends on: 1659054
Depends on: 1692988
Depends on: 1693073
Depends on: 1685463
Depends on: 1693083
Depends on: 1686681
Depends on: 1700515
Depends on: 1700439
Depends on: 1693065
Depends on: 1713276
No longer depends on: 1698248
Depends on: 1714839
No longer depends on: 1714839
Depends on: 1716689
Depends on: 1716638
Depends on: 1718309
Depends on: 1721617
Depends on: 1722221
Depends on: 1723238
Depends on: 1722474
Depends on: 1723540
Depends on: 1724385
Depends on: 1710815
Depends on: 1726186

This bug a blocked by Bug 1683808 (and similar). The correct way forward here is to implement GPU process on Wayland (by dmabuf EGL framebuffer) and then implement VAAPI in GPU process as we actually can't use VAAPI in RDD process.

Depends on: 1728476
No longer depends on: 1728476
Depends on: 1729902

Just as note: Maybe I missed something:
start firefox 92 (both VAAPI settings are enabled) in Linux Mint 20.2:

MOZ_X11_EGL=1 MOZ_LOG="PlatformDecoderModule:5" firefox 2>&1 | grep 'VA-API'
[RDD 93271: Main Thread]: D/PlatformDecoderModule VA-API FFmpeg is disabled by platform

(In reply to star from comment #76)

VA-API FFmpeg is disabled by platform

gfx.x11-egl.force-enabled=true (can be set instead of using the MOZ_X11_EGL=1 env var. As of yesterday (bug 1695933), Mesa 21 users don't need to set this anymore.)
gfx.webrender.all=true (force-enabling by setting this pref is not needed for most people as they already use it by default)
media.ffmpeg.vaapi.enabled=true
media.rdd-ffvpx.enabled=false (until bug 1683808 has been done)
media.rdd-vpx.enabled=false (until bug 1683808 has been done)
https://addons.mozilla.org/firefox/addon/enhanced-h264ify/
https://wiki.archlinux.org/title/Firefox#Hardware_video_acceleration

I used the setup and with my startup (MOZ_X11_EGL=1 MOZ_LOG="PlatformDecoderModule:5" firefox 2>&1 | grep 'VA-API')
I could observe it seems to use VAAPI.
At least I get a lot of
[Child 98006: MediaPDecoder #6]: D/PlatformDecoderModule Reusing VA-API DMABufSurface UID = 21
[Child 98006: MediaPDecoder #7]: D/PlatformDecoderModule VA-API Got one frame output with pts=76533333dts=76533334 duration=33333 opaque=-9223372036854775808

BUT: the CPU usage seem not to be lower than before. Is it because of the log? Anyhow, I changed to your
gfx.x11-egl.force-enabled=true and restarted, but CPU usage keeps nearly same level. So its not caused by logs prints.
Don't know how can I really measure that....

(In reply to Martin Stránský [:stransky] (ni? me) from comment #75)

This bug a blocked by Bug 1683808 (and similar). The correct way forward here is to implement GPU process on Wayland (by dmabuf EGL framebuffer) and then implement VAAPI in GPU process as we actually can't use VAAPI in RDD process.

For the record, there's also work on a so-called UtilityProcess (bug 1731890). From a Matrix chat:

gerard-majax: I'm not sure exactly about VA-API specifically, but the plan is to have such a utilityprocess that can be used by different actors, and each with a specific sandboxing
and from that discussion, I think the first customer might be moving RDD audio decoders to this new

So in case it proves hard to make the GPU work (especially on Wayland), this might be a good way to go.

See Also: → 1731890
Depends on: 1737116
Depends on: 1738545
Depends on: 1739499
Depends on: 1739546
Depends on: 1742229
Depends on: 1698778
Depends on: 1742416

With latest nightly (including 1698778 fixes) VA-API appears to work on AMD with just needing to switch media.ffmpeg.vaapi.enabled to 1, including VP9 and ffvpx (?).

(In reply to Julian Sikorski from comment #80)

With latest nightly (including 1698778 fixes) VA-API appears to work on AMD with just needing to switch media.ffmpeg.vaapi.enabled to 1, including VP9 and ffvpx (?).

Yes, it's supposed so due to fixed Bug 1698778.

(In reply to Martin Stránský [:stransky] (ni? me) from comment #81)

(In reply to Julian Sikorski from comment #80)

With latest nightly (including 1698778 fixes) VA-API appears to work on AMD with just needing to switch media.ffmpeg.vaapi.enabled to 1, including VP9 and ffvpx (?).

Yes, it's supposed so due to fixed Bug 1698778.

Does this mean that Firefox flatpak will also work with VAAPI from this version on? :)

Depends on: 1743159
Depends on: 1743577
Summary: [Linux/EGL] Implement ffmpeg/VAAPI video playback → [meta][Linux/EGL] Implement ffmpeg/VAAPI video playback
Attachment #9130334 - Attachment is obsolete: true
Attachment #9169571 - Attachment is obsolete: true
Depends on: 1743638
Depends on: 1743647
Depends on: 1743750
No longer depends on: 1742993
Depends on: 1744037
Depends on: 1743980
Depends on: 1743880
Depends on: 1743772
No longer depends on: 1743980
Depends on: 1744254
Depends on: 1679600
Depends on: 1745225
Depends on: 1746232
Depends on: 1700815
Depends on: 1748038

For the record, there's a new project trying to make nvidia drivers work with this: https://github.com/elFarto/nvidia-vaapi-driver

See also bug 1748460

Depends on: 1748862
Depends on: 1749408
Depends on: 1749623
Depends on: 1749609
No longer depends on: 1749609
No longer depends on: 1713276
Depends on: 1750389
Depends on: 1750663
Depends on: 1751363
Depends on: 1751710
No longer depends on: 1751363
Depends on: 1751859
Depends on: 1751987
Depends on: 1751991
Depends on: 1752097
Depends on: 1752106

To test VA-API by mozilla testsuite you can run it with:

MOZ_DISABLE_RDD_SANDBOX=1 ./mach mochitest dom/media/test --setpref media.ffmpeg.vaapi.enabled=true
MOZ_DISABLE_RDD_SANDBOX=1 ./mach reftest dom/media/test --setpref media.ffmpeg.vaapi.enabled=true

MOZ_DISABLE_RDD_SANDBOX=1 is needed for Bug 1751363 and Bug 1751709.

Looks like there are some decode failures during testing, will look at it.

Testing results of mochitest on Mesa Intel(R) UHD Graphics 630 (CFL GT2)

mochitest-browser
~~~~~~~~~~~~~~~~~
Ran 55 checks (52 subtests, 3 tests)
Expected results: 55
Unexpected results: 0
OK

mochitest-chrome
~~~~~~~~~~~~~~~~
Ran 47 checks (46 subtests, 1 tests)
Expected results: 46
Unexpected results: 0

Unexpected Results
------------------
ERROR TEST-UNEXPECTED-FAIL | dom/media/test/chrome/test_accumulated_play_time.html | application terminated with exit code -11

mochitest-plain
~~~~~~~~~~~~~~~
Ran 246 checks (10 asserts, 218 subtests, 18 tests)
Expected results: 236
Skipped: 7 tests
Unexpected results: 2
  subtest: 2 (2 fail)

Unexpected Results
------------------
dom/media/test/test_background_video_drawimage_with_suspended_video.html
  FAIL Pixels aren't all the same color
    SimpleTest.ok@SimpleTest/SimpleTest.js:417:16
    drawVideoToCanvas@dom/media/test/test_background_video_drawimage_with_suspended_video.html:53:5
    runTest/<@dom/media/test/test_background_video_drawimage_with_suspended_video.html:71:26
  FAIL Pixels aren't all the same color
    SimpleTest.ok@SimpleTest/SimpleTest.js:417:16
    drawVideoToCanvas@dom/media/test/test_background_video_drawimage_with_suspended_video.html:53:5
    runTest/<@dom/media/test/test_background_video_drawimage_with_suspended_video.html:71:26

Reftest also needs some fixes:

REFTEST INFO | Successful: 28 (28 pass, 0 load only)
REFTEST INFO | Unexpected: 12 (11 unexpected fail, 0 unexpected pass, 0 unexpected asserts, 1 failed load, 0 exception)
REFTEST INFO | Known problems: 3 (0 known fail, 0 known asserts, 0 random, 3 skipped, 0 slow)
Depends on: 1752282
Depends on: 1752493
Depends on: 1752497
Depends on: 1752605
Depends on: 1752666
Depends on: 1754074

VA-API does not work since some Nightly releases even when disabling the sandbox – currently I am using 99.0a1 (20220213092920).

I am using

MOZ_LOG="PlatformDecoderModule:1" MOZ_DISABLE_RDD_SANDBOX=1 nightly

and get

[Child 201138, MediaDecoderStateMachine #1] WARNING: Decoder=7f5ecce31c00 Decode error: NS_ERROR_DOM_MEDIA_DECODE_ERR (0x806e0004) - mozilla::MediaResult mozilla::FFmpegVideoDecoder<58>::CreateImageVAAPI(int64_t, int64_t, int64_t, MediaDataDecoder::DecodedData &): VAAPI dmabuf allocation error: file /builds/worker/checkouts/gecko/dom/media/MediaDecoderStateMachine.cpp:3579

Is there a way to get it working by setting other options?

Depends on: 1755180

(In reply to Paul Menzel from comment #89)

Is there a way to get it working by setting other options?

Please file a new bug for it.

(In reply to Martin Stránský [:stransky] (ni? me) from comment #90)

(In reply to Paul Menzel from comment #89)

Is there a way to get it working by setting other options?

Please file a new bug for it.

It’s fixed today (20220214213941). The Debian (non-free) package intel-media-va-driver-non-free was also updated to 22.1.1+ds1-1 on February 7th, 2022, and then to 22.2.0+ds1-1 on February 13th, 2022 (on my system February 14th, 2022). No idea if that’s related.

Are any settings needed besides media.ffmpeg.vaapi.enabled=true in Firefox 98.0b7?

I noticed that VAAP isn't working now neither for H.264 nor for VP9. (X11/EGL).

To clarify - I'm not using any flatpak.

vainfo: Driver version: Mesa Gallium driver 21.3.5 for AMD Radeon RX 6800 XT (SIENNA_CICHLID, DRM 3.44.0, 5.16.10, LLVM 13.0.1)

You need to start Firefox with MOZ_DISABLE_RDD_SANDBOX=1 environment variable until bug 1751363 is fixed.

Depends on: 1756753
Depends on: 1757791
Depends on: 1756598
Depends on: 1757998
Depends on: 1758463
Depends on: 1758431
Depends on: 1758473
No longer depends on: 1758431
Depends on: 1751363
Depends on: 1758948
Depends on: 1759137
Depends on: 1759464
Depends on: 1759784
Depends on: 1759842
Depends on: 1759947
Depends on: 1760414
Depends on: 1760941
Depends on: 1761217
Depends on: 1761471
Depends on: 1762378
Depends on: 1760905
Depends on: 1762544
Depends on: 1759596
No longer depends on: 1749623
Depends on: 1762725
Depends on: 1763298
Depends on: 1763673
Depends on: 1763720
Depends on: 1765530
Depends on: 1765824
No longer depends on: 1765824
No longer depends on: 1754074
No longer depends on: 1758473
No longer depends on: 1760941
No longer depends on: 1757791
No longer depends on: 1759596
Regressed by: 1766124
Depends on: 1766124
No longer regressed by: 1766124
Depends on: 1766517
No longer depends on: 1766517
Depends on: 1766693
Depends on: 1767431
Depends on: 1768385
Depends on: 1768838
Depends on: 1769499
Depends on: 1768830
Depends on: 1770407
Depends on: 1770840
Depends on: 1765350
No longer depends on: 1759947
Depends on: 1771861
Depends on: 1771898
Depends on: 1772086
No longer depends on: 1771898
Depends on: 1772142
Depends on: 1772978

Please add bug #1772028 as a dependency too.

Depends on: 1772028
Depends on: 1773282
Depends on: 1773377
Depends on: 1772838
No longer depends on: 1772838
Depends on: 1776362
Depends on: 1776126
Depends on: 1777200
Depends on: 1777910
Depends on: 1779186
Depends on: 1777811
Depends on: 1780962
Depends on: 1780731
Depends on: egl-linux-v4l
Depends on: 1783987
Depends on: 1783584
Depends on: 1771555
No longer depends on: egl-linux-v4l
See Also: → egl-linux-v4l
Depends on: 1770319
Depends on: 1787102
Depends on: 1788519
Depends on: 1790729
Depends on: 1793507
Severity: normal → S3

Opened bug 1799350 for AV1 VA-API failure, since bug 1745225 is closed.

Depends on: 1799747
No longer depends on: 1787102
Depends on: 1688947
Depends on: 1800459
Depends on: 1801054
Depends on: 1801576
Depends on: 1802844
Depends on: 1805102
Depends on: 1805975
Depends on: 1805621
Depends on: 1806769
Depends on: 1808487
Depends on: 1807762
Depends on: 1809162
Depends on: 1809167
Depends on: 1809026
Depends on: 1811210
No longer depends on: 1802844
No longer depends on: 1779186
No longer depends on: 1805975
No longer depends on: 1790729
No longer depends on: 1780962
No longer depends on: 1659054
No longer depends on: 1772028
No longer depends on: 1799747
Depends on: 1811878
No longer depends on: 1806769
Depends on: 1813500
Depends on: 1817812
Depends on: 1820827
Depends on: 1820416
Depends on: 1812982
No longer depends on: 1812982
Depends on: 1823282
Depends on: 1823333
Depends on: 1824307
Depends on: 1824458
Depends on: 1824943
Depends on: 1825260
Depends on: 1818815
Depends on: 1829462
Depends on: 1829541
Depends on: 1829542
Depends on: 1829840
Depends on: 1831038
Depends on: 1832080
Depends on: 1832133
No longer depends on: 1832133
Depends on: 1805975
Depends on: 1790729
Depends on: 1659054
Duplicate of this bug: egl-linux-vaapi-glitches
Depends on: 1833207
Depends on: 1833497
Depends on: 1834339
Depends on: 1835797
Depends on: 1836021
Depends on: 1837138
Depends on: 1837553
Depends on: 1837627
Depends on: 1839615
Depends on: 1839809
No longer depends on: 1837553
Depends on: 1841575
Depends on: 1841252
Depends on: 1843425
Depends on: 1843679
Depends on: 1844118
No longer depends on: 1839809
No longer depends on: 1837627
Depends on: 1846701
Depends on: 1846932
Depends on: 1853024
Depends on: 1853259
Depends on: 1854097
Depends on: 1850852
Depends on: 1844859
Depends on: 1830627
Depends on: 1854342

Here's another one to add to the texting...

https://www.prolux-solutions.com/de/de/

Or a different message when the video is opened outside the page...

https://www.prolux-solutions.com/media/content/videos/home/STORAC_Animation_VRFB_mit_Sprecher_mit_Vorschaubild_DE.mp4#t=0.1

"The video can't be played because the file is corrupt."

I can download the video using one of the video download extensions.

Far as I can see, everything Firefox fails to play can be downloaded and played on VLC. And that applies to this video above.

Do you want me to try Chome? I've avoided installing spyware on the Linux box but I will do so if it provides more info on this.

What bothers me is that I see a huge issue with this and yet I don't see a lot of talk about it here or on any of the Linux OS channels. I'm using Rocky which is down from Redhat.

It doesn't "feel like" an decoder issue, rather like the file is not correctly identified because some quirky format isn't correctly recognised.

@Yani, please create a new issue and attach all the details, like about:support. The video plays fine on a Dell Latitude E7250 (Intel IGD) with Debian sid/unstable with Nightly 120.0a1 (20231008210545).

Depends on: 1858077
Depends on: 1858904
Depends on: 1864306
Depends on: 1864774
Depends on: 1864784
Depends on: 1870180
Depends on: 1862516
Depends on: 1870551

I noticed that without media.ffmpeg.vaapi.enabled set to true (and by default it's false), Firefox now fails to enable hardware decoding on AMD / Mesa. Is that expected? I'm using KDE Plasma Wayland session.

(In reply to Shmerl from comment #100)

I noticed that without media.ffmpeg.vaapi.enabled set to true (and by default it's false), Firefox now fails to enable hardware decoding on AMD / Mesa. Is that expected? I'm using KDE Plasma Wayland session.

Yes, AMD is disabled from default VA-API due to various radeon driver issues. See Bug 1837140.

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: