5 5
Myy

Mainline VPU

Recommended Posts

4 hours ago, Kwiboo said:

with ffmpeg command the cpu will be busy doing pixel format conversion (sw pixel conversion is also activated with null output).

This can be avoided by using

-hwaccel_output_format drm_prime

argument. Works good with null output.

Share this post


Link to post
Share on other sites
(edited)

confirmed x264 10bit hwaccel is working. thanks so much, @Kwiboo

 

using kodi-git, ffmpeg 4.2.3 + new rkvdec branch work, linux 5.7-rc7 + librelec rockchip patches + linuxtv WIP branch diff + drm-next branch diff

 

questions:

 

will the ffmpeg patches be upstreamed? or moved into a separate library? wondering how this will evolve.

 

(and echoing my previous questions wondering about vaapi support for hantro+rkvdec and ffmpeg -decoders list issue above.)

Edited by xmixahlx

Share this post


Link to post
Share on other sites
2 hours ago, xmixahlx said:

will the ffmpeg patches be upstreamed? or moved into a separate library? wondering how this will evolve.

That is the plan. However that can't happen until codecs interfaces in kernel are promoted to stable. That will probably take at least a few more kernel cycles. Nobody wants to screw it up user facing API so everyone is cautious. Actually, not all features are yet supported in kernel, that's also the reason you need kernel patches with experimental improvements.

 

2 hours ago, xmixahlx said:

wondering about vaapi support

Work on initial vaapi library stalled, but others might take over... Nothing useful at the moment unless you understand both APIs well and you're able to fix it.

Share this post


Link to post
Share on other sites

thanks @jernej ! really appreciate all of your work with @Kwiboo to provide these capabilities. awesome.

 

unfortunately, i'm not a kernel developer, but would love to help out. testing? documentation? LMK.

 

vaapi support for pinebookpro to allow hwaccel on firefox/wayland would be epic. in addition to the obvious youtube/video functions it seems like that would make jitsi meet usable, too.

Share this post


Link to post
Share on other sites

FYI, VAAPI lib is just userspace translation layer between two apis. No kernel hacking needed, apart from applying LE patches.

Share this post


Link to post
Share on other sites

thanks. i have collected the vaapi hantro bits on the pine64 forum thread, which basically refers to ph5's work on libva, libva-v4l2-request, and gstreamer-vaapi to add support for hantro, but it seems broken.

 

vaapi for mpeg2 on hantro does work -- tested on vlc.

Share this post


Link to post
Share on other sites
2 hours ago, xmixahlx said:

but it seems broken.

You have to make sure that libva-v4l2-request uses  same *-ctrl.h files that are in kernel sources under include/media/. For starters, just copy them over from kernel to lib source folder. Then you might need to adjust lib sources to new field names. Unless these files are completely the same, you won't have any success.

Share this post


Link to post
Share on other sites

thanks for the guidance, @jernej!

 

with both hantro and rkvdec supporting h264, and rkvdec having expanded capabilities, does it make sense to rely/prioritize rkvdec when detected? how does ffmpeg/v4l2-request decide which one to use?

Share this post


Link to post
Share on other sites

as i posted in the pine64 hwaccel thread:

the kernel i am testing now, working well is: (getting ready for 5.7!)

next20200529
pinebookpro patch: https://pastebin.com/YK4LXeyv
Kwiboo's LibreElec 1000 patch
Kwiboo's v4l2 patch: https://pastebin.com/gaQSZQ6B
Kwiboo's drm patch: https://pastebin.com/b1BWZ8Cg

responsiveness is actually significantly better than 5.7-rc7 (!?), and my nvme drive is also much better handled. interesting.

also, i would recommend compiling hantro and rkvdec into the kernel, not modules, so that they are detected in the same order consistently.

Share this post


Link to post
Share on other sites

also:

 

ffmpeg 4.2.3 with v4l2-request-hwaccel-4.2.2-rkvdec patch: https://pastebin.com/uF5kpABD
kodi-git
libva-v4l2-request_master + pulls 29+30
libva_master + pulls 332+340
(for libva 332 delete the 5th chunk, version bump, or apply against libva 2.6.0)
 

Share this post


Link to post
Share on other sites

@Kwiboo fyi i did encounter graphical artifacts similar to what rubenvb has noted with 10bit h264 and h265 using ffmpeg_master, ffmpeg_v4l2-request-hwaccel-master, and cherry picking last few rkvdec commits from v4l2-request-hwaccel-4.2.2-rkvdec.

 

however, the system stack i mentioned above earlier today works perfectly.

 

 

Share this post


Link to post
Share on other sites
2 hours ago, xmixahlx said:

fyi i did encounter graphical artifacts similar to what rubenvb has noted with 10bit h264 and h265 using ffmpeg_master, ffmpeg_v4l2-request-hwaccel-master, and cherry picking last few rkvdec commits from v4l2-request-hwaccel-4.2.2-rkvdec.

 

Thanks for testing, normally I try to keep ffmpeg_v4l2-request-hwaccel-master and ffmpeg_v4l2-request-hwaccel-4.2.2 in sync but I have not yet updated or tested the -master branch, I will try to update it in next few days.

 

 

Background on my different ffmpeg branches:

v4l2-request-hwaccel-x.y.z: Main branch that most of the time matches the patches used in LibreELEC master, x.y.z (4.2.2) matches the ffmpeg version used in Kodi master. This branch is usually tested on Rockchip/hantro/rkvdec by me and on Allwinner/cedrus by @jernej, fixup commits usually gets squashed right before we pick out a new patchset for use in LibreELEC.

v4l2-request-hwaccel-master: This branch should contain the patches in v4l2-request-hwaccel-x.y.z on top of ffmpeg master, may see some delays at times.

v4l2-request-hwaccel-4.2.2-rkvdec: work-in-progress branch for parts that currently only is expected to work on rockchip/rkvdec, e.g. 10-bit hevc decoding on cedrus stops working in this branch due to patches to handle rkvdec 10-bit decoding, patches should eventually flow to -x.y.z/-master branches once ready.

 

Share this post


Link to post
Share on other sites

understood. tyvm for context.

 

5.7 is released! thanks again for your work and others getting patches upstreamed.

 

i noticed vc1 and mpeg4 hwaccel work in @jernej
repos. is that coming to rk3399?

 

also bootlin has been active with v4l2-request-test with new hantro and other branches. and the bext branch was deleted in libva-v4l2-request. hopeful that these will be updated for v4l2 in 5.7!

 

with ffmpeg not displaying v4l2request hwaccels any more, it seems that mpv hwdec doesn't work. should i be doing something different?

Share this post


Link to post
Share on other sites
24 minutes ago, xmixahlx said:

i noticed vc1 and mpeg4 hwaccel work in @jernej
repos.

That is very much work in progress, so it doesn't work anywhere yet, except basic MPEG4 on Cedrus. I'm not sure if I'll ever finish MPEG4 because it has a ton of variants and even Cortex A7 is capable decoding it. On the other hand, VC-1 has only 3 profiles and it's much more demanding for CPU decoding.

Share this post


Link to post
Share on other sites

understood. it does seem mpeg4 is a mess,.

 

hwaccel is a much bigger topic on the pinebookpro due to battery, although i'm sure power usage is still a concern with the usual sbc format.

 

obviously with libreelec and kodi you are focused on hwaccel decoding as many formats as possible. (thank you!) do you have any insight into hantro encoding status? i see hantro-vpu-enc detected in the kernel with jpeg support and just not aware of what is happening on that front.

Share this post


Link to post
Share on other sites
25 minutes ago, xmixahlx said:

do you have any insight into hantro encoding status?

I don't. As you said, I'm solely focused on decoding. Maybe @Kwiboo knows?

Share this post


Link to post
Share on other sites
(edited)

@Kwiboo did you have any idea about hantro encoding? ^

 

also, ffmpeg 4.3 and linux 5.8-rc1 are released and can't wait to update!

 

fyi, i have a new project, pbp-tools on git that is collecting these hwaccel things for pinebook pro:

https://github.com/xmixahlx/pbp-tools

https://forum.pine64.org/showthread.php?tid=10190

 

any idea on the ffmpeg -decoders bug not listing v4lrequest? that seems to break things like mpv that use detection. ffmpeg and kodi work OK.

 

Edited by xmixahlx

Share this post


Link to post
Share on other sites

@Kwiboo ffmpeg 4.3-rkvdec working well with most supported codecs. i do have some new pixellations with vp9. just checked, they were there in the 4.2.2-rkvdec, too.

 

in kodi, vp9 10/12bit and x265 12bit formats that are not hwaccel supported just freeze or crash kodi and are not being decoded via SW.

 

Share this post


Link to post
Share on other sites
23 hours ago, xmixahlx said:

ffmpeg 4.3-rkvdec working well with most supported codecs

 

Thanks for testing!, kodi has just moved to ffmpeg 4.3 so LibreELEC will follow as soon as all ffmpeg patches has been rebased.

 

23 hours ago, xmixahlx said:

i do have some new pixellations with vp9

 

Do you have a sample to share? Also note that only Profile 0 is supported and videos using frame resize is expected to have issues.

 

23 hours ago, xmixahlx said:

in kodi, vp9 10/12bit and x265 12bit formats that are not hwaccel supported just freeze or crash kodi and are not being decoded via SW.

 

I do not expect these formats to work, the kodi drmprime video codec and renderer sw decoding parts have mainly been tested with 8-bit video.

The crash should probably be fixed, unfortunately 10/12-bit sw decoding is not high on priority list, in future when ffmpeg filters are supported in kodi drmprime video codec converting to a 8-bit pixel format may be possible.

Share this post


Link to post
Share on other sites

@Kwiboo i've also tested your latest linux-rockchip WIP branch work patched to linux 5.8-rc2 with ffmpeg 4.3-rkvdec. working great on pinebookpro, minus the vp9 issue.

 

for the vp9 issue, this was either broken after rebasing 4.2.2, or with a newer kernel. i do recall it working fine earlier.


i've uploaded my clips here: (they are all the same short sample re-encoded with ffmpeg from big buck bunny with various codecs and settings for testing. )

https://github.com/xmixahlx/media-samples/tree/master/videos

 

 

Share this post


Link to post
Share on other sites

I'm not sure I understood everything in this thread (in fact, I'm sure I got less than 10%).
Anyway... My goal is to have hardware encoding and decoding for the Orange Pi 4B and RK3399 running Armbian.
I'm still running 5.4.49-rockchip64 kernel.
Is there an article somewhere for newbies in order to get a working hardware-enhanced version of ffmpeg for these boards?

Thanks.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
5 5