JMCC Posted July 6, 2021 Posted July 6, 2021 1 hour ago, rubenvb said: Does Armbian keep track of any of these media patches somewhere? Here. And read the posts above about Kodi 2 Quote
jernej Posted July 6, 2021 Posted July 6, 2021 3 hours ago, rubenvb said: only the "outdated" (mismatching with kernels >=5.11) in the LibreElec repos. Yeah, LE currently uses 5.10 kernel because we are preparing stable release (no ETA, but hopefully soon). Only after that Linux will be updated to whatever current stable release will be at the time. However, v4l2 request api ffmpeg patches for Linux 5.13 can be found here. Note that they are untested and you still need corresponding kernel patches too. 2 Quote
rubenvb Posted August 5, 2021 Posted August 5, 2021 On 7/6/2021 at 5:04 PM, jernej said: However, v4l2 request api ffmpeg patches for Linux 5.13 can be found here. Note that they are untested and you still need corresponding kernel patches too. I just tested this with vanilla Linux 5.13.7 and at least H264 playback with DRM Prime Direct To Plane rendering in Kodi 19.1 works as expected. HEVC content still just doesn't render at all, but maybe I need some extra kernel patches for that still. Thanks for the link to the up to date FFmpeg patches, you're a lifesaver! 0 Quote
jernej Posted August 6, 2021 Posted August 6, 2021 20 hours ago, rubenvb said: HEVC content still just doesn't render at all, but maybe I need some extra kernel patches for that still. Pick patches 14-23 here: https://github.com/jernejsk/LibreELEC.tv/tree/linux-5.13/projects/Allwinner/patches/linux With that, HEVC should work without problem. Official HEVC API still needs some work. 20 hours ago, rubenvb said: Thanks for the link to the up to date FFmpeg patches, you're a lifesaver! Not a problem, I usually update them with every stable kernel release, just to check on progress. 1 Quote
kprasadvnsi Posted September 7, 2021 Posted September 7, 2021 On 8/6/2021 at 2:18 AM, rubenvb said: I just tested this with vanilla Linux 5.13.7 and at least H264 playback with DRM Prime Direct To Plane rendering in Kodi 19.1 works as expected. HEVC content still just doesn't render at all, but maybe I need some extra kernel patches for that still. Thanks for the link to the up to date FFmpeg patches, you're a lifesaver! I am trying to compile FFmpeg on Armbian with 5.13.12 kernel. I am getting compilation errors. maybe I did configuration wrong. what configure parameters you use? 0 Quote
Werner Posted September 7, 2021 Posted September 7, 2021 3 hours ago, kprasadvnsi said: g compilation errors What errors? Nobody here is a clairvoyant. 0 Quote
kprasadvnsi Posted September 7, 2021 Posted September 7, 2021 I am building FFmpeg from here on a Orange Pi 4 with Armbian Hirsute edge 5.13.12 kernel. I have configured FFmpeg with following parameters. ./configure --enable-libdrm --enable-v4l2-request --enable-libudev --enable-static --disable-shared I am getting these errors on compile. AR libavdevice/libavdevice.a AR libavfilter/libavfilter.a AR libavformat/libavformat.a CC libavcodec/v4l2_request_mpeg2.o CC libavcodec/vaapi_encode.o CC libavcodec/vaapi_encode_h264.o CC libavcodec/vaapi_encode_h265.o libavcodec/v4l2_request_mpeg2.c:24:37: error: field ‘sequence’ has incomplete type 24 | struct v4l2_ctrl_mpeg2_sequence sequence; | ^~~~~~~~ libavcodec/v4l2_request_mpeg2.c:25:36: error: field ‘picture’ has incomplete type 25 | struct v4l2_ctrl_mpeg2_picture picture; | ^~~~~~~ libavcodec/v4l2_request_mpeg2.c:26:41: error: field ‘quantisation’ has incomplete type 26 | struct v4l2_ctrl_mpeg2_quantisation quantisation; | ^~~~~~~~~~~~ libavcodec/v4l2_request_mpeg2.c: In function ‘v4l2_request_mpeg2_start_frame’: libavcodec/v4l2_request_mpeg2.c:39:10: error: ‘struct v4l2_ctrl_mpeg2_sequence’ has no member named ‘horizontal_size’ 39 | .horizontal_size = s->width, | ^~~~~~~~~~~~~~~ libavcodec/v4l2_request_mpeg2.c:39:28: warning: excess elements in struct initializer 39 | .horizontal_size = s->width, | ^ libavcodec/v4l2_request_mpeg2.c:39:28: note: (near initialization for ‘(anonymous)’) libavcodec/v4l2_request_mpeg2.c:40:10: error: ‘struct v4l2_ctrl_mpeg2_sequence’ has no member named ‘vertical_size’ 40 | .vertical_size = s->height, | ^~~~~~~~~~~~~ libavcodec/v4l2_request_mpeg2.c:40:26: warning: excess elements in struct initializer 40 | .vertical_size = s->height, | ^ libavcodec/v4l2_request_mpeg2.c:40:26: note: (near initialization for ‘(anonymous)’) libavcodec/v4l2_request_mpeg2.c:41:10: error: ‘struct v4l2_ctrl_mpeg2_sequence’ has no member named ‘vbv_buffer_size’ 41 | .vbv_buffer_size = req->output.size, | ^~~~~~~~~~~~~~~ libavcodec/v4l2_request_mpeg2.c:41:28: warning: excess elements in struct initializer 41 | .vbv_buffer_size = req->output.size, | ^~~ libavcodec/v4l2_request_mpeg2.c:41:28: note: (near initialization for ‘(anonymous)’) libavcodec/v4l2_request_mpeg2.c:44:10: error: ‘struct v4l2_ctrl_mpeg2_sequence’ has no member named ‘profile_and_level_indication’ 44 | .profile_and_level_indication = 0, | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~ libavcodec/v4l2_request_mpeg2.c:44:41: warning: excess elements in struct initializer 44 | .profile_and_level_indication = 0, | ^ libavcodec/v4l2_request_mpeg2.c:44:41: note: (near initialization for ‘(anonymous)’) libavcodec/v4l2_request_mpeg2.c:45:10: error: ‘struct v4l2_ctrl_mpeg2_sequence’ has no member named ‘chroma_format’ 45 | .chroma_format = s->chroma_format, | ^~~~~~~~~~~~~ libavcodec/v4l2_request_mpeg2.c:45:26: warning: excess elements in struct initializer 45 | .chroma_format = s->chroma_format, | ^ libavcodec/v4l2_request_mpeg2.c:45:26: note: (near initialization for ‘(anonymous)’) libavcodec/v4l2_request_mpeg2.c:37:60: error: invalid use of undefined type ‘struct v4l2_ctrl_mpeg2_sequence’ 37 | controls->sequence = (struct v4l2_ctrl_mpeg2_sequence) { | ^ libavcodec/v4l2_request_mpeg2.c:49:37: error: ‘V4L2_MPEG2_SEQ_FLAG_PROGRESSIVE’ undeclared (first use in this function); did you mean ‘V4L2_DV_BT_CAP_PROGRESSIVE’? 49 | controls->sequence.flags |= V4L2_MPEG2_SEQ_FLAG_PROGRESSIVE; | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | V4L2_DV_BT_CAP_PROGRESSIVE libavcodec/v4l2_request_mpeg2.c:49:37: note: each undeclared identifier is reported only once for each function it appears in libavcodec/v4l2_request_mpeg2.c:53:10: error: ‘struct v4l2_ctrl_mpeg2_picture’ has no member named ‘picture_coding_type’ 53 | .picture_coding_type = s->pict_type, | ^~~~~~~~~~~~~~~~~~~ libavcodec/v4l2_request_mpeg2.c:53:32: warning: excess elements in struct initializer 53 | .picture_coding_type = s->pict_type, | ^ libavcodec/v4l2_request_mpeg2.c:53:32: note: (near initialization for ‘(anonymous)’) libavcodec/v4l2_request_mpeg2.c:56:10: error: ‘struct v4l2_ctrl_mpeg2_picture’ has no member named ‘f_code’ 56 | .f_code[0][0] = s->mpeg_f_code[0][0], | ^~~~~~ libavcodec/v4l2_request_mpeg2.c:56:25: warning: excess elements in struct initializer 56 | .f_code[0][0] = s->mpeg_f_code[0][0], | ^ libavcodec/v4l2_request_mpeg2.c:56:25: note: (near initialization for ‘(anonymous)’) libavcodec/v4l2_request_mpeg2.c:57:10: error: ‘struct v4l2_ctrl_mpeg2_picture’ has no member named ‘f_code’ 57 | .f_code[0][1] = s->mpeg_f_code[0][1], | ^~~~~~ libavcodec/v4l2_request_mpeg2.c:57:25: warning: excess elements in struct initializer 57 | .f_code[0][1] = s->mpeg_f_code[0][1], | ^ libavcodec/v4l2_request_mpeg2.c:57:25: note: (near initialization for ‘(anonymous)’) libavcodec/v4l2_request_mpeg2.c:58:10: error: ‘struct v4l2_ctrl_mpeg2_picture’ has no member named ‘f_code’ 58 | .f_code[1][0] = s->mpeg_f_code[1][0], | ^~~~~~ libavcodec/v4l2_request_mpeg2.c:58:25: warning: excess elements in struct initializer 58 | .f_code[1][0] = s->mpeg_f_code[1][0], | ^ libavcodec/v4l2_request_mpeg2.c:58:25: note: (near initialization for ‘(anonymous)’) libavcodec/v4l2_request_mpeg2.c:59:10: error: ‘struct v4l2_ctrl_mpeg2_picture’ has no member named ‘f_code’ 59 | .f_code[1][1] = s->mpeg_f_code[1][1], | ^~~~~~ libavcodec/v4l2_request_mpeg2.c:59:25: warning: excess elements in struct initializer 59 | .f_code[1][1] = s->mpeg_f_code[1][1], | ^ libavcodec/v4l2_request_mpeg2.c:59:25: note: (near initialization for ‘(anonymous)’) libavcodec/v4l2_request_mpeg2.c:60:10: error: ‘struct v4l2_ctrl_mpeg2_picture’ has no member named ‘picture_structure’ 60 | .picture_structure = s->picture_structure, | ^~~~~~~~~~~~~~~~~ libavcodec/v4l2_request_mpeg2.c:60:30: warning: excess elements in struct initializer 60 | .picture_structure = s->picture_structure, | ^ libavcodec/v4l2_request_mpeg2.c:60:30: note: (near initialization for ‘(anonymous)’) libavcodec/v4l2_request_mpeg2.c:61:10: error: ‘struct v4l2_ctrl_mpeg2_picture’ has no member named ‘intra_dc_precision’ 61 | .intra_dc_precision = s->intra_dc_precision, | ^~~~~~~~~~~~~~~~~~ libavcodec/v4l2_request_mpeg2.c:61:31: warning: excess elements in struct initializer 61 | .intra_dc_precision = s->intra_dc_precision, | ^ libavcodec/v4l2_request_mpeg2.c:61:31: note: (near initialization for ‘(anonymous)’) libavcodec/v4l2_request_mpeg2.c:51:58: error: invalid use of undefined type ‘struct v4l2_ctrl_mpeg2_picture’ 51 | controls->picture = (struct v4l2_ctrl_mpeg2_picture) { | ^ libavcodec/v4l2_request_mpeg2.c:65:36: error: ‘V4L2_MPEG2_PIC_FLAG_TOP_FIELD_FIRST’ undeclared (first use in this function) 65 | controls->picture.flags |= V4L2_MPEG2_PIC_FLAG_TOP_FIELD_FIRST; | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ libavcodec/v4l2_request_mpeg2.c:68:36: error: ‘V4L2_MPEG2_PIC_FLAG_FRAME_PRED_DCT’ undeclared (first use in this function) 68 | controls->picture.flags |= V4L2_MPEG2_PIC_FLAG_FRAME_PRED_DCT; | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ libavcodec/v4l2_request_mpeg2.c:71:36: error: ‘V4L2_MPEG2_PIC_FLAG_CONCEALMENT_MV’ undeclared (first use in this function) 71 | controls->picture.flags |= V4L2_MPEG2_PIC_FLAG_CONCEALMENT_MV; | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ libavcodec/v4l2_request_mpeg2.c:74:36: error: ‘V4L2_MPEG2_PIC_FLAG_INTRA_VLC’ undeclared (first use in this function) 74 | controls->picture.flags |= V4L2_MPEG2_PIC_FLAG_INTRA_VLC; | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ libavcodec/v4l2_request_mpeg2.c:77:36: error: ‘V4L2_MPEG2_PIC_FLAG_Q_SCALE_TYPE’ undeclared (first use in this function) 77 | controls->picture.flags |= V4L2_MPEG2_PIC_FLAG_Q_SCALE_TYPE; | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ libavcodec/v4l2_request_mpeg2.c:80:36: error: ‘V4L2_MPEG2_PIC_FLAG_ALT_SCAN’ undeclared (first use in this function) 80 | controls->picture.flags |= V4L2_MPEG2_PIC_FLAG_ALT_SCAN; | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~ libavcodec/v4l2_request_mpeg2.c:83:36: error: ‘V4L2_MPEG2_PIC_FLAG_REPEAT_FIRST’ undeclared (first use in this function) 83 | controls->picture.flags |= V4L2_MPEG2_PIC_FLAG_REPEAT_FIRST; | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ libavcodec/v4l2_request_mpeg2.c:86:36: error: ‘V4L2_MPEG2_PIC_FLAG_PROGRESSIVE’ undeclared (first use in this function); did you mean ‘V4L2_DV_BT_CAP_PROGRESSIVE’? 86 | controls->picture.flags |= V4L2_MPEG2_PIC_FLAG_PROGRESSIVE; | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | V4L2_DV_BT_CAP_PROGRESSIVE libavcodec/v4l2_request_mpeg2.c: In function ‘v4l2_request_mpeg2_end_frame’: libavcodec/v4l2_request_mpeg2.c:121:19: error: ‘V4L2_CID_STATELESS_MPEG2_SEQUENCE’ undeclared (first use in this function); did you mean ‘V4L2_CID_STATELESS_VP8_FRAME’? 121 | .id = V4L2_CID_STATELESS_MPEG2_SEQUENCE, | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | V4L2_CID_STATELESS_VP8_FRAME libavcodec/v4l2_request_mpeg2.c:126:19: error: ‘V4L2_CID_STATELESS_MPEG2_PICTURE’ undeclared (first use in this function); did you mean ‘V4L2_CID_STATELESS_VP8_FRAME’? 126 | .id = V4L2_CID_STATELESS_MPEG2_PICTURE, | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | V4L2_CID_STATELESS_VP8_FRAME libavcodec/v4l2_request_mpeg2.c:131:19: error: ‘V4L2_CID_STATELESS_MPEG2_QUANTISATION’ undeclared (first use in this function) 131 | .id = V4L2_CID_STATELESS_MPEG2_QUANTISATION, | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ make: *** [ffbuild/common.mak:59: libavcodec/v4l2_request_mpeg2.o] Error 1 make: *** Waiting for unfinished jobs.... 0 Quote
jernej Posted September 7, 2021 Posted September 7, 2021 You are using sources meant to be used on 5.14. 0 Quote
kprasadvnsi Posted September 7, 2021 Posted September 7, 2021 23 minutes ago, jernej said: You are using sources meant to be used on 5.14. where can I find ffmpeg source for 5.13 kernel? 0 Quote
jernej Posted September 7, 2021 Posted September 7, 2021 I guess this branch should work: https://github.com/jernejsk/FFmpeg/commits/v4l2-request-hwaccel-4.3.1-new 5.13 is not something we cared for at LE (no build contains that version). This branch will be removed in near future. 0 Quote
kprasadvnsi Posted September 8, 2021 Posted September 8, 2021 FFmpeg build fine with v4l2-request-hwaccel-4.3.1-new branch. The media file bbb_sunflower_1080p_30fps_normal.mp4 decoding fine at 90fps with one A72 core at 100% uses. I have tried building MPV player with it but it doesn't look like using the VPU. A72 cores are at 30-40% and A53 are around 15-20%. How can I check if it actually using VPU? MPV playback log file. mpv_sucks.log 1 Quote
kprasadvnsi Posted September 8, 2021 Posted September 8, 2021 To answer my own question. The mpv player is not using the VPU. I don't know what options to set for mpv to work. 0 Quote
usual user Posted September 8, 2021 Posted September 8, 2021 21 hours ago, jernej said: You are using sources meant to be used on 5.14. A few days ago I also tried to build a ffmpeg with v4l2-request-hwaccel. Because my distribution already uses 4.4, I used these patches. Initially, I got similar error messages as @kprasadvnsi, but the 5.14 kernel header hint was the missing link. I also have a flawlessly compiled ffmpeg package now, but I also don't know if additional patches are needed for other mainline components or how to check the v4l2-request-hwaccel functionality. 1 Quote
jernej Posted September 8, 2021 Posted September 8, 2021 Test command would be something like: ffmpeg -loglevel debug -hwaccel drm -hwaccel_output_format drm_prime -i video.mkv -f null - I think it should be pretty self evident from the log if request api is used or not. Notes: 1. If you don't use LE kernel patches, decoding interlaced H264 content won't work on RK. HEVC also won't work without kernel patches. 2. mpv most probably needs patching. At least it did in the past. 0 Quote
kprasadvnsi Posted September 8, 2021 Posted September 8, 2021 37 minutes ago, jernej said: Test command would be something like: ffmpeg -loglevel debug -hwaccel drm -hwaccel_output_format drm_prime -i video.mkv -f null - I think it should be pretty self evident from the log if request api is used or not. Notes: 1. If you don't use LE kernel patches, decoding interlaced H264 content won't work on RK. HEVC also won't work without kernel patches. 2. mpv most probably needs patching. At least it did in the past. I am getting much better performance using your test commands. I am getting 235 fps at 5% average CPU load. Where should I look for patching in the mpv to get this working? 1 Quote
usual user Posted September 8, 2021 Posted September 8, 2021 I don't know what I really need to look for, but since "v4l2_request" in ffmpeg.log is mentioned several times, I think "v4l2-request-hwaccel" is present. 1 hour ago, jernej said: If you don't use LE kernel patches, decoding interlaced H264 content won't work on RK. HEVC also won't work without kernel patches. Guess now I have to figure where LE is hideing the current 5.14.0 kernel patches. 0 Quote
jernej Posted September 8, 2021 Posted September 8, 2021 1 hour ago, kprasadvnsi said: I am getting much better performance using your test commands. I am getting 235 fps at 5% average CPU load. Note that this is pure decoding speed (which can be optimized further). Rendering may add additional overhead. 1 hour ago, kprasadvnsi said: Where should I look for patching in the mpv to get this working? Last time I heard, this patch was needed: https://github.com/Kwiboo/mpv/commit/91f2b090eef12f48cf1d9e9157e5eefe6a60be6a If this doesn't work, then I don't know. Note that ffmpeg request api support is still experimental. It's quiet likely patches will be rewritten before they will be submitted to ffmpeg mailing list again. That's why there is no official support for it in any player. You are very lucky that somebody in the past actually looked into mpv. 1 hour ago, usual user said: Guess now I have to figure where LE is hideing the current 5.14.0 kernel patches. LE switched to 5.14 last week, so patches are already in. RK patches are here: https://github.com/LibreELEC/LibreELEC.tv/tree/master/projects/Rockchip/patches/linux/default I suggest taking at least v4l patches. 0 Quote
kprasadvnsi Posted September 9, 2021 Posted September 9, 2021 Quote Last time I heard, this patch was needed: https://github.com/Kwiboo/mpv/commit/91f2b090eef12f48cf1d9e9157e5eefe6a60be6a If this doesn't work, then I don't know. These changes are already merged in upstream mpv. The patch was for cedrus driver and tested on a Allwinner H3 device with following parameters. --vo=gpu --gpu-context=drm --hwdec=drm Using these parameters with my mpv build it gives the following error. kp@orangepi4:~/trails/mpv-build/mpv/build$ ./mpv --log-file=mpv_sucks.log --vo=gpu --gpu-context=drm --hwdec=drm bbb_sunflower_1080p_30fps_normal.mp4 Error parsing option gpu-context (option parameter could not be parsed) Setting commandline option --gpu-context=drm failed. Exiting... (Fatal error) The only option that didn't throw error is --gpu-context=x11egl 0 Quote
jernej Posted September 9, 2021 Posted September 9, 2021 9 hours ago, kprasadvnsi said: These changes are already merged in upstream mpv. The patch was for cedrus driver Ah, good to know. BTW, that's not only for Cedrus but for rkvdec and Hantro drivers too. They all provide same interface. 9 hours ago, kprasadvnsi said: Using these parameters with my mpv build it gives the following error. It would be nice to see that more detailed log. 0 Quote
usual user Posted September 9, 2021 Posted September 9, 2021 My kernel has now 0010, 0011, 0020, 1000, 1001, 2000 and 2001 LE patch applied. I don't see any significant change in behavior, so I can't say if the ffmpeg-driven video pipeline realy uses hardware acceleration. The mpv command of @kprasadvnsi is playing for me. In the mpv.log the Codec list mentions, "h264_v4l2m2m (h264) - V4L2 mem2mem H.264 decoder wrapper", but I don't know if this is used. Neither I don't know how to select which decoder (hantro, rkvdec) will be used, it's the first time I use mpv. 0 Quote
jock Posted September 10, 2021 Posted September 10, 2021 Everything well with h264 and h265 on my side, with both hantro and rkvdec drivers on kernel 5.14 on rockchip RK322x chip (roughly same performance as H3) and latest LE patches applied. One thing that always makes me have headaches are the userland kernel headers, which are not the same kernel headers used to compile modules and provided by armbian as a deb package. The userland kernel headers on Armbian comes from the distribution (debian or ubuntu) and match the distribution kernel version, not the really installed kernel version. Mismatch may cause compiling troubles (missing or duplicate structs or costants) or even subtle errors during runtime (unable to initialize things...) To make the userland kernel headers, compile the kernel on the board (with LE patches applied) and then running make headers_install INSTALL_HDR_PATH=/usr will install them in the proper place Just after doing it is possible to go further compiling and installing ffmpeg and then compiling and installing mpv. This is what I use to compile and install ffmpeg: ./configure --arch=armhf --extra-libs='-Lpthread -Lm' --disable-debug --enable-stripping --enable-gpl --enable-nonfree --enable-logging --disable-doc --enable-pic --enable-optimizations --enable-ffmpeg --enable-avdevice --enable-avcodec --enable-avformat --enable-swscale --enable-postproc --enable-avfilter --disable-devices --enable-pthreads --disable-w32threads --enable-network --disable-gnutls --disable-gray --enable-swscale-alpha --disable-small --enable-dct --enable-fft --enable-mdct --enable-rdft --disable-crystalhd --disable-vaapi --disable-vdpau --enable-libdrm --enable-version3 --enable-runtime-cpudetect --enable-hardcoded-tables --enable-encoder=ac3 --enable-encoder=aac --enable-encoder=wmav2 --enable-encoder=mjpeg --enable-encoder=png --enable-hwaccels --enable-muxer=spdif --enable-muxer=adts --enable-muxer=asf --enable-muxer=ipod --enable-muxer=mpegts --enable-demuxers --enable-parsers --enable-bsfs --enable-filters --enable-bzlib --enable-libspeex --enable-libx264 --enable-libx265 --enable-zlib --enable-asm --enable-neon --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-openssl --enable-libv4l2 --enable-v4l2-request --enable-libudev make -j4 sudo make install This is what I use to compile and install mpv: ./waf configure --enable-egl-drm And finally the mpv command line is (run on pure virtual terminal, no X11 at all): mpv -v --vo=gpu --hwdec=drm --gpu-context=drm --gpu-hwdec-interop=drmprime-drm --drm-draw-plane=overlay --drm-drmprime-video-plane=primary bbb_sunflower_1080p_30fps_normal.mp4 the --drm-draw-plane and --drm-drmprime-video-plane arguments are very imporant for my setup, probably because rk322x has no YUV capabilities for overlay plane and MPV refuses to show anything. Also they make the video and draw planes order correct (video below, GUI on top). 1 Quote
kprasadvnsi Posted September 11, 2021 Posted September 11, 2021 16 hours ago, jock said: To make the userland kernel headers, compile the kernel on the board (with LE patches applied) and then running Which LE patches I need? 0 Quote
usual user Posted September 11, 2021 Posted September 11, 2021 Thank you @jock for your detailed explanation. Since I use a different distribution, your instructions don't apply to my environment, but I know how to do it for myself. I just need to know what to do and @jernej has already pointed this out (needed patches and build requirements). 22 hours ago, jock said: One thing that always makes me have headaches are the userland kernel headers, which are not the same kernel headers used to compile modules I haven't considered this aspect so far, as it hasn't been mentioned that LE patched headers are required. I just applied all the LE kernel patches and just omitted obviously not necessary ones. Because ffmpeg build flawless with the native headers, I moved on. I have installed round about twenty kernels at the same time and it will be decided at boot which one to use. Therefore, I only move in new headers when necessary. gst-play-1.0, ffplay and mpv now play bbb_sunflower_1080p_30fps_normal.mp4 with similar achievement. For gst-play-1.0 I know how to visualize the composed video pipeline. And due to the v4l2slh264dec (V4L2 Stateless H.264 Video Decoder) component in gst-play-pipiline.pdf I know for sure that hardware acceleration is used. So I guess the ffmpeg video pipeline is also using hardware acceleration. gst-play-1.0 plays bbb_sunflower_2160p_30fps_normal.mp4 with same CPU utilization. For ffplay and mpv CPU utilization ramps up significantly and frame drops are reported. The video also plays jerky. I can't say which components of the ffmpeg video pipeline is the limiting factor here. Because the kernel patches also benefit the gstreamer framework, I will probably continue to use them. Let's see what explodes when 5.15.0-rc1 is available next week. 0 Quote
jernej Posted September 11, 2021 Posted September 11, 2021 5 hours ago, usual user said: I haven't considered this aspect so far, as it hasn't been mentioned that LE patched headers are required. ffmpeg patches use only public, stable headers - at 5.14 that means MPEG2, VP8 and H264. For HEVC, ffmpeg patches contain their own copy of header file. That's why it's important to have matching kernel headers (for stable codecs) and LE patches (for unstable/non-public API and other improvements - if you care). 5 hours ago, usual user said: For ffplay and mpv CPU utilization ramps up significantly and frame drops are reported. AFAIK ffplay doesn't use HW acceleration. At least I never managed to force it. That's why I gave ffmpeg based test command. I can't help you with mpv... 5 hours ago, usual user said: Let's see what explodes when 5.15.0-rc1 is available next week. Only HEVC patches must be realigned, other codecs has been declared stable and thus API will not change. However, drivers implementing that API were improved in some areas. 0 Quote
usual user Posted September 11, 2021 Posted September 11, 2021 Thank you @jernej for your detailed background information. 3 hours ago, jernej said: AFAIK ffplay doesn't use HW acceleration. At least I never managed to force it. That's why I gave ffmpeg based test command. I can't help you with mpv... In the past I didn't care much about ffmpeg driven video pipelines. I made several attempts, but because of the sparse documentation and not even the basic support in mainline, I gave up pretty quickly. I do this out of curiosity and for early use of future maninline support. So as long as I can't find information on how to use it in daily use cases, I'll ignore it again. At least I have now some kernel patches that benefit gstreamer framework that I can track till they land in mainline. 0 Quote
jernej Posted September 11, 2021 Posted September 11, 2021 1 hour ago, usual user said: At least I have now some kernel patches that benefit gstreamer framework that I can track till they land in mainline. Current master branch of gst-plugins-bad (what will land in next stable release) should already work pretty well with all stable codecs (MPEG2, H264 and VP8). HEVC patches won't help, because LE kernel patches add missing functionalities in API and those changes are naturally compatible only with ffmpeg patches from LE. But with every kernel release those patches get smaller. 0 Quote
usual user Posted September 12, 2021 Posted September 12, 2021 11 hours ago, jernej said: Current master branch of gst-plugins-bad (what will land in next stable release) should already work pretty well with all stable codecs (MPEG2, H264 and VP8). I'm currently at gstreamer1-1.19.1, which works for my daily use cases flawless with the distribution package. But as soon as a deficiency should show up, it would be no big deal for me to rebuild with current master or pull in in-flight patches. But till then I will let the improvements trickle in via the regular updates. Out of curiosity, I just started two instances of gst-play-1.0 with different videos at the same time, and both videos play flawlessly at the same time. OK, the mixed sound is a bit confusing. 0 Quote
usual user Posted September 15, 2021 Posted September 15, 2021 (edited) On 9/11/2021 at 11:25 AM, usual user said: Let's see what explodes when 5.15.0-rc1 is available next week. Moved on to 5.15.0-rc1. But applying the LE patches makes the kernel unbootable for rockchip. I had now omitted 2001 also, as it no longer applies cleanly, and of course the cherry-picking 0010, which have landed. Guess I have to take a closer look at which patches are really important to me and see if they are the cause. With only linux-0011-v4l2-from-list, linux-1001-v4l2-rockchip and 2000-v4l-wip-rkvdec-vp9 applied the kernel keeps working. Although the drm patches seem interesting, I'll leave them out for now. Edited September 15, 2021 by usual user additional information 0 Quote
jock Posted September 15, 2021 Posted September 15, 2021 7 hours ago, usual user said: Moved on to 5.15.0-rc1. But applying the LE patches makes the kernel unbootable for rockchip. I had now omitted 2001 also, as it no longer applies cleanly, and of course the cherry-picking 0010, which have landed. Guess I have to take a closer look at which patches are really important to me and see if they are the cause. With only linux-0011-v4l2-from-list, linux-1001-v4l2-rockchip and 2000-v4l-wip-rkvdec-vp9 applied the kernel keeps working. Although the drm patches seem interesting, I'll leave them out for now. I'm in the middle of exploring how LE patches behave with rockchip 32 bit (rk3288 actually) and kernel 5.14. I already did the process some weeks ago for kernel 5.10 and on my rk3288 tv box it worked so far (hantro and rkvdec both were working flawlessy). If you're interested I can point the repository to you when I'm done. 0 Quote
usual user Posted September 15, 2021 Posted September 15, 2021 1 hour ago, jock said: If you're interested I can point the repository to you when I'm done. Thank you for the offer @jock. But 5.14.0 wise I had all VPU relevant LE patches and even the drm ones applied. With gstreamer framework it is working, even for bbb_sunflower_2160p_30fps_normal.mp4 without any problem. It is mpv that seems not be able to cope with the hight data rate. Useing --hwdec=drm-copy makes the CPU utilization ramp up go away and the video plays smoothly, but to slow and the sound get out of sync. I don't think I am missing anything kernel wise. Just tried to drop 2000-v4l-wip-rkvdec-vp9 in favour of this more recent patchset. Resolving a merge conflict to apply the LE patches on top was easy: Spoiler --- linux-0011-v4l2-from-list.patch.orig<------>2021-09-08 20:04:26.000000000 +0200 +++ linux-0011-v4l2-from-list.patch<--->2021-09-15 19:00:00.000000000 +0200 @@ -756,9 +756,9 @@ index d31344c4acaa..d068383aeea8 100644 <----><------><------>return -EINVAL; .. <----><------>if (ctx->valid_fmt && ctx->valid_fmt != rkvdec_valid_fmt(ctx, ctrl)) -@@ -155,6 +151,9 @@ static const struct rkvdec_ctrls rkvdec_h264_ctrls = { +@@ -101,6 +151,9 @@ static const struct rkvdec_ctrls rkvdec_ .. - static const u32 rkvdec_h264_decoded_fmts[] = { + static const u32 rkvdec_h264_vp9_decoded_fmts[] = { <---->V4L2_PIX_FMT_NV12, +<---->V4L2_PIX_FMT_NV15, +<---->V4L2_PIX_FMT_NV16, But the kernel build drops out with this: Spoiler drivers/staging/media/hantro/imx8m_vpu_hw.c:145:27: error: ‘V4L2_PIX_FMT_NV12_4L4’ undeclared here (not in a function); did you mean ‘V4L2_PIX_FMT_NV12M’? 145 | .fourcc = V4L2_PIX_FMT_NV12_4L4, | ^~~~~~~~~~~~~~~~~~~~~ | V4L2_PIX_FMT_NV12M So I will stay at the status quo of LE patches on 5.15.0-rc1 for now and wait till they rebase. VPU wise everything from LE is in place and seems to work. 0 Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.