iamdrq Posted December 1, 2021 Posted December 1, 2021 3 hours ago, usual user said: Yes, but I prefer the video pipeline that gst-play auto negotiates because it gives me a resizeable, semitransparent window while moveing and window decorations. It is probably to blame a misconfigured display pipeline of the compositor if the video pipeline is identical. Maybe attach the video pipeline that is negotiated with this command on GNOME: GST_DEBUG_DUMP_DOT_DIR=. gst-play-1.0 demo320x240.mp4.149ba2bb88584b89814a1c41b5feef77.mp4 For visualisation convert the dot-file by: dot -Tpdf foo.dot > foo.pdf I found this isn't video decode problem just now. gst-launch-1.0 videotestsrc ! glimagesink -v this test also not work. 0 Quote
usual user Posted December 1, 2021 Posted December 1, 2021 9 hours ago, iamdrq said: this test also not work. Plays for me: video-pipeline-glimagesink.pdf But auto negotiation selects better default: video-pipeline-autovideosink.pdf What MESA version you are running? Spoiler glxinfo -B name of display: :0 display: :0 screen: 0 direct rendering: Yes Extended renderer info (GLX_MESA_query_renderer): Vendor: Panfrost (0xffffffff) Device: Mali-T860 (Panfrost) (0xffffffff) Version: 21.3.0 Accelerated: yes Video memory: 3834MB Unified memory: yes Preferred profile: core (0x1) Max core profile version: 3.1 Max compat profile version: 3.1 Max GLES1 profile version: 1.1 Max GLES[23] profile version: 3.1 OpenGL vendor string: Panfrost OpenGL renderer string: Mali-T860 (Panfrost) OpenGL core profile version string: 3.1 Mesa 21.3.0 OpenGL core profile shading language version string: 1.40 OpenGL core profile context flags: (none) OpenGL version string: 3.1 Mesa 21.3.0 OpenGL shading language version string: 1.40 OpenGL context flags: (none) OpenGL ES profile version string: OpenGL ES 3.1 Mesa 21.3.0 OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.10 0 Quote
iamdrq Posted December 2, 2021 Posted December 2, 2021 7 hours ago, usual user said: Plays for me: video-pipeline-glimagesink.pdf But auto negotiation selects better default: video-pipeline-autovideosink.pdf What MESA version you are running? Reveal hidden contents glxinfo -B name of display: :0 display: :0 screen: 0 direct rendering: Yes Extended renderer info (GLX_MESA_query_renderer): Vendor: Panfrost (0xffffffff) Device: Mali-T860 (Panfrost) (0xffffffff) Version: 21.3.0 Accelerated: yes Video memory: 3834MB Unified memory: yes Preferred profile: core (0x1) Max core profile version: 3.1 Max compat profile version: 3.1 Max GLES1 profile version: 1.1 Max GLES[23] profile version: 3.1 OpenGL vendor string: Panfrost OpenGL renderer string: Mali-T860 (Panfrost) OpenGL core profile version string: 3.1 Mesa 21.3.0 OpenGL core profile shading language version string: 1.40 OpenGL core profile context flags: (none) OpenGL version string: 3.1 Mesa 21.3.0 OpenGL shading language version string: 1.40 OpenGL context flags: (none) OpenGL ES profile version string: OpenGL ES 3.1 Mesa 21.3.0 OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.10 Big thanks for your Mesa version info. I found my version is 21.2.2 (this is latest version on my running ubuntu now). It's worked when upgraded to test version 21.3.0~rc5-1 (download from debian package system by hand) ! 0 Quote
usual user Posted December 2, 2021 Posted December 2, 2021 3 hours ago, iamdrq said: It's worked when upgraded to test version 21.3.0~rc5-1 But I still don't understand why you insist to involve the 3D accelerator in the video pipeline while the VOP can scan out video formats directly. 0 Quote
jernej Posted December 2, 2021 Posted December 2, 2021 9 hours ago, usual user said: But I still don't understand why you insist to involve the 3D accelerator in the video pipeline while the VOP can scan out video formats directly. GPU rendering is usually preferred in X11 environment. I don't think DRM rendering is implemented in most players under X11. Another common reason would be to use various fancy shaders for post-processing, for features that are not yet supported in driver or not at all in HW. 0 Quote
iamdrq Posted December 3, 2021 Posted December 3, 2021 On 12/2/2021 at 2:45 PM, usual user said: But I still don't understand why you insist to involve the 3D accelerator in the video pipeline while the VOP can scan out video formats directly. Oh, What is the VOP ? The auto negotiation selects XvImageSink(It has a window border better than waylandsink,glimagesink) that renders video frames to a drawable (XWindow) , but I use wayland it start a xwayland do it that I think it cost something.I did some test on my device, the best way is glimagesink that 4K 30fps normal paly only around 18% cpu (but it 's strange that some 1080p video waylandsink better than glimagesink), and I found this gl way also work on browser like webkit. 0 Quote
usual user Posted December 3, 2021 Posted December 3, 2021 5 hours ago, iamdrq said: Oh, What is the VOP ? VOP is Video Output Processor, the term Rockchip uses for the display subsystem. Its resources are exposed through the KMS/DRM framework. The rk3399 even has two of them (vopl and vopb in DT) In a virtual console you use it e. g. via GBM. No 3D accelerator involved for standard video playback. In Xwindow/Wayland, the display subsystem is occupied by the server, so an application must forward video content to it and cannot directly access KMS/DRM. Xorg uses the modesetting driver which emulates all 2D action via 3D accelerator (glamor) and transmits all results via dumb buffer to KMS/DRM. This prevents the use of specific acceleration features of the display subsystem. In Wayland world this is a completely different story. The DRM backend in Weston and the KWin Wayland compositor at least implement proper KMS/DRM API and make use of avilable display subsystem acceleration features. Thus, video content can be played without redirection via the 3D accelerator, as on a virtual console. Of course, the 3D accelerator, if it fits, is used for all graphic eye candy by the compositor. In IBM PC architecture world the modesetting way is the "right thing to do" (TM) as the GPU card also provides the scan out engine. Content uploaded to the GPU will never return to the CPU for display. The scan out is done direct in hardware from the GPU card. 5 hours ago, iamdrq said: 4K 30fps normal paly only around 18% I get similar relatively rising values with the start of XvImageSink playback, but since the overall value fluctuates a lot, it's difficult to know which CPU cycles are really associated with video playback or are attributable to other desktop activities. 0 Quote
usual user Posted December 11, 2021 Posted December 11, 2021 Damn, missed the rebase of the "Draft: v4l2codecs: Implement VP9 v4l2 decoder" patches and the source branch has been deleted. With gstreamer main branch as of 11.12.2021 I get this video-pipeline-vp9.pdf and everything is working as expected. Just discovered how to gather some video playback statistics while playing online videos. I still don't know which backend is used, but since I can play e.g. three videos in parallel, I'm pretty sure the VPU will be used. The eagle has landed. \o/ 2 Quote
kprasadvnsi Posted January 20, 2022 Posted January 20, 2022 On 12/11/2021 at 6:33 PM, usual user said: Damn, missed the rebase of the "Draft: v4l2codecs: Implement VP9 v4l2 decoder" patches and the source branch has been deleted. With gstreamer main branch as of 11.12.2021 I get this video-pipeline-vp9.pdf and everything is working as expected. Just discovered how to gather some video playback statistics while playing online videos. I still don't know which backend is used, but since I can play e.g. three videos in parallel, I'm pretty sure the VPU will be used. The eagle has landed. \o/ Please let us know how to replicate this on a SBC. very excited to see VP9 video playing on a browser. 0 Quote
usual user Posted January 20, 2022 Posted January 20, 2022 1 hour ago, kprasadvnsi said: Please let us know how to replicate this on a SBC. Sorry, I don't understand. For gstreamer and the kernel the support has already landed, i.e with the release of gstreamer-1.0 1.22 and kernel 5.17.0 it is working out of the box. FFMPEG still needs some floating around patches applied [1]. How this is already to be had as an early bird, is discussed in the previous posts. Either you know how to apply it for your distribution or you wait until the respective releases trickle down via updates. Unfortunately, I can't help for Armbian because I don't have the required build requirements available. With my distribution I build on target (aarch64). Kernel wise I could probably help out, but with the applications it stops because my builds will not work in your environment. [1] @jernej pointed to the repositories of the respective versions. 0 Quote
kprasadvnsi Posted January 21, 2022 Posted January 21, 2022 12 hours ago, usual user said: Sorry, I don't understand. For gstreamer and the kernel the support has already landed, i.e with the release of gstreamer-1.0 1.22 and kernel 5.17.0 it is working out of the box. FFMPEG still needs some floating around patches applied [1]. How this is already to be had as an early bird, is discussed in the previous posts. Either you know how to apply it for your distribution or you wait until the respective releases trickle down via updates. Unfortunately, I can't help for Armbian because I don't have the required build requirements available. With my distribution I build on target (aarch64). Kernel wise I could probably help out, but with the applications it stops because my builds will not work in your environment. [1] @jernej pointed to the repositories of the respective versions. I have tried ffmpeg few months ago. I have no idea how to work with gstreamer. Some things are not clear from the previous discussion. 1. What OS you are using? 2. is it using Wayland or X11? 3. What is the kernel version? if any patches are applied then where can I find them? 4. Which gstreamer you are using? I can only see version 1.19 0 Quote
usual user Posted January 21, 2022 Posted January 21, 2022 On 1/21/2022 at 10:29 AM, kprasadvnsi said: 1. What OS you are using? I am running Linux via the fedora distribution, but the distribution does not really matter. Stock fedora would not have any VP9 acceleration support in place. Only the particular rebuild packages make the difference. On 1/21/2022 at 10:29 AM, kprasadvnsi said: 2. is it using Wayland or X11? The plasma user is running the Plasma DE with Wayland backend and the lxqt user is running the LXQt DE with Xorg backend. On 1/21/2022 at 10:29 AM, kprasadvnsi said: 3. What is the kernel version? Currently installed: 5.11.12, 5.12.0, 5.12.10, 5.13.0.rc1, 5.13.0.rc2, 5.13.0.rc3, 5.13.0.rc4, 5.13.0.rc5, 5.13.0.rc6, 5.13.0.rc7, 5.13.0, 5.14.0.rc1, 5.14.0.rc2, 5.14.0.rc3, 5.14.0, 5.15.0.rc6, 5.15.0, 5.16.0.rc2, 5.16.0.rc6, 5.16.0.rc7, 5.16.0 Currently running 5.16.0 with media staging commits at time of building on top, i.e. technically 5.17.0.rc1 media wise. On 1/21/2022 at 10:29 AM, kprasadvnsi said: 4. Which gstreamer you are using? I have build from the development branch, the 1.20 release is just around the corner. 0 Quote
umtkyck Posted February 2, 2022 Posted February 2, 2022 Hi all, I am tring to find a way to run HW Decode for Orange pi pc H3, almost tried everything. Pls ping me for useful solution. Armbian: Linux orangepipc 5.10.60-sunxi #21.08.1 SMP Wed Aug 25 18:19:32 UTC 2021 armv7l armv7l armv7l GNU/Linux Umit 0 Quote
usual user Posted February 10, 2022 Posted February 10, 2022 I moved on to 5.17.0.rc3 with the latest LE patches on top. The H264, VP8 and VP9 decoders work as before. I have omitted current media staging patches (5.18.0) so far because the LE HEVC patch can no longer be applied afterwards. The kernel now makes a HEVC decoder available again: Spoiler Driver Info: Driver name : rkvdec Card type : rkvdec Bus info : platform:rkvdec Driver version : 5.17.0 Capabilities : 0x84204000 Video Memory-to-Memory Multiplanar Streaming Extended Pix Format Device Capabilities Device Caps : 0x04204000 Video Memory-to-Memory Multiplanar Streaming Extended Pix Format Media Driver Info: Driver name : rkvdec Model : rkvdec Serial : Bus info : platform:rkvdec Media version : 5.17.0 Hardware revision: 0x00000000 (0) Driver version : 5.17.0 Interface Info: ID : 0x0300000c Type : V4L Video Entity Info: ID : 0x00000001 (1) Name : rkvdec-source Function : V4L2 I/O Pad 0x01000002 : 0: Source Link 0x02000008: to remote pad 0x1000004 of entity 'rkvdec-proc' (Video Decoder): Data, Enabled, Immutable Priority: 2 Format Video Capture Multiplanar: Width/Height : 64/64 Pixel Format : 'NV12' (Y/CbCr 4:2:0) Field : None Number of planes : 1 Flags : Colorspace : Rec. 709 Transfer Function : Default YCbCr/HSV Encoding: Default Quantization : Default Plane 0 : Bytes per Line : 64 Size Image : 8192 Format Video Output Multiplanar: Width/Height : 64/64 Pixel Format : 'S265' (HEVC Parsed Slice Data) Field : None Number of planes : 1 Flags : Colorspace : Rec. 709 Transfer Function : Default YCbCr/HSV Encoding: Default Quantization : Default Plane 0 : Bytes per Line : 0 Size Image : 8192 Codec Controls h264_level 0x00990a67 (menu) : min=0 max=15 default=0 value=0 (1) 0: 1 1: 1b 2: 1.1 3: 1.2 4: 1.3 5: 2 6: 2.1 7: 2.2 8: 3 9: 3.1 10: 3.2 11: 4 12: 4.1 13: 4.2 14: 5 15: 5.1 h264_profile 0x00990a6b (menu) : min=0 max=4 default=2 value=2 (Main) 0: Baseline 1: Constrained Baseline 2: Main 4: High vp9_profile 0x00990b00 (menu) : min=0 max=0 default=0 value=0 (0) 0: 0 hevc_profile 0x00990b67 (menu) : min=0 max=2 default=0 value=0 (Main) 0: Main 1: Main Still Picture 2: Main 10 hevc_level 0x00990b68 (menu) : min=0 max=8 default=0 value=0 (1) 0: 1 1: 2 2: 2.1 3: 3 4: 3.1 5: 4 6: 4.1 7: 5 8: 5.1 hevc_sequence_parameter_set 0x00990cf0 (unknown): type=120 value=unsupported payload type flags=has-payload hevc_picture_parameter_set 0x00990cf1 (unknown): type=121 value=unsupported payload type flags=has-payload hevc_slice_parameters 0x00990cf2 (unknown): type=122 dims=[32] flags=has-payload hevc_scaling_matrix 0x00990cf3 (unknown): type=123 value=unsupported payload type flags=has-payload hevc_decode_parameters 0x00990cf4 (unknown): type=124 value=unsupported payload type flags=has-payload hevc_decode_mode 0x00990cf7 (menu) : min=1 max=1 default=1 value=1 (Frame-Based) 1: Frame-Based hevc_start_code 0x00990cf8 (menu) : min=1 max=1 default=1 value=1 (Annex B Start Code) 1: Annex B Start Code Stateless Codec Controls h264_decode_mode 0x00a40900 (menu) : min=1 max=1 default=1 value=1 (Frame-Based) 1: Frame-Based h264_start_code 0x00a40901 (menu) : min=1 max=1 default=1 value=1 (Annex B Start Code) 1: Annex B Start Code h264_sequence_parameter_set 0x00a40902 (h264-sps): value=unsupported payload type flags=has-payload h264_picture_parameter_set 0x00a40903 (h264-pps): value=unsupported payload type flags=has-payload h264_scaling_matrix 0x00a40904 (h264-scaling-matrix): value=unsupported payload type flags=has-payload h264_decode_parameters 0x00a40907 (h264-decode-params): value=unsupported payload type flags=has-payload vp9_frame_decode_parameters 0x00a40a2c (vp9-frame): value=unsupported payload type flags=has-payload vp9_probabilities_updates 0x00a40a2d (unknown): type=260 value=unsupported payload type flags=has-payload MPV and gstreamer enumerate HEVC components: Spoiler # mpv --vd=help | grep v4l h263_v4l2m2m (h263) - V4L2 mem2mem H.263 decoder wrapper h264_v4l2m2m (h264) - V4L2 mem2mem H.264 decoder wrapper hevc_v4l2m2m (hevc) - V4L2 mem2mem HEVC decoder wrapper mpeg4_v4l2m2m (mpeg4) - V4L2 mem2mem MPEG4 decoder wrapper mpeg1_v4l2m2m (mpeg1video) - V4L2 mem2mem MPEG1 decoder wrapper mpeg2_v4l2m2m (mpeg2video) - V4L2 mem2mem MPEG2 decoder wrapper vc1_v4l2m2m (vc1) - V4L2 mem2mem VC1 decoder wrapper vp8_v4l2m2m (vp8) - V4L2 mem2mem VP8 decoder wrapper vp9_v4l2m2m (vp9) - V4L2 mem2mem VP9 decoder wrapper # gst-inspect-1.0 | grep v4l v4l2codecs: v4l2slh264dec: V4L2 Stateless H.264 Video Decoder v4l2codecs: v4l2slh265dec: V4L2 Stateless H.265 Video Decoder v4l2codecs: v4l2slmpeg2dec: V4L2 Stateless Mpeg2 Video Decoder v4l2codecs: v4l2slvp8alphadecodebin: VP8 Alpha Decoder v4l2codecs: v4l2slvp8dec: V4L2 Stateless VP8 Video Decoder v4l2codecs: v4l2slvp9alphadecodebin: VP9 Alpha Decoder v4l2codecs: v4l2slvp9dec: V4L2 Stateless VP9 Video Decoder video4linux2: v4l2convert: V4L2 Video Converter video4linux2: v4l2deviceprovider (GstDeviceProviderFactory) video4linux2: v4l2jpegenc: V4L2 JPEG Encoder video4linux2: v4l2radio: Radio (video4linux2) Tuner video4linux2: v4l2sink: Video (video4linux2) Sink video4linux2: v4l2src: Video (video4linux2) Source video4linux2: v4l2video1convert: V4L2 Video Converter But none of them work with it. MVP falls back to software decoding and gstreamer throws an error message: Spoiler # gst-discoverer-1.0 Big_Buck_Bunny_1080_10s_30MB.vp9.webm Analyzing file: Big_Buck_Bunny_1080_10s_30MB.vp9.webm Done discovering file: Big_Buck_Bunny_1080_10s_30MB.vp9.webm Properties: Duration: 0:00:10.000000000 Seekable: yes Live: no container #0: WebM video #1: VP9 Stream ID: 1632297b08bd9df093d463cfc8ce6477e1c4c9bb1cb47a46e5c2ab57e7981d63/001:001 Width: 1920 Height: 1080 Depth: 24 Frame rate: 30/1 Pixel aspect ratio: 1/1 Interlaced: false Bitrate: 0 Max bitrate: 0 # gst-discoverer-1.0 Big_Buck_Bunny_1080_10s_30MB.h265.mp4 Analyzing file: Big_Buck_Bunny_1080_10s_30MB.h265.mp4 Done discovering file: Big_Buck_Bunny_1080_10s_30MB.h265.mp4 An error was encountered while discovering the file Driver does not support the selected stream. I'm not sure if there is actually an error, or if I'm trying to play a stream that is not supported by the hardware. Can anyone confirm that the HEVC decoder really works with the LE patches for a rk3399 SOC? If someone wants to try a preview of 5.17.0 and wants to save the hassle of self-building, I can upload my build. It is built as a generic kernel, meaning it should fit for any aarch64 SOC as long as it has sufficient mainline support and a DTB is available. Either a DTB that comes with the kernel, or one you supply which obeys mainline bindings. I can teach you how to put it alongside your current running kernel and decide at boot time which to run. 0 Quote
spikerguy Posted February 16, 2022 Posted February 16, 2022 On 2/10/2022 at 11:32 PM, usual user said: I moved on to 5.17.0.rc3 with the latest LE patches on top. The H264, VP8 and VP9 decoders work as before. I have omitted current media staging patches (5.18.0) so far because the LE HEVC patch can no longer be applied afterwards. The kernel now makes a HEVC decoder available again: Hide contents Driver Info: Driver name : rkvdec Card type : rkvdec Bus info : platform:rkvdec Driver version : 5.17.0 Capabilities : 0x84204000 Video Memory-to-Memory Multiplanar Streaming Extended Pix Format Device Capabilities Device Caps : 0x04204000 Video Memory-to-Memory Multiplanar Streaming Extended Pix Format Media Driver Info: Driver name : rkvdec Model : rkvdec Serial : Bus info : platform:rkvdec Media version : 5.17.0 Hardware revision: 0x00000000 (0) Driver version : 5.17.0 Interface Info: ID : 0x0300000c Type : V4L Video Entity Info: ID : 0x00000001 (1) Name : rkvdec-source Function : V4L2 I/O Pad 0x01000002 : 0: Source Link 0x02000008: to remote pad 0x1000004 of entity 'rkvdec-proc' (Video Decoder): Data, Enabled, Immutable Priority: 2 Format Video Capture Multiplanar: Width/Height : 64/64 Pixel Format : 'NV12' (Y/CbCr 4:2:0) Field : None Number of planes : 1 Flags : Colorspace : Rec. 709 Transfer Function : Default YCbCr/HSV Encoding: Default Quantization : Default Plane 0 : Bytes per Line : 64 Size Image : 8192 Format Video Output Multiplanar: Width/Height : 64/64 Pixel Format : 'S265' (HEVC Parsed Slice Data) Field : None Number of planes : 1 Flags : Colorspace : Rec. 709 Transfer Function : Default YCbCr/HSV Encoding: Default Quantization : Default Plane 0 : Bytes per Line : 0 Size Image : 8192 Codec Controls h264_level 0x00990a67 (menu) : min=0 max=15 default=0 value=0 (1) 0: 1 1: 1b 2: 1.1 3: 1.2 4: 1.3 5: 2 6: 2.1 7: 2.2 8: 3 9: 3.1 10: 3.2 11: 4 12: 4.1 13: 4.2 14: 5 15: 5.1 h264_profile 0x00990a6b (menu) : min=0 max=4 default=2 value=2 (Main) 0: Baseline 1: Constrained Baseline 2: Main 4: High vp9_profile 0x00990b00 (menu) : min=0 max=0 default=0 value=0 (0) 0: 0 hevc_profile 0x00990b67 (menu) : min=0 max=2 default=0 value=0 (Main) 0: Main 1: Main Still Picture 2: Main 10 hevc_level 0x00990b68 (menu) : min=0 max=8 default=0 value=0 (1) 0: 1 1: 2 2: 2.1 3: 3 4: 3.1 5: 4 6: 4.1 7: 5 8: 5.1 hevc_sequence_parameter_set 0x00990cf0 (unknown): type=120 value=unsupported payload type flags=has-payload hevc_picture_parameter_set 0x00990cf1 (unknown): type=121 value=unsupported payload type flags=has-payload hevc_slice_parameters 0x00990cf2 (unknown): type=122 dims=[32] flags=has-payload hevc_scaling_matrix 0x00990cf3 (unknown): type=123 value=unsupported payload type flags=has-payload hevc_decode_parameters 0x00990cf4 (unknown): type=124 value=unsupported payload type flags=has-payload hevc_decode_mode 0x00990cf7 (menu) : min=1 max=1 default=1 value=1 (Frame-Based) 1: Frame-Based hevc_start_code 0x00990cf8 (menu) : min=1 max=1 default=1 value=1 (Annex B Start Code) 1: Annex B Start Code Stateless Codec Controls h264_decode_mode 0x00a40900 (menu) : min=1 max=1 default=1 value=1 (Frame-Based) 1: Frame-Based h264_start_code 0x00a40901 (menu) : min=1 max=1 default=1 value=1 (Annex B Start Code) 1: Annex B Start Code h264_sequence_parameter_set 0x00a40902 (h264-sps): value=unsupported payload type flags=has-payload h264_picture_parameter_set 0x00a40903 (h264-pps): value=unsupported payload type flags=has-payload h264_scaling_matrix 0x00a40904 (h264-scaling-matrix): value=unsupported payload type flags=has-payload h264_decode_parameters 0x00a40907 (h264-decode-params): value=unsupported payload type flags=has-payload vp9_frame_decode_parameters 0x00a40a2c (vp9-frame): value=unsupported payload type flags=has-payload vp9_probabilities_updates 0x00a40a2d (unknown): type=260 value=unsupported payload type flags=has-payload MPV and gstreamer enumerate HEVC components: Hide contents # mpv --vd=help | grep v4l h263_v4l2m2m (h263) - V4L2 mem2mem H.263 decoder wrapper h264_v4l2m2m (h264) - V4L2 mem2mem H.264 decoder wrapper hevc_v4l2m2m (hevc) - V4L2 mem2mem HEVC decoder wrapper mpeg4_v4l2m2m (mpeg4) - V4L2 mem2mem MPEG4 decoder wrapper mpeg1_v4l2m2m (mpeg1video) - V4L2 mem2mem MPEG1 decoder wrapper mpeg2_v4l2m2m (mpeg2video) - V4L2 mem2mem MPEG2 decoder wrapper vc1_v4l2m2m (vc1) - V4L2 mem2mem VC1 decoder wrapper vp8_v4l2m2m (vp8) - V4L2 mem2mem VP8 decoder wrapper vp9_v4l2m2m (vp9) - V4L2 mem2mem VP9 decoder wrapper # gst-inspect-1.0 | grep v4l v4l2codecs: v4l2slh264dec: V4L2 Stateless H.264 Video Decoder v4l2codecs: v4l2slh265dec: V4L2 Stateless H.265 Video Decoder v4l2codecs: v4l2slmpeg2dec: V4L2 Stateless Mpeg2 Video Decoder v4l2codecs: v4l2slvp8alphadecodebin: VP8 Alpha Decoder v4l2codecs: v4l2slvp8dec: V4L2 Stateless VP8 Video Decoder v4l2codecs: v4l2slvp9alphadecodebin: VP9 Alpha Decoder v4l2codecs: v4l2slvp9dec: V4L2 Stateless VP9 Video Decoder video4linux2: v4l2convert: V4L2 Video Converter video4linux2: v4l2deviceprovider (GstDeviceProviderFactory) video4linux2: v4l2jpegenc: V4L2 JPEG Encoder video4linux2: v4l2radio: Radio (video4linux2) Tuner video4linux2: v4l2sink: Video (video4linux2) Sink video4linux2: v4l2src: Video (video4linux2) Source video4linux2: v4l2video1convert: V4L2 Video Converter But none of them work with it. MVP falls back to software decoding and gstreamer throws an error message: Reveal hidden contents # gst-discoverer-1.0 Big_Buck_Bunny_1080_10s_30MB.vp9.webm Analyzing file: Big_Buck_Bunny_1080_10s_30MB.vp9.webm Done discovering file: Big_Buck_Bunny_1080_10s_30MB.vp9.webm Properties: Duration: 0:00:10.000000000 Seekable: yes Live: no container #0: WebM video #1: VP9 Stream ID: 1632297b08bd9df093d463cfc8ce6477e1c4c9bb1cb47a46e5c2ab57e7981d63/001:001 Width: 1920 Height: 1080 Depth: 24 Frame rate: 30/1 Pixel aspect ratio: 1/1 Interlaced: false Bitrate: 0 Max bitrate: 0 # gst-discoverer-1.0 Big_Buck_Bunny_1080_10s_30MB.h265.mp4 Analyzing file: Big_Buck_Bunny_1080_10s_30MB.h265.mp4 Done discovering file: Big_Buck_Bunny_1080_10s_30MB.h265.mp4 An error was encountered while discovering the file Driver does not support the selected stream. I'm not sure if there is actually an error, or if I'm trying to play a stream that is not supported by the hardware. Can anyone confirm that the HEVC decoder really works with the LE patches for a rk3399 SOC? If someone wants to try a preview of 5.17.0 and wants to save the hassle of self-building, I can upload my build. It is built as a generic kernel, meaning it should fit for any aarch64 SOC as long as it has sufficient mainline support and a DTB is available. Either a DTB that comes with the kernel, or one you supply which obeys mainline bindings. I can teach you how to put it alongside your current running kernel and decide at boot time which to run. Which device are you using for testing? I can give it a try on rk3399 devices. 0 Quote
usual user Posted February 16, 2022 Posted February 16, 2022 On 2/16/2022 at 7:33 AM, spikerguy said: Which device are you using for testing? For rk3399 I have a NanoPC-T4. But any SBC with rk3399 should be supported by my kernel as the mainline support for rk3399 SOC is already very advanced and no major features are missing. I also run the same SD card on an ODROID N2+ for performance comparisons. On my LX2160A device it is also running, but I only use it as workhorse. In the meantime, I have discovered this, looks like something is still in flight before the HEVC decoder will be usable. Looks like the discussion about HEVC has stalled. Nevertheless, I continued with the 5.17.0 release. Media staging and LE patches are applied, so it is at 5.18.0+ media wise. I skipped the LE HEVC patches as they have merge conflicts and certainly need to be adapted to the latest changes of the api. For those who are interested I have attached a fluster-run.log, it shows what the usable decoders support. If someone still wants to play with my kernel build, let me know and I'll talk you through the procedur. 0 Quote
rubenvb Posted July 14, 2022 Posted July 14, 2022 It seems HEVC is going to be pushed to mainline "soon": https://www.phoronix.com/scan.php?page=news_item&px=RkVDEC-HEVC-Patches I've had HEVC patches applied all since they were available, but none of my HEVC content ever really played well post kernel 5.6 era. It's probably a 10bit color conversion thing, I wonder if this patch series also tackles that, or if the problem actually lies in FFMPEG or even Kodi itself. Is there any news on the merging of v4l2-request hwaccel support in FFMPEG's releases? It seems like this has been a proof-of-concept patch series since the beginning. For which I am eternally grateful, obviously. Just wondering if anyone knows if anyone is working on getting that solidified in the FFMPEG codebase. 0 Quote
jernej Posted July 14, 2022 Posted July 14, 2022 4 hours ago, rubenvb said: Just wondering if anyone knows if anyone is working on getting that solidified in the FFMPEG codebase. Not at the moment. Thing is that it's, as you said, proof of concept. RPi variant is probably the one that it will be sent upstream (asynchronous job submission, which is much more performant), but it needs to be tweaked to be more universal. Hopefully we'll start working on this soon, since HEVC request api is the only common interface between RPi4 driver and Cedrus, Hantro and RKVDEC. FYI, RKVDEC HEVC will most likely miss 5.20, since it was posted just yesterday and cut off date for new media features is end of this week. 1 Quote
usual user Posted July 18, 2022 Posted July 18, 2022 On 7/14/2022 at 12:31 PM, rubenvb said: It seems HEVC is going to be pushed to mainline "soon" Went on to 5.19.0-rc6 with all in flight work on top. With all this in place I for the first time can confirm the HEVC decoder is working for me. \o/ See fluster-run.log for reference. 0 Quote
jernej Posted July 18, 2022 Posted July 18, 2022 Deadline for media patches was extended for another week, due to additional RC version for 5.19. Let's see if RKVDEC HEVC manages to sneak in. 0 Quote
usual user Posted July 18, 2022 Posted July 18, 2022 2 hours ago, jernej said: Let's see if RKVDEC HEVC manages to sneak in. If it doesn't land, it's not a show stopper. Adding this patch is a trifle. What I fear more is the availability of the necessary rebase of the LE patches: linux-0011-v4l2-from-list.patch linux-0020-drm-from-list.patch linux-1000-drm-rockchip.patch linux-1001-v4l2-rockchip.patch linux-2001-v4l2-wip-iep-driver.patch because I think some of it is still needed so that the VOP can also process all new output of the decoder. 0 Quote
jernej Posted July 18, 2022 Posted July 18, 2022 Well, you can always wait until any issue is resolved by LE developers. 0 Quote
usual user Posted July 19, 2022 Posted July 19, 2022 10 hours ago, jernej said: Well, you can always wait until any issue is resolved by LE developers. So another reason why it doesn't hurt if RKVDEC HEVC only lands in 5.20.0. What is it good for if you can decode a video but can't see the result? What I miss is where rk3399 DRM KMS development is taking place, all current media improvements have been promoted mainly by developments for other SOCs. I do not expect DRM KMS development to be a duty of LE developers. 0 Quote
jernej Posted July 19, 2022 Posted July 19, 2022 9 hours ago, usual user said: all current media improvements have been promoted mainly by developments for other SOCs. That's not the case. Collabora was contracted to work on Hantro and RKVDEC driver, that's why there is a major push to improve these drivers and make API stable. While LE always had good enough patches for missing functionality, there was lack of time to actually push them in mainline. 9 hours ago, usual user said: I do not expect DRM KMS development to be a duty of LE developers. It's not, but @Kwiboofixed most issues for media playback long time ago, so his DRM/KMS patches are still maintained in LE. 0 Quote
sputnik2022 Posted August 2, 2022 Posted August 2, 2022 we used armbian 22.05,linux 5.17.9.We tested the mainline Linux VPU decoding capability on the rock 3a board（soc rk3568）. We used ffmpeg from https://github.com/jernejsk/FFmpeg and configuration:--enable-libdrm --enable-libv4l2 --enable-v4l2-request --enable-libudev --enable-static --disable-sharedb0. When we use the following command prompt an error： ./ffmpeg -loglevel debug -hwaccel drm -i ../Jellyfish_1080_10s_30MB.mp4 -pix_fmt bgra -f fbdev /dev/fb0 Successfully opened the file. Device creation failed: -14. [h264 @ 0xaaaaac42f520] No device available for decoder: device type drm needed for codec h264. Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native)) Device setup failed for decoder on input stream #0:0 : Bad address The VPU driver has been loaded correctly： root@rock-3a:~/FFmpeg# dmesg |grep vpu [ 53.304850] hantro_vpu: module is from the staging directory, the quality is unknown, you have been warned. [ 53.328532] hantro-vpu fdea0000.video-codec: Adding to iommu group 0 [ 53.331978] hantro-vpu fdea0000.video-codec: registered rockchip,rk3328-vpu-dec as /dev/video0 and root@rock-3a:~/FFmpeg# dmesg |grep drm [ 2.752665] rockchip-drm display-subsystem: bound fe040000.vop (ops vop2_component_ops) [ 2.756569] rockchip-drm display-subsystem: bound fe0a0000.hdmi (ops dw_hdmi_rockchip_ops) [ 2.757416] [drm] Initialized rockchip 1.0.0 20140818 for display-subsystem on minor 0 [ 2.757569] rockchip-drm display-subsystem: [drm] Cannot find any crtc or sizes [ 2.757724] rockchip-drm display-subsystem: [drm] Cannot find any crtc or sizes [ 4.028876] [drm] Initialized panfrost 1.2.0 20180908 for fde60000.gpu on minor 1 [ 51.128088] systemd[1]: Condition check resulted in Load Kernel Module drm being skipped. [12442.376669] rockchip-drm display-subsystem: [drm] fb0: rockchipdrmfb frame buffer device and root@rock-3a:~/FFmpeg# v4l2-ctl -d 0 --list-formats --list-ctrls Codec Controls h264_profile 0x00990a6b (menu) : min=0 max=4 default=2 value=2 Stateless Codec Controls h264_decode_mode 0x00a40900 (menu) : min=1 max=1 default=1 value=1 h264_start_code 0x00a40901 (menu) : min=1 max=1 default=1 value=1 h264_sequence_parameter_set 0x00a40902 (unknown): type=200 flags=has-payload h264_picture_parameter_set 0x00a40903 (unknown): type=201 flags=has-payload h264_scaling_matrix 0x00a40904 (unknown): type=202 flags=has-payload h264_decode_parameters 0x00a40907 (unknown): type=204 flags=has-payload vp8_frame_parameters 0x00a409c8 (unknown): type=240 flags=has-payload mpeg_2_sequence_header 0x00a409dc (unknown): type=251 flags=has-payload mpeg_2_picture_header 0x00a409dd (unknown): type=252 flags=has-payload mpeg_2_quantisation_matrices 0x00a409de (unknown): type=250 flags=has-payload ioctl: VIDIOC_ENUM_FMT Type: Video Capture Multiplanar [0]: 'NV12' (Y/CbCr 4:2:0) Anyone who has tested VPU decoding on rock-3a？ Can someone help me? Appreciate，thx！！ 0 Quote
Pavel Azizov Posted June 26, 2023 Posted June 26, 2023 Hello. I am interested in RK3399 VPU functionality on 6.x Linux kernel. Particularly, hardware JPEG decoding and h264 encoding using FFMPEG. What is the state of such functionality in 6.x kernel? As far as I understand, it is supported in 4.x kernels. Where to look if it will be necessary to port such functionality? Is there any list of useful links, for example? 0 Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.