Jump to content

Myy

Recommended Posts

3 hours ago, usual user said:

Yes, but I prefer the video pipeline that gst-play auto negotiates because it gives me a resizeable, semitransparent window while moveing and window decorations.

 

It is probably to blame a misconfigured display pipeline of the compositor if the video pipeline is identical.
Maybe attach the video pipeline that is negotiated with this command on GNOME:

GST_DEBUG_DUMP_DOT_DIR=. gst-play-1.0 demo320x240.mp4.149ba2bb88584b89814a1c41b5feef77.mp4

For visualisation convert the dot-file by:

dot -Tpdf foo.dot > foo.pdf

 

I found this isn't video decode problem just now.

gst-launch-1.0 videotestsrc ! glimagesink -v

this test also not work.

Link to comment
Share on other sites

9 hours ago, iamdrq said:

this test also not work.

Plays for me: video-pipeline-glimagesink.pdf

But auto negotiation selects better default: video-pipeline-autovideosink.pdf

What MESA version you are running?

Spoiler
glxinfo -B

name of display: :0
display: :0  screen: 0
direct rendering: Yes
Extended renderer info (GLX_MESA_query_renderer):
    Vendor: Panfrost (0xffffffff)
    Device: Mali-T860 (Panfrost) (0xffffffff)
    Version: 21.3.0
    Accelerated: yes
    Video memory: 3834MB
    Unified memory: yes
    Preferred profile: core (0x1)
    Max core profile version: 3.1
    Max compat profile version: 3.1
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.1
OpenGL vendor string: Panfrost
OpenGL renderer string: Mali-T860 (Panfrost)
OpenGL core profile version string: 3.1 Mesa 21.3.0
OpenGL core profile shading language version string: 1.40
OpenGL core profile context flags: (none)

OpenGL version string: 3.1 Mesa 21.3.0
OpenGL shading language version string: 1.40
OpenGL context flags: (none)

OpenGL ES profile version string: OpenGL ES 3.1 Mesa 21.3.0
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.10

 

 

Link to comment
Share on other sites

7 hours ago, usual user said:

Plays for me: video-pipeline-glimagesink.pdf

But auto negotiation selects better default: video-pipeline-autovideosink.pdf

What MESA version you are running?

  Reveal hidden contents
glxinfo -B

name of display: :0
display: :0  screen: 0
direct rendering: Yes
Extended renderer info (GLX_MESA_query_renderer):
    Vendor: Panfrost (0xffffffff)
    Device: Mali-T860 (Panfrost) (0xffffffff)
    Version: 21.3.0
    Accelerated: yes
    Video memory: 3834MB
    Unified memory: yes
    Preferred profile: core (0x1)
    Max core profile version: 3.1
    Max compat profile version: 3.1
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.1
OpenGL vendor string: Panfrost
OpenGL renderer string: Mali-T860 (Panfrost)
OpenGL core profile version string: 3.1 Mesa 21.3.0
OpenGL core profile shading language version string: 1.40
OpenGL core profile context flags: (none)

OpenGL version string: 3.1 Mesa 21.3.0
OpenGL shading language version string: 1.40
OpenGL context flags: (none)

OpenGL ES profile version string: OpenGL ES 3.1 Mesa 21.3.0
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.10

 

 

Big thanks for your Mesa version info. I found my version is 21.2.2 (this is latest version on my running ubuntu now). It's worked when upgraded to test version 21.3.0~rc5-1 (download from debian package system by hand) !

Link to comment
Share on other sites

9 hours ago, usual user said:

But I still don't understand why you insist to involve the 3D accelerator in the video pipeline while the VOP can scan out video formats directly.

GPU rendering is usually preferred in X11 environment. I don't think DRM rendering is implemented in most players under X11. Another common reason would be to use various fancy shaders for post-processing, for features that are not yet supported in driver or not at all in HW.

Link to comment
Share on other sites

On 12/2/2021 at 2:45 PM, usual user said:

But I still don't understand why you insist to involve the 3D accelerator in the video pipeline while the VOP can scan out video formats directly.

Oh, What is the VOP ? The auto negotiation selects XvImageSink(It has a window border better than waylandsink,glimagesink) that renders video frames to a drawable (XWindow) , but I use wayland it start a xwayland do it that I think it cost something.I did some test on my device, the best way is glimagesink that 4K 30fps normal paly only around 18% cpu (but  it 's strange  that some 1080p video waylandsink better than glimagesink), and I found  this gl way also work on browser like webkit.

Link to comment
Share on other sites

5 hours ago, iamdrq said:

Oh, What is the VOP ?

VOP is Video Output Processor, the term Rockchip uses for the display subsystem. Its resources are exposed through the KMS/DRM framework. The rk3399 even has two of them (vopl and vopb in DT)
In a virtual console you use it e. g. via GBM. No 3D accelerator involved for standard video playback.
In Xwindow/Wayland, the display subsystem is occupied by the server, so an application must forward video content to it and cannot directly access KMS/DRM.
Xorg uses the modesetting driver which emulates all 2D action via 3D accelerator (glamor) and transmits all results via dumb buffer to KMS/DRM. This prevents the use of specific acceleration features of the display subsystem.
In Wayland world this is a completely different story. The DRM backend in Weston and the KWin Wayland compositor at least implement proper KMS/DRM API and make use of avilable display subsystem acceleration features. Thus, video content can be played without redirection via the 3D accelerator, as on a virtual console. Of course, the 3D accelerator, if it fits, is used for all graphic eye candy by the compositor.

In IBM PC architecture world the modesetting way is the "right thing to do" (TM) as the GPU card also provides the scan out engine. Content uploaded to the GPU will never return to the CPU for display. The scan out is done direct in hardware from the GPU card.

5 hours ago, iamdrq said:

4K 30fps normal paly only around 18%

I get similar relatively rising values with the start of XvImageSink playback, but since the overall value fluctuates a lot, it's difficult to know which CPU cycles are really associated with video playback or are attributable to other desktop activities.

Link to comment
Share on other sites

Damn, missed the rebase of the  "Draft: v4l2codecs: Implement VP9 v4l2 decoder" patches and the source branch has been deleted.
With gstreamer main branch as of 11.12.2021 I get this video-pipeline-vp9.pdf and everything is working as expected.

Just discovered how to gather some video playback statistics while playing online videos. I still don't know which backend is used, but since I can play e.g. three videos in parallel, I'm pretty sure the VPU will be used.YouTube.thumb.png.5eb8b97b19a1de2629512356028c8a20.png

The eagle  has landed. \o/

Link to comment
Share on other sites

On 12/11/2021 at 6:33 PM, usual user said:

Damn, missed the rebase of the  "Draft: v4l2codecs: Implement VP9 v4l2 decoder" patches and the source branch has been deleted.
With gstreamer main branch as of 11.12.2021 I get this video-pipeline-vp9.pdf and everything is working as expected.

Just discovered how to gather some video playback statistics while playing online videos. I still don't know which backend is used, but since I can play e.g. three videos in parallel, I'm pretty sure the VPU will be used.YouTube.thumb.png.5eb8b97b19a1de2629512356028c8a20.png

The eagle  has landed. \o/

Please let us know how to replicate this on a SBC. very excited to see VP9 video playing on a browser.

Link to comment
Share on other sites

1 hour ago, kprasadvnsi said:

Please let us know how to replicate this on a SBC.

Sorry, I don't understand. For gstreamer and the kernel the support has already landed, i.e with the release of gstreamer-1.0 1.22 and kernel 5.17.0 it is working out of the box. FFMPEG still needs some floating around patches applied [1]. How this is already to be had as an early bird, is discussed in the previous posts. Either you know how to apply it for your distribution or you wait until the respective releases trickle down via updates. Unfortunately, I can't help for Armbian because I don't have the required build requirements available. With my distribution I build on target (aarch64). Kernel wise I could probably help out, but with the applications it stops because my builds will not work in your environment.
[1] @jernej pointed to the repositories of the respective versions.

Link to comment
Share on other sites

12 hours ago, usual user said:

Sorry, I don't understand. For gstreamer and the kernel the support has already landed, i.e with the release of gstreamer-1.0 1.22 and kernel 5.17.0 it is working out of the box. FFMPEG still needs some floating around patches applied [1]. How this is already to be had as an early bird, is discussed in the previous posts. Either you know how to apply it for your distribution or you wait until the respective releases trickle down via updates. Unfortunately, I can't help for Armbian because I don't have the required build requirements available. With my distribution I build on target (aarch64). Kernel wise I could probably help out, but with the applications it stops because my builds will not work in your environment.
[1] @jernej pointed to the repositories of the respective versions.

 

I have tried ffmpeg few months ago. I have no idea how to work with gstreamer.

Some things are not clear from the previous discussion.

1. What OS you are using?

2. is it using Wayland or X11?

3. What is the kernel version? if any patches are applied then where can I find them?

4. Which gstreamer you are using? I can only see version 1.19 

Link to comment
Share on other sites

On 1/21/2022 at 10:29 AM, kprasadvnsi said:

1. What OS you are using?

I am running Linux via the fedora distribution, but the distribution does not really matter. Stock fedora would not have any VP9 acceleration support in place. Only the particular rebuild packages make the difference.

 

On 1/21/2022 at 10:29 AM, kprasadvnsi said:

2. is it using Wayland or X11?

The plasma user is running the Plasma DE with Wayland backend and the lxqt user is running the LXQt DE with Xorg backend.

 

On 1/21/2022 at 10:29 AM, kprasadvnsi said:

3. What is the kernel version?

Currently installed:
5.11.12, 5.12.0, 5.12.10, 5.13.0.rc1, 5.13.0.rc2, 5.13.0.rc3, 5.13.0.rc4, 5.13.0.rc5, 5.13.0.rc6, 5.13.0.rc7, 5.13.0, 5.14.0.rc1, 5.14.0.rc2, 5.14.0.rc3, 5.14.0, 5.15.0.rc6, 5.15.0, 5.16.0.rc2, 5.16.0.rc6, 5.16.0.rc7, 5.16.0
Currently running 5.16.0 with media staging commits at time of building on top, i.e. technically 5.17.0.rc1 media wise.

 

On 1/21/2022 at 10:29 AM, kprasadvnsi said:

4. Which gstreamer you are using?

I have build from the development branch, the 1.20 release is just around the corner.

Link to comment
Share on other sites

Hi all,

 

I am tring to find a way to run HW Decode for Orange pi pc H3, almost tried everything. Pls ping me for useful solution.

 

Armbian:

Linux orangepipc 5.10.60-sunxi #21.08.1 SMP Wed Aug 25 18:19:32 UTC 2021 armv7l armv7l armv7l GNU/Linux

 

Umit

Link to comment
Share on other sites

I moved on to 5.17.0.rc3 with the latest LE patches on top. The H264, VP8 and VP9 decoders work as before. I have omitted current media staging patches (5.18.0) so far because the LE HEVC patch can no longer be applied afterwards.
The kernel now makes a HEVC decoder available again:

Spoiler

Driver Info:
        Driver name      : rkvdec
        Card type        : rkvdec
        Bus info         : platform:rkvdec
        Driver version   : 5.17.0
        Capabilities     : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : rkvdec
        Model            : rkvdec
        Serial           :
        Bus info         : platform:rkvdec
        Media version    : 5.17.0
        Hardware revision: 0x00000000 (0)
        Driver version   : 5.17.0
Interface Info:
        ID               : 0x0300000c
        Type             : V4L Video
Entity Info:
        ID               : 0x00000001 (1)
        Name             : rkvdec-source
        Function         : V4L2 I/O
        Pad 0x01000002   : 0: Source
          Link 0x02000008: to remote pad 0x1000004 of entity 'rkvdec-proc' (Video Decoder): Data, Enabled, Immutable
Priority: 2
Format Video Capture Multiplanar:
        Width/Height      : 64/64
        Pixel Format      : 'NV12' (Y/CbCr 4:2:0)
        Field             : None
        Number of planes  : 1
        Flags             :
        Colorspace        : Rec. 709
        Transfer Function : Default
        YCbCr/HSV Encoding: Default
        Quantization      : Default
        Plane 0           :
           Bytes per Line : 64
           Size Image     : 8192
Format Video Output Multiplanar:
         Width/Height      : 64/64
        Pixel Format      : 'S265' (HEVC Parsed Slice Data)
        Field             : None
        Number of planes  : 1
        Flags             :
        Colorspace        : Rec. 709
        Transfer Function : Default
        YCbCr/HSV Encoding: Default
        Quantization      : Default
        Plane 0           :
           Bytes per Line : 0
           Size Image     : 8192

Codec Controls

                     h264_level 0x00990a67 (menu)   : min=0 max=15 default=0 value=0 (1)
                                0: 1
                                1: 1b
                                2: 1.1
                                3: 1.2
                                4: 1.3
                                5: 2
                                6: 2.1
                                7: 2.2
                                8: 3
                                9: 3.1
                                10: 3.2
                                11: 4
                                12: 4.1
                                13: 4.2
                                14: 5
                                15: 5.1
                   h264_profile 0x00990a6b (menu)   : min=0 max=4 default=2 value=2 (Main)
                                0: Baseline
                                1: Constrained Baseline
                                2: Main
                                4: High
                    vp9_profile 0x00990b00 (menu)   : min=0 max=0 default=0 value=0 (0)
                                0: 0
                   hevc_profile 0x00990b67 (menu)   : min=0 max=2 default=0 value=0 (Main)
                                0: Main
                                1: Main Still Picture
                                2: Main 10
                     hevc_level 0x00990b68 (menu)   : min=0 max=8 default=0 value=0 (1)

                                0: 1
                                1: 2
                                2: 2.1
                                3: 3
                                4: 3.1
                                5: 4
                                6: 4.1
                                7: 5
                                8: 5.1
    hevc_sequence_parameter_set 0x00990cf0 (unknown): type=120 value=unsupported payload type flags=has-payload
     hevc_picture_parameter_set 0x00990cf1 (unknown): type=121 value=unsupported payload type flags=has-payload
          hevc_slice_parameters 0x00990cf2 (unknown): type=122 dims=[32] flags=has-payload
            hevc_scaling_matrix 0x00990cf3 (unknown): type=123 value=unsupported payload type flags=has-payload
         hevc_decode_parameters 0x00990cf4 (unknown): type=124 value=unsupported payload type flags=has-payload
               hevc_decode_mode 0x00990cf7 (menu)   : min=1 max=1 default=1 value=1 (Frame-Based)
                                1: Frame-Based
                hevc_start_code 0x00990cf8 (menu)   : min=1 max=1 default=1 value=1 (Annex B Start Code)
                                1: Annex B Start Code

Stateless Codec Controls

               h264_decode_mode 0x00a40900 (menu)   : min=1 max=1 default=1 value=1 (Frame-Based)
                                1: Frame-Based
                h264_start_code 0x00a40901 (menu)   : min=1 max=1 default=1 value=1 (Annex B Start Code)
                                1: Annex B Start Code
    h264_sequence_parameter_set 0x00a40902 (h264-sps): value=unsupported payload type flags=has-payload
     h264_picture_parameter_set 0x00a40903 (h264-pps): value=unsupported payload type flags=has-payload
            h264_scaling_matrix 0x00a40904 (h264-scaling-matrix): value=unsupported payload type flags=has-payload
         h264_decode_parameters 0x00a40907 (h264-decode-params): value=unsupported payload type flags=has-payload
    vp9_frame_decode_parameters 0x00a40a2c (vp9-frame): value=unsupported payload type flags=has-payload
      vp9_probabilities_updates 0x00a40a2d (unknown): type=260 value=unsupported payload type flags=has-payload

MPV and gstreamer enumerate HEVC components:

Spoiler

# mpv --vd=help | grep v4l
h263_v4l2m2m (h263) - V4L2 mem2mem H.263 decoder wrapper
h264_v4l2m2m (h264) - V4L2 mem2mem H.264 decoder wrapper
hevc_v4l2m2m (hevc) - V4L2 mem2mem HEVC decoder wrapper
mpeg4_v4l2m2m (mpeg4) - V4L2 mem2mem MPEG4 decoder wrapper
mpeg1_v4l2m2m (mpeg1video) - V4L2 mem2mem MPEG1 decoder wrapper
mpeg2_v4l2m2m (mpeg2video) - V4L2 mem2mem MPEG2 decoder wrapper
vc1_v4l2m2m (vc1) - V4L2 mem2mem VC1 decoder wrapper
vp8_v4l2m2m (vp8) - V4L2 mem2mem VP8 decoder wrapper
vp9_v4l2m2m (vp9) - V4L2 mem2mem VP9 decoder wrapper

# gst-inspect-1.0 | grep v4l
v4l2codecs:  v4l2slh264dec: V4L2 Stateless H.264 Video Decoder
v4l2codecs:  v4l2slh265dec: V4L2 Stateless H.265 Video Decoder
v4l2codecs:  v4l2slmpeg2dec: V4L2 Stateless Mpeg2 Video Decoder
v4l2codecs:  v4l2slvp8alphadecodebin: VP8 Alpha Decoder
v4l2codecs:  v4l2slvp8dec: V4L2 Stateless VP8 Video Decoder
v4l2codecs:  v4l2slvp9alphadecodebin: VP9 Alpha Decoder
v4l2codecs:  v4l2slvp9dec: V4L2 Stateless VP9 Video Decoder
video4linux2:  v4l2convert: V4L2 Video Converter
video4linux2:  v4l2deviceprovider (GstDeviceProviderFactory)
video4linux2:  v4l2jpegenc: V4L2 JPEG Encoder
video4linux2:  v4l2radio: Radio (video4linux2) Tuner
video4linux2:  v4l2sink: Video (video4linux2) Sink
video4linux2:  v4l2src: Video (video4linux2) Source
video4linux2:  v4l2video1convert: V4L2 Video Converter

But none of them work with it. MVP falls back to software decoding and gstreamer throws an error message:

Spoiler

# gst-discoverer-1.0 Big_Buck_Bunny_1080_10s_30MB.vp9.webm
Analyzing file: Big_Buck_Bunny_1080_10s_30MB.vp9.webm
Done discovering file: Big_Buck_Bunny_1080_10s_30MB.vp9.webm

Properties:
  Duration: 0:00:10.000000000
  Seekable: yes
  Live: no
  container #0: WebM
    video #1: VP9
      Stream ID: 1632297b08bd9df093d463cfc8ce6477e1c4c9bb1cb47a46e5c2ab57e7981d63/001:001
      Width: 1920
      Height: 1080
      Depth: 24
      Frame rate: 30/1
      Pixel aspect ratio: 1/1
      Interlaced: false
      Bitrate: 0
      Max bitrate: 0

# gst-discoverer-1.0 Big_Buck_Bunny_1080_10s_30MB.h265.mp4
Analyzing file: Big_Buck_Bunny_1080_10s_30MB.h265.mp4
Done discovering file: Big_Buck_Bunny_1080_10s_30MB.h265.mp4
An error was encountered while discovering the file
 Driver does not support the selected stream.

I'm not sure if there is actually an error, or if I'm trying to play a stream that is not supported by the hardware.
Can anyone confirm that the HEVC decoder really works with the LE patches for a rk3399 SOC?

 

If someone wants to try a preview of 5.17.0 and wants to save the hassle of self-building, I can upload my build. It is built as a generic kernel, meaning it should fit for any aarch64 SOC as long as it has sufficient mainline support and a DTB is available. Either a DTB that comes with the kernel, or one you supply which obeys mainline bindings.
I can teach you how to put it alongside your current running kernel and decide at boot time which to run.

Link to comment
Share on other sites

On 2/10/2022 at 11:32 PM, usual user said:

I moved on to 5.17.0.rc3 with the latest LE patches on top. The H264, VP8 and VP9 decoders work as before. I have omitted current media staging patches (5.18.0) so far because the LE HEVC patch can no longer be applied afterwards.
The kernel now makes a HEVC decoder available again:

  Hide contents

Driver Info:
        Driver name      : rkvdec
        Card type        : rkvdec
        Bus info         : platform:rkvdec
        Driver version   : 5.17.0
        Capabilities     : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : rkvdec
        Model            : rkvdec
        Serial           :
        Bus info         : platform:rkvdec
        Media version    : 5.17.0
        Hardware revision: 0x00000000 (0)
        Driver version   : 5.17.0
Interface Info:
        ID               : 0x0300000c
        Type             : V4L Video
Entity Info:
        ID               : 0x00000001 (1)
        Name             : rkvdec-source
        Function         : V4L2 I/O
        Pad 0x01000002   : 0: Source
          Link 0x02000008: to remote pad 0x1000004 of entity 'rkvdec-proc' (Video Decoder): Data, Enabled, Immutable
Priority: 2
Format Video Capture Multiplanar:
        Width/Height      : 64/64
        Pixel Format      : 'NV12' (Y/CbCr 4:2:0)
        Field             : None
        Number of planes  : 1
        Flags             :
        Colorspace        : Rec. 709
        Transfer Function : Default
        YCbCr/HSV Encoding: Default
        Quantization      : Default
        Plane 0           :
           Bytes per Line : 64
           Size Image     : 8192
Format Video Output Multiplanar:
         Width/Height      : 64/64
        Pixel Format      : 'S265' (HEVC Parsed Slice Data)
        Field             : None
        Number of planes  : 1
        Flags             :
        Colorspace        : Rec. 709
        Transfer Function : Default
        YCbCr/HSV Encoding: Default
        Quantization      : Default
        Plane 0           :
           Bytes per Line : 0
           Size Image     : 8192

Codec Controls

                     h264_level 0x00990a67 (menu)   : min=0 max=15 default=0 value=0 (1)
                                0: 1
                                1: 1b
                                2: 1.1
                                3: 1.2
                                4: 1.3
                                5: 2
                                6: 2.1
                                7: 2.2
                                8: 3
                                9: 3.1
                                10: 3.2
                                11: 4
                                12: 4.1
                                13: 4.2
                                14: 5
                                15: 5.1
                   h264_profile 0x00990a6b (menu)   : min=0 max=4 default=2 value=2 (Main)
                                0: Baseline
                                1: Constrained Baseline
                                2: Main
                                4: High
                    vp9_profile 0x00990b00 (menu)   : min=0 max=0 default=0 value=0 (0)
                                0: 0
                   hevc_profile 0x00990b67 (menu)   : min=0 max=2 default=0 value=0 (Main)
                                0: Main
                                1: Main Still Picture
                                2: Main 10
                     hevc_level 0x00990b68 (menu)   : min=0 max=8 default=0 value=0 (1)

                                0: 1
                                1: 2
                                2: 2.1
                                3: 3
                                4: 3.1
                                5: 4
                                6: 4.1
                                7: 5
                                8: 5.1
    hevc_sequence_parameter_set 0x00990cf0 (unknown): type=120 value=unsupported payload type flags=has-payload
     hevc_picture_parameter_set 0x00990cf1 (unknown): type=121 value=unsupported payload type flags=has-payload
          hevc_slice_parameters 0x00990cf2 (unknown): type=122 dims=[32] flags=has-payload
            hevc_scaling_matrix 0x00990cf3 (unknown): type=123 value=unsupported payload type flags=has-payload
         hevc_decode_parameters 0x00990cf4 (unknown): type=124 value=unsupported payload type flags=has-payload
               hevc_decode_mode 0x00990cf7 (menu)   : min=1 max=1 default=1 value=1 (Frame-Based)
                                1: Frame-Based
                hevc_start_code 0x00990cf8 (menu)   : min=1 max=1 default=1 value=1 (Annex B Start Code)
                                1: Annex B Start Code

Stateless Codec Controls

               h264_decode_mode 0x00a40900 (menu)   : min=1 max=1 default=1 value=1 (Frame-Based)
                                1: Frame-Based
                h264_start_code 0x00a40901 (menu)   : min=1 max=1 default=1 value=1 (Annex B Start Code)
                                1: Annex B Start Code
    h264_sequence_parameter_set 0x00a40902 (h264-sps): value=unsupported payload type flags=has-payload
     h264_picture_parameter_set 0x00a40903 (h264-pps): value=unsupported payload type flags=has-payload
            h264_scaling_matrix 0x00a40904 (h264-scaling-matrix): value=unsupported payload type flags=has-payload
         h264_decode_parameters 0x00a40907 (h264-decode-params): value=unsupported payload type flags=has-payload
    vp9_frame_decode_parameters 0x00a40a2c (vp9-frame): value=unsupported payload type flags=has-payload
      vp9_probabilities_updates 0x00a40a2d (unknown): type=260 value=unsupported payload type flags=has-payload

MPV and gstreamer enumerate HEVC components:

  Hide contents

# mpv --vd=help | grep v4l
h263_v4l2m2m (h263) - V4L2 mem2mem H.263 decoder wrapper
h264_v4l2m2m (h264) - V4L2 mem2mem H.264 decoder wrapper
hevc_v4l2m2m (hevc) - V4L2 mem2mem HEVC decoder wrapper
mpeg4_v4l2m2m (mpeg4) - V4L2 mem2mem MPEG4 decoder wrapper
mpeg1_v4l2m2m (mpeg1video) - V4L2 mem2mem MPEG1 decoder wrapper
mpeg2_v4l2m2m (mpeg2video) - V4L2 mem2mem MPEG2 decoder wrapper
vc1_v4l2m2m (vc1) - V4L2 mem2mem VC1 decoder wrapper
vp8_v4l2m2m (vp8) - V4L2 mem2mem VP8 decoder wrapper
vp9_v4l2m2m (vp9) - V4L2 mem2mem VP9 decoder wrapper

# gst-inspect-1.0 | grep v4l
v4l2codecs:  v4l2slh264dec: V4L2 Stateless H.264 Video Decoder
v4l2codecs:  v4l2slh265dec: V4L2 Stateless H.265 Video Decoder
v4l2codecs:  v4l2slmpeg2dec: V4L2 Stateless Mpeg2 Video Decoder
v4l2codecs:  v4l2slvp8alphadecodebin: VP8 Alpha Decoder
v4l2codecs:  v4l2slvp8dec: V4L2 Stateless VP8 Video Decoder
v4l2codecs:  v4l2slvp9alphadecodebin: VP9 Alpha Decoder
v4l2codecs:  v4l2slvp9dec: V4L2 Stateless VP9 Video Decoder
video4linux2:  v4l2convert: V4L2 Video Converter
video4linux2:  v4l2deviceprovider (GstDeviceProviderFactory)
video4linux2:  v4l2jpegenc: V4L2 JPEG Encoder
video4linux2:  v4l2radio: Radio (video4linux2) Tuner
video4linux2:  v4l2sink: Video (video4linux2) Sink
video4linux2:  v4l2src: Video (video4linux2) Source
video4linux2:  v4l2video1convert: V4L2 Video Converter

But none of them work with it. MVP falls back to software decoding and gstreamer throws an error message:

  Reveal hidden contents

# gst-discoverer-1.0 Big_Buck_Bunny_1080_10s_30MB.vp9.webm
Analyzing file: Big_Buck_Bunny_1080_10s_30MB.vp9.webm
Done discovering file: Big_Buck_Bunny_1080_10s_30MB.vp9.webm

Properties:
  Duration: 0:00:10.000000000
  Seekable: yes
  Live: no
  container #0: WebM
    video #1: VP9
      Stream ID: 1632297b08bd9df093d463cfc8ce6477e1c4c9bb1cb47a46e5c2ab57e7981d63/001:001
      Width: 1920
      Height: 1080
      Depth: 24
      Frame rate: 30/1
      Pixel aspect ratio: 1/1
      Interlaced: false
      Bitrate: 0
      Max bitrate: 0

# gst-discoverer-1.0 Big_Buck_Bunny_1080_10s_30MB.h265.mp4
Analyzing file: Big_Buck_Bunny_1080_10s_30MB.h265.mp4
Done discovering file: Big_Buck_Bunny_1080_10s_30MB.h265.mp4
An error was encountered while discovering the file
 Driver does not support the selected stream.

I'm not sure if there is actually an error, or if I'm trying to play a stream that is not supported by the hardware.
Can anyone confirm that the HEVC decoder really works with the LE patches for a rk3399 SOC?

 

If someone wants to try a preview of 5.17.0 and wants to save the hassle of self-building, I can upload my build. It is built as a generic kernel, meaning it should fit for any aarch64 SOC as long as it has sufficient mainline support and a DTB is available. Either a DTB that comes with the kernel, or one you supply which obeys mainline bindings.
I can teach you how to put it alongside your current running kernel and decide at boot time which to run.

Which device are you using for testing?  

 

I can give it a try on rk3399 devices.  

Link to comment
Share on other sites

On 2/16/2022 at 7:33 AM, spikerguy said:

Which device are you using for testing?

For rk3399 I have a NanoPC-T4. But any SBC with rk3399 should be supported by my kernel as the mainline support for rk3399 SOC is already very advanced and no major features are missing.
I also run the same SD card on an ODROID N2+ for performance comparisons.
On my LX2160A device it is also running, but I only use it as workhorse.

In the meantime, I have discovered this, looks like something is still in flight before the HEVC decoder will be usable.

Looks like the discussion about HEVC has stalled. Nevertheless, I continued with the 5.17.0 release. Media staging and LE patches are applied, so it is at 5.18.0+ media wise. I skipped the LE HEVC patches as they have merge conflicts and certainly need to be adapted to the latest changes of the api.
For those who are interested I have attached a fluster-run.log, it shows what the usable decoders support.
If someone still wants to play with my kernel build, let me know and I'll talk you through the procedur.

Link to comment
Share on other sites

It seems HEVC is going to be pushed to mainline "soon":
https://www.phoronix.com/scan.php?page=news_item&px=RkVDEC-HEVC-Patches

 

I've had HEVC patches applied all since they were available, but none of my HEVC content ever really played well post kernel 5.6 era. It's probably a 10bit color conversion thing, I wonder if this patch series also tackles that, or if the problem actually lies in FFMPEG or even Kodi itself.

 

Is there any news on the merging of v4l2-request hwaccel support in FFMPEG's releases? It seems like this has been a proof-of-concept patch series since the beginning. For which I am eternally grateful, obviously. Just wondering if anyone knows if anyone is working on getting that solidified in the FFMPEG codebase.

Link to comment
Share on other sites

4 hours ago, rubenvb said:

Just wondering if anyone knows if anyone is working on getting that solidified in the FFMPEG codebase.

Not at the moment. Thing is that it's, as you said, proof of concept. RPi variant is probably the one that it will be sent upstream (asynchronous job submission, which is much more performant), but it needs to be tweaked to be more universal. Hopefully we'll start working on this soon, since HEVC request api is the only common interface between RPi4 driver and Cedrus, Hantro and RKVDEC.

 

FYI, RKVDEC HEVC will most likely miss 5.20, since it was posted just yesterday and cut off date for new media features is end of this week.

Link to comment
Share on other sites

2 hours ago, jernej said:

Let's see if RKVDEC HEVC manages to sneak in.

If it doesn't land, it's not a show stopper. Adding this patch is a trifle.

What I fear more is the availability of the necessary rebase of the LE patches:

linux-0011-v4l2-from-list.patch
linux-0020-drm-from-list.patch
linux-1000-drm-rockchip.patch
linux-1001-v4l2-rockchip.patch
linux-2001-v4l2-wip-iep-driver.patch

because I think some of it is still needed so that the VOP can also process all new output of the decoder.

Link to comment
Share on other sites

10 hours ago, jernej said:

Well, you can always wait until any issue is resolved by LE developers.

So another reason why it doesn't hurt if RKVDEC HEVC only lands in 5.20.0. What is it good for if you can decode a video but can't see the result? What I miss is where rk3399 DRM KMS development is taking place, all current media improvements have been promoted mainly by developments for other SOCs. I do not expect DRM KMS development to be a duty of LE developers.

Link to comment
Share on other sites

9 hours ago, usual user said:

all current media improvements have been promoted mainly by developments for other SOCs.

That's not the case. Collabora was contracted to work on Hantro and RKVDEC driver, that's why there is a major push to improve these drivers and make API stable. While LE always had good enough patches for missing functionality, there was lack of time to actually push them in mainline.

 

9 hours ago, usual user said:

I do not expect DRM KMS development to be a duty of LE developers.

It's not, but @Kwiboofixed most issues for media playback long time ago, so his DRM/KMS patches are still maintained in LE.

Link to comment
Share on other sites

we used armbian 22.05,linux 5.17.9.We tested the mainline Linux VPU decoding capability on the rock 3a board(soc rk3568). We used  ffmpeg from https://github.com/jernejsk/FFmpeg   and configuration:--enable-libdrm --enable-libv4l2 --enable-v4l2-request --enable-libudev --enable-static --disable-sharedb0.   When we use the following command prompt an error:

 ./ffmpeg -loglevel debug -hwaccel drm -i ../Jellyfish_1080_10s_30MB.mp4 -pix_fmt bgra -f fbdev /dev/fb0

 

Successfully opened the file.
Device creation failed: -14.
[h264 @ 0xaaaaac42f520] No device available for decoder: device type drm needed for codec h264.
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native))
Device setup failed for decoder on input stream #0:0 : Bad address

 

The VPU driver has been loaded correctly:

root@rock-3a:~/FFmpeg# dmesg |grep vpu
[   53.304850] hantro_vpu: module is from the staging directory, the quality is unknown, you have been warned.
[   53.328532] hantro-vpu fdea0000.video-codec: Adding to iommu group 0
[   53.331978] hantro-vpu fdea0000.video-codec: registered rockchip,rk3328-vpu-dec as /dev/video0

 

and

root@rock-3a:~/FFmpeg# dmesg |grep drm
[    2.752665] rockchip-drm display-subsystem: bound fe040000.vop (ops vop2_component_ops)
[    2.756569] rockchip-drm display-subsystem: bound fe0a0000.hdmi (ops dw_hdmi_rockchip_ops)
[    2.757416] [drm] Initialized rockchip 1.0.0 20140818 for display-subsystem on minor 0
[    2.757569] rockchip-drm display-subsystem: [drm] Cannot find any crtc or sizes
[    2.757724] rockchip-drm display-subsystem: [drm] Cannot find any crtc or sizes
[    4.028876] [drm] Initialized panfrost 1.2.0 20180908 for fde60000.gpu on minor 1
[   51.128088] systemd[1]: Condition check resulted in Load Kernel Module drm being skipped.
[12442.376669] rockchip-drm display-subsystem: [drm] fb0: rockchipdrmfb frame buffer device

 

and

 

root@rock-3a:~/FFmpeg# v4l2-ctl -d 0 --list-formats --list-ctrls

Codec Controls

                   h264_profile 0x00990a6b (menu)   : min=0 max=4 default=2 value=2

Stateless Codec Controls

               h264_decode_mode 0x00a40900 (menu)   : min=1 max=1 default=1 value=1
                h264_start_code 0x00a40901 (menu)   : min=1 max=1 default=1 value=1
    h264_sequence_parameter_set 0x00a40902 (unknown): type=200 flags=has-payload
     h264_picture_parameter_set 0x00a40903 (unknown): type=201 flags=has-payload
            h264_scaling_matrix 0x00a40904 (unknown): type=202 flags=has-payload
         h264_decode_parameters 0x00a40907 (unknown): type=204 flags=has-payload
           vp8_frame_parameters 0x00a409c8 (unknown): type=240 flags=has-payload
         mpeg_2_sequence_header 0x00a409dc (unknown): type=251 flags=has-payload
          mpeg_2_picture_header 0x00a409dd (unknown): type=252 flags=has-payload
   mpeg_2_quantisation_matrices 0x00a409de (unknown): type=250 flags=has-payload
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture Multiplanar

        [0]: 'NV12' (Y/CbCr 4:2:0)

 

Anyone who has tested VPU decoding on rock-3a?

Can someone help me? Appreciate,thx!!

Link to comment
Share on other sites

Hello.

I am interested in RK3399 VPU functionality on 6.x Linux kernel.
Particularly, hardware JPEG decoding and h264 encoding using FFMPEG.

 

What is the state of such functionality in 6.x kernel?
 

As far as I understand, it is supported in 4.x kernels.
Where to look if it will be necessary to port such functionality?
Is there any list of useful links, for example?

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...

Important Information

Terms of Use - Privacy Policy - Guidelines