Jump to content

Myy

Members
 View Profile  See their activity

  • Posts

    365

  • Joined

  • Last visited

 Content Type 

Posts posted by Myy

    The VPU driver

    in Tinkerboard

    Posted

    So... I tested it with the old patched version of FFMPEG but it didn't work (´・ω・`　)

     

    When I mean "it didn't work", mpv played the video but when I checked the logs quickly after, I found that it tried to use the V4L2 output, and then failed and then switched back to the software decoder, which IS fast enough to decode and display MPEG2 at ~30 FPS.

     

    Now, I'm compiling the new FFMPEG but it takes a while...

     

    !! HOWEVER !!

     

    I made a special compilation script that fetches Kwiboo's kernel and compile it automatically, with the old Mali drivers though (I should try with your r28p0).

    As always, the script is autonomous, so you can just grab the script, run it and have a freshly patched and compiled kernel, ready to install in your system.

    Then you can always edit it and play with it.

     

    https://raw.githubusercontent.com/Miouyouyou/RockMyy/VPU-V4L2-5.x/GetPatchAndCompileKernel.sh

     

    I should follow with a few scripts to compile the patched FFMPEG and MPV, so that everyone can test the whole thing.

     

    Now, maybe I'm just missing something, and maybe someone else can test the whole thing and be like "You're fucking dumb ! You forgot to activate this here ! And now everything works !".

     

    Anyway, have a try everyone ( `・ω・́)

    0

    The VPU driver

    in Tinkerboard

    Posted

    VIDIOC_QUERYBUF hits v4l_querybuf, which calls the driver vidioc_querybuf function pointer, which actually points to the global v4l2 helper : v4l2_m2m_ioctl_querybuf .

     

    This helpers calls v4l2_m2m_querybuf which sets its return value with the return value of vb2_querybuf.

     

    At that point, I thought that either the type, or the index of the queried buffer was invalid. Turns out that it wasn't the case.

    EINVAL (-22) is set by __verify_planes_array, which checks if the queried buffer "length" field is at least equal to the queued buffer "num_planes" field (and if the index isn't absurd). The check fails, __verify_planes_array returns -22 and the returned value bobble up to the ioctl handler, which sends the errno value to the v4l2-request-test software.

     

    Now I need to understand why the tester sets the buffer "length" to some value (3) when queuing it, and then use another value (1) when querying it.

    Trying to modify the software by all the buffers length to 3 just led to some a SIGSEGV... However, the VIDIOC_QUERYBUF returned 0 (SUCCESS) before the SIGSEGV. So there's that...

     

    So yeah, tomorrow will be : debugging the tester... and learning a little more about how it works.

    0

    The VPU driver

    in Tinkerboard

    Posted

    So, in order : The kernel used for V4L2 patches seems to be the one implementing the Allwinner SoC VPU : https://github.com/bootlin/linux-cedrus/tree/cedrus/5.0/next

    It's the one providing all the H264/HEVC definitions at least.

     

    At least, I know where to be base myself on.

     

    Also, it seems that the tester I used was sending the VIDIOC_TRY_FMT ioctl query, with the following format type : V4L2_BUF_TYPE_VIDEO_CAPTURE

    The VIDIOC_TRY_FMT ioctl is handled by v4l_try_fmt , and v4l_try_fmt call the v4l2_ioctl_ops->vidioc_try_fmt_vid_cap setup by the driver.

    Turns out that the Rockchip VPU driver doesn't implement this function pointer.

    It implements vidioc_try_fmt_vid_cap_mplane instead, which is only called if the type is V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE .

     

    By default, on the tester I used, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE was not used.

    However, turns out that Ezequiel already provided a version testing for Multiplanar first : https://github.com/ezequielgarcia/v4l2-request-test/commit/9135eab85e9a1acc6e575fa5225b81e66f70e0d0

    Which is available in its "non-sunxi" branch.

    Now, when using the "Network" function of Github, I see that Randy Li also provided some improvements to test his own driver : https://github.com/ezequielgarcia/v4l2-request-test/network

     

    That said, with the new tester I get stuck at the VIDIOC_QUERYBUF request failing now...

    0

    The VPU driver

    in Tinkerboard

    Posted

    Okay, I'm trying (again) this week-end. So the Randy-Li patch not including VPU Decoder v1 for RK3288 chips, I've giving up on it at the moment.

    I'm retrying the Ezequiel patch, with Kwiboo additions, now trying to understand how the V4L2 request thing work, and why when using the v4l2-request-test tester, the driver sends back a "Invalid request" response...

    1

    How to test wayland on armbian?

    in Beginners

    Posted

    Your first best bet would be to try using weston. If weston doesn't work, you'll to need to fix this first.

     

    Also know that, a lot of Wayland compositors have their own way to work with the DRM/OpenGL ES stack and... it tends to play badly with embedded systems due to some "weird" mechanisms for initializing the graphic stack. So if a wayland compositor (beside weston which tends to have some 'state of art' way of initializing the graphic stack) doesn't work, it doesn't mean that others won't.

    0

    The VPU driver

    in Tinkerboard

    Posted

    Before I try to look for solutions concerning a problem that might not exist, is this video in the right format ? 

    [mpegvideo @ 0x13d62a0] Estimating duration from bitrate, this may be inaccurate
Input #0, mpegvideo, from '/tmp/grb_2.mpg':
  Duration: 00:00:27.86, bitrate: 5999 kb/s
    Stream #0:0: Video: mpeg2video (Main), yuv420p(tv, bt470bg, bottom first), 720x480 [SAR 8:9 DAR 4:3], 6000 kb/s, 29.97 fps, 29.97 tbr, 1200k tbn, 59.94 tbc
At least one output file must be specified

    I mean, I understand that there's NV12 (YV12) and NV21, and I'm not sure in which format this video is.

    0

    The VPU driver

    in Tinkerboard

    Posted · Edited by Tido
    added spoiler

    Interesting tools.

     

    I got these logs, which seems to indicate that I forgot something in my kernel, since MP2G is not available. I got a MG2S format with mpeg_2_slice_parameters, however it seems that FFMPEG used /dev/video2 instead of /dev/video1 anyway.

     

    Here are the logs excerpt :

     

    /dev/video0 (RGA)

    Spoiler

      

    
Driver Info (not using libv4l2):
        Driver name   : rockchip-rga
        Card type     : rockchip-rga
        Bus info      : platform:rga
        Driver version: 5.0.0
        Capabilities  : 0x84208000
                Video Memory-to-Memory
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04208000
                Video Memory-to-Memory
                Streaming
                Extended Pix Format
Priority: 2
Format Video Capture:
        Width/Height      : 100/100
        Pixel Format      : 'BA24'
        Field             : None
        Bytes per Line    : 400
        Size Image        : 40000
        Colorspace        : Default
        Transfer Function : Default (maps to Rec. 709)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Full Range)
        Flags             : 
Format Video Output:
        Width/Height      : 100/100
        Pixel Format      : 'BA24'
        Field             : None
        Bytes per Line    : 400
        Size Image        : 40000
        Colorspace        : Default
        Transfer Function : Default (maps to Rec. 709)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Full Range)
        Flags             : 
Selection: compose, Left 0, Top 0, Width 100, Height 100
Selection: compose_default, Left 0, Top 0, Width 100, Height 100
Selection: compose_bounds, Left 0, Top 0, Width 100, Height 100
Selection: crop, Left 0, Top 0, Width 100, Height 100
Selection: crop_default, Left 0, Top 0, Width 100, Height 100
Selection: crop_bounds, Left 0, Top 0, Width 100, Height 100

User Controls

                horizontal_flip 0x00980914 (bool)   : default=0 value=0
                  vertical_flip 0x00980915 (bool)   : default=0 value=0
                         rotate 0x00980922 (int)    : min=0 max=270 step=90 default=0 value=0 flags=modify-layout
               background_color 0x00980923 (int)    : min=0 max=16777215 step=1 default=0 value=0

    /dev/video1

    Spoiler

      

    
Driver Info (not using libv4l2):
        Driver name   : rockchip-vpu
        Card type     : rockchip,rk3288-vpu-dec
        Bus info      : platform: rockchip-vpu
        Driver version: 5.0.0
        Capabilities  : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
Priority: 2
Format Video Capture Multiplanar:
        Width/Height      : 48/48
        Pixel Format      : 'NV12'
        Field             : None
        Number of planes  : 1
        Flags             : 
        Colorspace        : JPEG
        Transfer Function : Default
        YCbCr Encoding    : Default
        Quantization      : Default
        Plane 0           :
           Bytes per Line : 48
           Size Image     : 3456
Format Video Output Multiplanar:
        Width/Height      : 48/48
        Pixel Format      : 'MG2S'
        Field             : None
        Number of planes  : 0
        Flags             : 
        Colorspace        : JPEG
        Transfer Function : Default
        YCbCr Encoding    : Default
        Quantization      : Default
        mpeg_2_slice_parameters 0x009909fa (unknown): type=103 flags=has-payload
   mpeg_2_quantization_matrices 0x009909fb (unknown): type=104 flags=has-payload

    /dev/video2

    Spoiler

      

    
Driver Info (not using libv4l2):
        Driver name   : rockchip-vpu
        Card type     : rockchip,rk3288-vpu-enc
        Bus info      : platform: rockchip-vpu
        Driver version: 5.0.0
        Capabilities  : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
Priority: 2
Format Video Capture Multiplanar:
        Width/Height      : 96/32
        Pixel Format      : 'JPEG'
        Field             : None
        Number of planes  : 1
        Flags             : 
        Colorspace        : JPEG
        Transfer Function : Default
        YCbCr Encoding    : Default
        Quantization      : Default
        Plane 0           :
           Bytes per Line : 0
           Size Image     : 6745
Format Video Output Multiplanar:
        Width/Height      : 96/32
        Pixel Format      : 'YM12'
        Field             : None
        Number of planes  : 3
        Flags             : 
        Colorspace        : JPEG
        Transfer Function : Default
        YCbCr Encoding    : Default
        Quantization      : Default
        Plane 0           :
           Bytes per Line : 96
           Size Image     : 3072
        Plane 1           :
           Bytes per Line : 48
           Size Image     : 768
        Plane 2           :
           Bytes per Line : 48
           Size Image     : 768

JPEG Compression Controls

            compression_quality 0x009d0903 (int)    : min=5 max=100 step=1 default=50 value=50

    MPV/FFMPEG

    Spoiler

      

    
Playing: /tmp/grb_2.mpg
[ffmpeg] Probing mpegvideo score:25 size:2048
[ffmpeg] Probing mpegvideo score:25 size:4096
[ffmpeg] Probing mpegvideo score:25 size:8192
[ffmpeg] Probing mp3 score:1 size:16384
[ffmpeg] Probing mpegvideo score:51 size:16384
[ffmpeg/demuxer] mpegvideo: Before avformat_find_stream_info() pos: 0 bytes read:16384 seeks:0 nb_streams:1
[ffmpeg/video] mpeg1video: Format yuv420p chosen by get_format().
[ffmpeg/demuxer] mpegvideo: All info found
[ffmpeg/demuxer] mpegvideo: Estimating duration from bitrate, this may be inaccurate
[ffmpeg/demuxer] mpegvideo: stream 0: start_time: -7686143364045.646 duration: 27.861
[ffmpeg/demuxer] mpegvideo: format: start_time: -9223372036854.775 duration: 27.861 bitrate=5999 kb/s
[ffmpeg/demuxer] mpegvideo: After avformat_find_stream_info() pos: 1223680 bytes read:1228800 seeks:0 frames:50
 (+) Video --vid=1 (mpeg2video 720x480 29.970fps)
[ffmpeg] AVHWDeviceContext: Opened DRM device /dev/dri/card0: driver rockchip version 1.0.0.
[ffmpeg/video] mpeg2video: Format drm_prime chosen by get_format().
[ffmpeg/video] mpeg2video: Format drm_prime requires hwaccel initialisation.
[ffmpeg/video] mpeg2video: ff_v4l2_request_init: avctx=0x1b2ddb0 hw_device_ctx=0x1b2c600 hw_frames_ctx=(nil)
[ffmpeg/video] mpeg2video: v4l2_request_probe_media_device: avctx=0x1b2ddb0 ctx=0x1b2f700 path=/dev/media0 driver=rockchip-vpu
[ffmpeg/video] mpeg2video: v4l2_request_probe_video_device: avctx=0x1b2ddb0 ctx=0x1b2f700 path=/dev/video2 capabilities=69222400
[ffmpeg/video] mpeg2video: v4l2_request_try_format: output buffer type do not support requests, capabilities 21
[ffmpeg/video] mpeg2video: v4l2_request_probe_video_device: try output format failed
[ffmpeg/video] mpeg2video: v4l2_request_probe_video_device: avctx=0x1b2ddb0 ctx=0x1b2f700 path=/dev/video1 capabilities=69222400
[ffmpeg/video] mpeg2video: v4l2_request_try_format: output buffer type do not support requests, capabilities 21
[ffmpeg/video] mpeg2video: v4l2_request_probe_video_device: try output format failed
[ffmpeg/video] mpeg2video: Failed setup for format drm_prime: hwaccel initialisation returned error.
[ffmpeg/video] mpeg2video: Format drm_prime not usable, retrying get_format() without it.
[ffmpeg/video] mpeg2video: Format yuv420p chosen by get_format().
[ffmpeg] detected 4 logical cores
[ffmpeg/video] mpeg2video: Format yuv420p chosen by get_format().
VO: [gpu] 720x480 => 720x540 yuv420p
V: 00:00:00 / 00:00:27 (0%)

    And then the video plays in a fluid manner if I use the Mali binary blob for OpenGL output.

    0

    The VPU driver

    in Tinkerboard

    Posted

    I'll recheck the headers to be sure, but I'm pretty confident I copied them correctly.

     

    I even had issues with the bootlin testers, since they redefine them in two headers in their repository, so I had to put two blanks files instead of their headers to compile the whole thing.

    0

    SixFab 4G and Tinkerboard

    in Tinkerboard

    Posted

    If you need Video Decoding support, you'll have to wait until the whole thing is ported and stable.

    If you need GPU support, then you can just grab the mali drivers from Rockchip and use them with recent kernels.

     

    The Debian package for the rockchip drivers is here : https://github.com/rockchip-linux/rk-rootfs-build/blob/master/packages/armhf/libmali/libmali-rk-midgard-t76x-r14p0-r0p0_1.6-1_armhf.deb

     

    Or, you could do something like this, if you don't want play too much with random Debian packages : 

    mkdir -p ~/.local/lib
cd ~/.local/lib
wget https://github.com/rockchip-linux/libmali/raw/29mirror/lib/arm-linux-gnueabihf/libmali-midgard-t76x-r14p0-r0p0-wayland.so -O libmali.so
for file_to_link in libGLESv1_CM.so{,.1,.1.0.0} libGLESv2.so{,.2,.2.0.0} libEGL.so{,.1,.1.0.0} libgbm.so{,.1,.1.0.0} libOpenCL.so{,.1,.1.0.0} libwayland-egl.so{,.1,.1.0.0}; do ln -s libmali.so "$PWD/$file_to_link"; done
echo "alias mali_launch=LD_LIBRARY_PATH=\$LD_LIBRARY_PATH:`echo ~/.local/lib`" >> ~/.bashrc

     

    0

    The VPU driver

    in Tinkerboard

    Posted

    Compiled MPV (with the habitual troubles, have to patch it for EGL, have to modify the build script to add udev to the list of libraries needed to link MPV) AND... it doesn't work.

     

    v4l2_request_probe_video_device: try output format failed... So... I'll have to debug the IOCTL to understand what's going on, why that doesn't work, are all the patches applied correctly, etc...

     

    Ugh...

     

    At least, I have the required softwares to test the whole thing. Still, the bootlin tester failing might just be a big "RED FLAG" after all.

     

    That said, I have to say that with the Mali binary OpenGL drivers, the picture is fluid on the screen...

    0

    The VPU driver

    in Tinkerboard

    Posted

    I have to add that I LOVE the fact that, by default, FFMPEG doesn't compile shared libraries. It's so fun to compile it TWICE, because you forgot this little "feature".

     

    Anyway, I was able to compile this patched FFMPEG, even though I had to resort to a few hacks to get the binaries compiled, since -ludev wasn't added automatically at compile time, and the linker complained about missing udev_ symbols references.

    Note that $LDFLAGS didn't do the trick, because for "some reason" only "-lLIBNAME" put at the end of the gcc command line were working...

    So, yeah, it basically went like this :
      

    make V=1 > command
# edit "command", add the "-ludev" at the end, save, close the file
source command
make

    A "better" way would be to edit the Makefile appropriately but I'm lazy and finding the right place to do it, in a very big Makefile, from an actual terminal... No...

     

    So, tomorrow, I'll try MPV and see if I can get something decoded.

    0

    The VPU driver

    in Tinkerboard

    Posted

    Alright, the module not loading was due to one my patch modifying the DTSI in order to adapt to Ayaka driver, which used names like "VEPU" and "VDPU". So I basically copied back the nodes as they're defined in the rk3288.dtsi of your kernel tree.

    So the module loads, media-test is able to list all the rockchip-vpu outputs (sink, ...). However this one, https://github.com/bootlin/v4l2-request-test , throws Invalid argument when trying to execute a VIDIOC_TRY_FMT . Given how IOCTL work, trying to recompile FFMPEG appears to be a "quicker" solution than debugging the tester from bootlin.

    Is there a "quick" tester that just tries to decode ONE frame of a specific format, that is up to date and works with the currently implemented V4L2 Request extension ?

    1

    The VPU driver

    in Tinkerboard

    Posted

    Alright, I've tested the inclusion of @Kwiboo patches into my compilation script (`・ω・́)

     

    However I missed a few patches here and there since they didn't apply to my patched branch... And I'm pretty sure I missed the ones on the rk3288.dtsi and rk3288-tinker.dts loading the driver correctly.

    I'll try to get this patch done tomorrow and test this with https://github.com/bootlin/v4l2-request-test first.

     

    I understand that you have a patched FFMPEG https://github.com/Kwiboo/FFmpeg/compare/4.0.3-Leia-Beta5...v4l2-request-hwaccel and MPV https://github.com/mpv-player/mpv/pull/6461 ready but...

    I'll have to remember how to cross-compile these projects, since I don't want to recompile them on the Tinky again (took 2H last time (´・ω・`))

     

    0

    The VPU driver

    in Tinkerboard

    Posted

    So, this Sunday, I took a look at the patches list, thinking I'll just have to get them and apply them to the mainline kernel and test them...

    Then I realized that, they're applied against the Rockchip tree... So I'll have to redo the whole patchset manually...

     

    This will take a few days, since I'll have to :

    - Take my patched Linux tree

    - Copy the whole VPU code from the Rockchip tree to mine

    - Generate the first patch

    - Copy every modified subsystem (V4L2, VOP, Videobuf2, Clocks, ...)

    - Generate a patch for each of these subsystems

    - Test the compilation

    - Apply additional modifications if necessary

    - Test the compiled kernel

    - Assemble the patches list

    - Put in the script

    - Test the script again

    And release...

     

    The last steps (Test the compilation, ...) are mundane. What takes time is the first steps.

     

    So, yeah, I'm on it. I just underestimated the integration time.

    0
×
×
  • Create New...

Important Information

Terms of Use - Privacy Policy - Guidelines

  I accept