Jump to content
  • 0

Tutorial: 3D, video acceleration and OpenCL in RK3288 boards with new 4.4 (default) kernel


JMCC
 Share

Question

[UPDATE 2018/05/21: I have created a script to configure everything described in this tutorial, plus some more things. There will be no updates to the tutorial from now on. Please refer all the questions and bug reports to the script. It can be found here:]

 

[UPDATE 2018/03/16: Fixed mpv package]

[UPDATE 2018/03/01: Added OpenCL testing through GPU cryptocurrency mining]

 

With the new default kernel based on rockchip-linux, boards with RK3288 SoC have now full kernel support for interacting with the GPU and hardware video decoder. Rockchip-linux also provides userspace libraries and other software to make everything work. In this tutorial, you'll learn to set everything up in a current Armbian Xenial default image.

 

(Note: rockchip-linux uses Debian Stretch as their base distro for all their packages. So we would be better off using the Stretch Armbian image, but it is still not stable enough for production. Therefore, we are going to install a few backports over the Xenial image, to make everything work)

 

Start by downloading and extracting this compressed file: https://mega.nz/#!hrxy0bra!pOD0L_vIaq3wn0DWwRGYGsT2PSUIJG-HZbcyrJ4m6As

It contains all the packages I am going to refer to in the tutorial

 

 

1. Install backported X server from the ubuntu repos

$ sudo apt install xserver-xorg-hwe-16.04 xserver-xorg-video-fbdev-hwe-16.04

2. Install libraries

Install all the packages under the "libs" directory, in the file you downloaded and extracted.

 

3. Install and configure tweaked X server

In order to make graphic acceleration work under X, rockchip-linux developers have created a tweaked X server. To enable it:

  1. You need to disable compositing. In the Xfce menu, navigate to "Settings > Window Manager Tweaks", choose the rightmost tab (Compositor), and uncheck "Enable display compositing".
  2. Install the packages under the "xserver" directory of our downloaded compressed file
  3. Make a backup of the file "/etc/X11/xorg.conf.d/01-armbian-defaults.conf", and replace its content with the following:
    Section "Device"
        Identifier  "Rockchip Graphics"
        Driver      "modesetting"
        Option      "AccelMethod"    "glamor"     ### "glamor" to enable 3D acceleration, "none" to disable 
        Option      "DRI"            "2"
    
        # You can disable Vsync for better 3D benchmark score, at the cost of worst video playing. Not recommended.
        # Option      "Dri2Vsync"      "false"
    
    EndSection
    Section "Screen"
        Identifier "Default Screen"
        SubSection "Display"
            Depth 24
            Modes "1920x1080" "1280x1024" "1024x768" "800x600"
        EndSubSection
    EndSection

    3. Restart X

 

4. Test 3D acceleration

$ sudo apt install glmark2-es2
$ glmark2-es2

If everything is working, you shoud see a window with rotating 3D objects, and console output shoud start like this:

=======================================================
    glmark2 2014.03+git20150611.fa71af2d
=======================================================
    OpenGL Information
    GL_VENDOR:     ARM
    GL_RENDERER:   Mali-T760
    GL_VERSION:    OpenGL ES 3.2 v1.r14p0-01rel0-git(966ed26).eedaf0c5244add8e249cecdd3c721329
=======================================================

 

5. Test video acceleration with MPV

Install the package under the directory "mpv".

When you play a video with this version, it will use GPU EGL accelerated video display, but decoding will still be via software. That means you can play up to 1080p@30fps HVEC quite smoothly, with around 80-90% CPU use. Anything above that resolution/fps will be jerky.

In order to use full acceleration with MPV, you need to be under Wayland. I intend to do some other post on that, God willing.

 

6. Test video acceleration with Gstreamer

The "official" way of using full HW video acceleration under X is via a special Gstreamer plugin created by the rockchip-linux developers. But we need to install a backported version of Gstreamer, in order for the plugin to work.

  1. Install all the packages in the "gstreamer-backport" directory
  2. Install all the packages in the "gstreamer" folder (these are for the rockchip plugin)
  3. Do "sudo apt -f install" for the unmet dependencies
  4. You can now play a video from command line with either of these commands:
    $ gst-launch-1.0 filesrc location=<videofile>  ! decodebin ! videoconvert ! rkximagesink
    $ gst-play-1.0 --videosink=rkximagesink <videofile>

    This plugin uses GPU acceleration for display, through a KMS overlay, and also hardware decoding via the rockchip mpp interface. You can play 4k@30fps HEVC with perfect smoothness, while the CPU stays under 15% use. If you don't have any 4k video, you can try with this one: http://www.libde265.org/hevc-bitstreams/sintel-4096x1744-cfg02.mkv

  5. (Notice that if you try to display any subtitle, it will make the plugin crash).

 

7. Install a Gstreamer demo player

Instead of launching videos through the command line, you can also install a simple video player that will use the rockchip plugin.

  1. Install the following packages:
    $ sudo apt-get install libqt5opengl5 libqt5qml5 libqt5quick5 libqt5widgets5 libqt5gui5 libqt5core5a qml-module-qtquick2 libqt5multimedia5 libqt5multimedia5-plugins libqt5multimediaquick-p5 qtmultimedia5-examples qtmultimedia5-doc-html

    2. Create the file "/usr/share/applications/demo-player.desktop" with the following content:

    #!/usr/bin/env xdg-open
    [Desktop Entry]
    Categories=Qt;KDE;AudioVideo;Player;Video;
    Comment=A gstreamer base player
    Exec=env QT_GSTREAMER_WIDGET_VIDEOSINK=rkximagesink /usr/lib/arm-linux-gnueabihf/qt5/examples/multimediawidgets/player/player --geometry 960x640+0+0
    GenericName=Media Player
    Icon=/usr/share/icons/gnome/48x48/categories/applications-multimedia.png
    Name=Media Player
    Type=Application
    X-KDE-StartupNotify=false
    Keywords=movie;player;media;kde;qt;
    X-Ayatana-Desktop-Shortcuts=Screen;Window

    Now you will see an entry "Media Player" in the Xfce menu, and you can also choose to play video files with this player by right-clicking on them and choosing "Open with..."

 

8. Enable acceleration for Chromium

For last, we are going to enable acceleration for Chromium Web Browser

  1. Backup the file "/etc/chromium-browser/default", and modify it to look like this:
    # Default settings for chromium-browser. This file is sourced by /bin/sh from
    # /usr/bin/chromium-browser
    
    # Options to pass to chromium-browser
    # A set of command line flags that we want to set by default.
    
    # Do not hide any extensions in the about:extensions dialog
    export CHROMIUM_FLAGS="$CHROMIUM_FLAGS --show-component-extension-options"
    
    # Don't use the GPU blacklist (bug #802933)
    export CHROMIUM_FLAGS="$CHROMIUM_FLAGS --ignore-gpu-blacklist"
    
    # Use EGL acceleration
    export CHROMIUM_FLAGS="$CHROMIUM_FLAGS --use-gl=egl"

     

  2. Launch Chrome, and enter in the address bar "chrome://gpu". If everything is OK, you should see something like this:

  3. 5a87664760b98_Screenshotfrom2018-02-1700-11-06.thumb.png.7e72fdd29689fe4dccb221fef9c64269.png

  4. Now, go to the Chrome Web Store and install the extension "h264ify". It will force youtube to use H264 instead of VP9, which has no acceleration with RK3288.

  5. Enter Youtube and play any 1080p video. It should play fairly well, dropping frames only when some kind of overlay pops up (like an ad on top of the video, or the controls overlay).

  6. Test WebGL, for example with this page: http://webglsamples.org/aquarium/aquarium.html

 

9. NEW: Testing OpenCL with a GPU cryptocurrency miner

  1. Download either of these compiled GPU miners (each of them features diferent algorithms): https://goo.gl/gHrPGg / https://goo.gl/MTSgBG
  2. Untar them, cd into the directory and run them (in case you get an error, try doing "sudo make install"). E.g.:
    ./sgminer -k lyra2rev2 -o stratum+tcp://<url of pool>:<port> -u <your username> -p d=0.050 -I 14 -w 64

     

  3. If you want to learn more, you can follow this tutorial: https://www.novaspirit.com/2017/12/21/gpu-mining-on-tinkerboard/

Mining with the skein algorithm:

5a97b311a15f6_2018-03-0108_56_55-juan@tinkerboard_.png.ac7656d3be2da516fce4195613db3246.png

 

 

Please report your experiences and suggestions. Wayland testing stays in the to-do list. Thanks,.

 

Link to comment
Share on other sites

Recommended Posts

  • 0

I'm working on a script to install all the stuff at once, but I'm not sure when it will be finished. In the meantime,in case anyone wants to try MPV with GBM (dsiplay) and RKMPP (decoding) acceleration, here are the instructions:

  1. After installing the base libs, configs and MPV according to the first post, download this package and install it.
  2. From then, you can launch mpv like this, in order to have full acceleration:
LD_LIBRARY_PATH=/opt/libmali-gbm:$LD_LIBRARY_PATH mpv --hwdec=rkmpp --vo=gpu --gpu-api=opengl --gpu-context=drm <file>

The deb package above will install the GBM version of libmali. I packaged it to install under /opt, so it did not interfere with system libs and we can still have X11 acceleration with the libmali-X11 version.

 

MPV can still be run normally, through X11-EGL, with the known performance limitations (see first post). But if you launch it with the command above, it will completely ignore the X server, and use directly GBM/KMS. That means it will only run in fullscreen, with no mouse support. You can control it with the keyboard and (I suppose) with a remote control or HDMI CEC. MPV on-screen display works normally,  so when you press the appropriate keys info is displayed on the screen (see MPV manpage for a list of keyboard controls). When you press "q" or "Q", it will exit and you will have your X11 session back as you left it.

 

In MPV's implementation of RKMPP, it is bound to GBM display, so the only way to use hardware video decoding is with this launcher. Performance is outstanding: Silk-smooth 4K playback, with minimal CPU usage.

Link to comment
Share on other sites

Armbian Linux community supported weekly builds download

  • 0

Hi, JMCC,

When playing video files with the MPV in Armbian, as mentioned above in your message, often there is a video playback error:

user@armbian:~$ LD_LIBRARY_PATH=/opt/libmali-gbm:$LD_LIBRARY_PATH mpv --hwdec=rkmpp --vo=gpu --gpu-api=opengl --gpu-context=drm /media/user/1803-416A/HEVC/Затерянные\ в\ космосе.1998.HEVC.mkv
Playing: /media/user/1803-416A/HEVC/Затерянные в космосе.1998.HEVC.mkv
 (+) Video --vid=1 (*) 'HEVC' (hevc 1920x800 23.976fps)
 (+) Audio --aid=1 --alang=rus (*) 'Kinomanija' (ac3 6ch 48000Hz)
     Audio --aid=2 --alang=rus 'Dub' (ac3 2ch 48000Hz)
     Audio --aid=3 --alang=rus 'R5' (ac3 6ch 48000Hz)
     Audio --aid=4 --alang=eng 'Original' (ac3 6ch 48000Hz)
     Subs  --sid=1 --slang=rus 'Full' (subrip)
     Subs  --sid=2 --slang=eng 'Full' (subrip)
File tags:
 Title: Lost in Space
[vo/gpu] VT_GETMODE failed: Inappropriate ioctl for device
[vo/gpu/opengl] Failed to set up VT switcher. Terminal switching will be unavailable.
mpi: mpp version: 598cae3 author: Jacob Chen DEBIAN: update rules for release_20171218-2
mpp_rt: NOT found ion allocator
mpp_rt: found drm allocator
H265D_PARSER: No start code is found.
Using hardware decoding (rkmpp).
AO: [pulse] 48000Hz 5.1(side) 6ch float
VO: [gpu] 1920x800 drm_prime
[vo/gpu] Using HW-overlay mode. No GL filtering is performed on the video!
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 02:10:14 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 02:10:14 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 02:10:14 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 02:10:14 (0%) A-V:  0.000

Exiting... (Quit)

Instead of the video image - an empty black screen, although there is sound.

I installed ffmpeg and mpv deb-packages and you collected the package GBM version of libmali in Debian firmware from Rockchip - there are no such problems, everything is beautifully played. I thought maybe Ffmpeg was assembled in Debian and Armbian Ubuntu with different keys. Checked, the difference, seemingly insignificant, ffmpeg in Debian compiled with additional keys:

 --enable-librsvg --enable-libxml2 --cross-prefix=arm-linux-gnueabihf- --arch=armhf --target-os=linux

I also checked that the libraries libsdl2, libxml2 and librsvg in Armbian are installed.

Any idea what might be the problem?

Link to comment
Share on other sites

  • 0
5 hours ago, pro777 said:

Any idea what might be the problem?

This tutorial is still here for educational purposes only. I created a script that will automatically install and configure everything. It is meant to be installed in a fresh Armbian image, either stable or development. You can download it here:

Now, you mention that you have a video playback error "often". So it means that it does not always happen, right? There is a reason for it.

 

First, let me explain briefly the difference between the approach you can read in Rockchip's repo and ours. According to their docs (https://github.com/rockchip-linux/rk-rootfs-build/tree/master/packages/armhf/others), they tell you to download the GBM libmali and replace with it the system one (which is the X11 version). Then they tell you to stop the X server, and run MPV. It will just use the system libmali, and display video thorugh GBM.  But you won't be able to use X11 acceleration until you restore the original X11 libmali.

 

We, instead, install the GBM libmali under /opt, and therefore there is no need to get rid of the system X11 libmali: both can be installed at the same time. But I am discovering now that, if both are not only installed, but also used at the same time, you get conflicts between them, taking over some resources and making it impossible for the other version of the library to use them. That is why you experience these black screens only sometimes: it happens only when you have done something that caused the X11 library to take possession of the resources the GBM library needs, or vice versa.

 

So far, I haven't been able to debug which are those resources causing the conflict, nor if there is a way to have the libraries release them. In the meantime, I recommend you two possible workarounds:

  1. When you hit any of those problems, reboot and the resource will be free, so you will be able to play the video.
  2. Stop the X server before using a GBM accelerated player. You do it with these steps:
    1. Change to a console virtual terminal, e.g. by pressing Ctrl+Alt+F1
    2. Log in
    3. Issue "sudo service nodm stop" or "sudo service lightdm stop", depending on the DM you are using

Please follow up in any new finding you do about this issue.

Link to comment
Share on other sites

  • 0

I wonder who put so much logs in the VPU code ? It's just unreadable ! ... :D

 

Jokes aside, if you see "init failed" in the logs, this means that the VPU driver could not load so... Yeah playing videos will be done with the CPU.

 

I'm currently trying to understand the different ways to make user-space memory usable with DMA operations. Technically the hardware have no idea of what is "user-memory" (as demonstrated by various Firewire/USB DMA security bypass issues on Linux/Mac/Windows), so it's more about kernel securities and IOMMU configuration.

 

That aside, I was included in a recent discussion with the Chromium developers and the Rockchip developers and it seems that the Chromium ones are using the v4l2 driver instead, which is more cleanly documented.
However the V4L2 API changed during the 4.x era. Most of the video formats related defines and structures were moved to the user-space library, so the driver can't be ported "as-is". Also, this driver lacks some formats.

Still, it's much cleaner and easier to understand : https://github.com/rockchip-linux/kernel/tree/release-4.4/drivers/media/platform/rockchip-vpu

Link to comment
Share on other sites

  • 0

It seems to be that one. I don't know why they chose to build their MPP thingy instead, though.

 

That said, up to some 4.x version of the kernel, the V4L2 kernel API had access to a lot of structures and macros that related to video formats (H264, H265, VP8, ...). These structures and macros has been moved the V4L2 user API afterwards, making such drivers a bit difficult to port.

 

Link to comment
Share on other sites

  • 0

thank you so much @JMCC

 

Have been trying to get this to work for a few days.  Now I have sgminer recognising OpenCL on my tinkerboard! (shame about lack of cryptonight support on ARM).

 

Running 16.04 xenial armbian build: Armbian_5.41_Tinkerboard_Ubuntu_xenial_default_4.4.119_desktop.7z

 

Just hope there's a currency this thing will still mine.

 

Once again, thank you for putting all the time into providing this tutorial for us.  We'd never get there without people like you!!

 

Regards,

 

Aaron.

 

Link to comment
Share on other sites

Guest
This topic is now closed to further replies.
 Share

×
×
  • Create New...