Tutorial: 3D, video acceleration and OpenCL in RK3288 boards with new 4.4 (default) kernel

[JMCC]

[UPDATE 2018/05/21: I have created a script to configure everything described in this tutorial, plus some more things. There will be no updates to the tutorial from now on. Please refer all the questions and bug reports to the script. It can be found here:]

 

[UPDATE 2018/03/16: Fixed mpv package]

[UPDATE 2018/03/01: Added OpenCL testing through GPU cryptocurrency mining]

 

With the new default kernel based on rockchip-linux, boards with RK3288 SoC have now full kernel support for interacting with the GPU and hardware video decoder. Rockchip-linux also provides userspace libraries and other software to make everything work. In this tutorial, you'll learn to set everything up in a current Armbian Xenial default image.

 

(Note: rockchip-linux uses Debian Stretch as their base distro for all their packages. So we would be better off using the Stretch Armbian image, but it is still not stable enough for production. Therefore, we are going to install a few backports over the Xenial image, to make everything work)

 

Start by downloading and extracting this compressed file: https://mega.nz/#!hrxy0bra!pOD0L_vIaq3wn0DWwRGYGsT2PSUIJG-HZbcyrJ4m6As

It contains all the packages I am going to refer to in the tutorial

 

 

1. Install backported X server from the ubuntu repos

$ sudo apt install xserver-xorg-hwe-16.04 xserver-xorg-video-fbdev-hwe-16.04

2. Install libraries

Install all the packages under the "libs" directory, in the file you downloaded and extracted.

 

3. Install and configure tweaked X server

In order to make graphic acceleration work under X, rockchip-linux developers have created a tweaked X server. To enable it:

1. You need to disable compositing. In the Xfce menu, navigate to "Settings > Window Manager Tweaks", choose the rightmost tab (Compositor), and uncheck "Enable display compositing".
2. Install the packages under the "xserver" directory of our downloaded compressed file
3. Make a backup of the file "/etc/X11/xorg.conf.d/01-armbian-defaults.conf", and replace its content with the following:

   Section "Device"
       Identifier  "Rockchip Graphics"
       Driver      "modesetting"
       Option      "AccelMethod"    "glamor"     ### "glamor" to enable 3D acceleration, "none" to disable 
       Option      "DRI"            "2"
   
       # You can disable Vsync for better 3D benchmark score, at the cost of worst video playing. Not recommended.
       # Option      "Dri2Vsync"      "false"
   
   EndSection
   Section "Screen"
       Identifier "Default Screen"
       SubSection "Display"
           Depth 24
           Modes "1920x1080" "1280x1024" "1024x768" "800x600"
       EndSubSection
   EndSection

   3. Restart X

 

4. Test 3D acceleration

$ sudo apt install glmark2-es2
$ glmark2-es2

If everything is working, you shoud see a window with rotating 3D objects, and console output shoud start like this:

=======================================================
    glmark2 2014.03+git20150611.fa71af2d
=======================================================
    OpenGL Information
    GL_VENDOR:     ARM
    GL_RENDERER:   Mali-T760
    GL_VERSION:    OpenGL ES 3.2 v1.r14p0-01rel0-git(966ed26).eedaf0c5244add8e249cecdd3c721329
=======================================================

 

5. Test video acceleration with MPV

Install the package under the directory "mpv".

When you play a video with this version, it will use GPU EGL accelerated video display, but decoding will still be via software. That means you can play up to 1080p@30fps HVEC quite smoothly, with around 80-90% CPU use. Anything above that resolution/fps will be jerky.

In order to use full acceleration with MPV, you need to be under Wayland. I intend to do some other post on that, God willing.

 

6. Test video acceleration with Gstreamer

The "official" way of using full HW video acceleration under X is via a special Gstreamer plugin created by the rockchip-linux developers. But we need to install a backported version of Gstreamer, in order for the plugin to work.

1. Install all the packages in the "gstreamer-backport" directory
2. Install all the packages in the "gstreamer" folder (these are for the rockchip plugin)
3. Do "sudo apt -f install" for the unmet dependencies
4. You can now play a video from command line with either of these commands:

   $ gst-launch-1.0 filesrc location=<videofile>  ! decodebin ! videoconvert ! rkximagesink
   $ gst-play-1.0 --videosink=rkximagesink <videofile>

   This plugin uses GPU acceleration for display, through a KMS overlay, and also hardware decoding via the rockchip mpp interface. You can play 4k@30fps HEVC with perfect smoothness, while the CPU stays under 15% use. If you don't have any 4k video, you can try with this one: http://www.libde265.org/hevc-bitstreams/sintel-4096x1744-cfg02.mkv

5. (Notice that if you try to display any subtitle, it will make the plugin crash).

 

7. Install a Gstreamer demo player

Instead of launching videos through the command line, you can also install a simple video player that will use the rockchip plugin.

1. Install the following packages:

   $ sudo apt-get install libqt5opengl5 libqt5qml5 libqt5quick5 libqt5widgets5 libqt5gui5 libqt5core5a qml-module-qtquick2 libqt5multimedia5 libqt5multimedia5-plugins libqt5multimediaquick-p5 qtmultimedia5-examples qtmultimedia5-doc-html

   2. Create the file "/usr/share/applications/demo-player.desktop" with the following content:

   #!/usr/bin/env xdg-open
   [Desktop Entry]
   Categories=Qt;KDE;AudioVideo;Player;Video;
   Comment=A gstreamer base player
   Exec=env QT_GSTREAMER_WIDGET_VIDEOSINK=rkximagesink /usr/lib/arm-linux-gnueabihf/qt5/examples/multimediawidgets/player/player --geometry 960x640+0+0
   GenericName=Media Player
   Icon=/usr/share/icons/gnome/48x48/categories/applications-multimedia.png
   Name=Media Player
   Type=Application
   X-KDE-StartupNotify=false
   Keywords=movie;player;media;kde;qt;
   X-Ayatana-Desktop-Shortcuts=Screen;Window

   Now you will see an entry "Media Player" in the Xfce menu, and you can also choose to play video files with this player by right-clicking on them and choosing "Open with..."

 

8. Enable acceleration for Chromium

For last, we are going to enable acceleration for Chromium Web Browser

1. Backup the file "/etc/chromium-browser/default", and modify it to look like this:

   # Default settings for chromium-browser. This file is sourced by /bin/sh from
   # /usr/bin/chromium-browser
   
   # Options to pass to chromium-browser
   # A set of command line flags that we want to set by default.
   
   # Do not hide any extensions in the about:extensions dialog
   export CHROMIUM_FLAGS="$CHROMIUM_FLAGS --show-component-extension-options"
   
   # Don't use the GPU blacklist (bug #802933)
   export CHROMIUM_FLAGS="$CHROMIUM_FLAGS --ignore-gpu-blacklist"
   
   # Use EGL acceleration
   export CHROMIUM_FLAGS="$CHROMIUM_FLAGS --use-gl=egl"

    

2. Launch Chrome, and enter in the address bar "chrome://gpu". If everything is OK, you should see something like this:

3. 

4. Now, go to the Chrome Web Store and install the extension "h264ify". It will force youtube to use H264 instead of VP9, which has no acceleration with RK3288.

5. Enter Youtube and play any 1080p video. It should play fairly well, dropping frames only when some kind of overlay pops up (like an ad on top of the video, or the controls overlay).

6. Test WebGL, for example with this page: http://webglsamples.org/aquarium/aquarium.html

 

9. NEW: Testing OpenCL with a GPU cryptocurrency miner

1. Download either of these compiled GPU miners (each of them features diferent algorithms): https://goo.gl/gHrPGg / https://goo.gl/MTSgBG
2. Untar them, cd into the directory and run them (in case you get an error, try doing "sudo make install"). E.g.:

   ./sgminer -k lyra2rev2 -o stratum+tcp://<url of pool>:<port> -u <your username> -p d=0.050 -I 14 -w 64

    

3. If you want to learn more, you can follow this tutorial: https://www.novaspirit.com/2017/12/21/gpu-mining-on-tinkerboard/

Mining with the skein algorithm:

 

 

Please report your experiences and suggestions. Wayland testing stays in the to-do list. Thanks,.

 

[pipozovato]

Hi, 

Great, I just posted a topic (1 hour ago) about slow video on Tinkerboard, maybe it's gona help me!

I'm just a beginer on Armbian and linux. I'm not sure I will do it well...

It's seem that we need a key to download the compressed file on https://mega.nz/#!UuRThALT ???

 

[JMCC]

3 hours ago, pipozovato said:

It's seem that we need a key to download the compressed file on https://mega.nz/#!UuRThALT ???

Yes, sorry, I corrected the link in the post. Should be this: https://mega.nz/#!UuRThALT!FcgA72sKKk179giSYzPooJQy6SbDYFpy4NJ8Gkp95Xc

[TonyMac32]

Good, I haven't had time to try it yet, on the list for today.  Then figuring out how far I can integrate it.  I'm going over the documentation to see, I know there are extra packages installed for sun*i mali, I'd ideally want to do that.  @zador.blood.stained and @Igor are the wizards that maintain the build system itself, I only work within it on board support, and haven't had much experience with the more advanced parts.  I will at least get the permissions sorted today so the level 0 instruction is unnecessary.  

[Tido]

While this is quite a long walk through, I wonder where @Myy is with its development for:

GPU driver

https://forum.armbian.com/topic/5416-gpu-driver/

 

VPU driver

https://forum.armbian.com/topic/4806-the-vpu-driver/

 

[TonyMac32]

Good question, I have quite lost track of that work in the last few weeks.  That is, of course for mainline, but will be relevant to this project.

[TonyMac32]

On 2/16/2018 at 6:25 PM, JMCC said:

Make a backup of the file "/etc/X11/xorg.conf.d/01-armbian-defaults.conf", and replace its content with the following:

 

I'm getting "Section" is not a valid keyword in this section"

[JMCC]

46 minutes ago, TonyMac32 said:

I'm getting "Section" is not a valid keyword in this section"

Mmmm, I just copied and pasted all the code into a blank file, and works for me. I'm doing it directly in the Tinkerboard, and using Vim. Maybe your text editor is messing line breaks or quotes, particularly if you are SSH'ing from Windows.

[TonyMac32]

I'll start over with a clean file and see.

 

@JMCC that cleared it right up.  Very nice.  it gets quite upset about the touchscreen being attached though, I'll probably have to figure that out.  Perhaps a script the user can run to enable this stuff would be best

 

glmark2-es2 score was 54, it seemed to be locking fps to the screen refresh, webgl worked perfectly, youtube was smooth at 1080p even without hardware decode. 

[JMCC]

5 hours ago, TonyMac32 said:

that cleared it right up

Glad it worked.

 

5 hours ago, TonyMac32 said:

glmark2-es2 score was 54, it seemed to be locking fps to the screen refresh

Yes, if you disable vsync you can get over 200 fps. But it is useless since your screen is still refreshing at 60hz, and you lose all that smoothness in video playing.

 

5 hours ago, TonyMac32 said:

it gets quite upset about the touchscreen being attached though

Probably needs some specific entries in Xorg config file. Unfortunately, I don't have the touchscreen to test.

 

5 hours ago, TonyMac32 said:

Perhaps a script the user can run to enable this stuff would be best

Sure. Maybe adding a menu item to armbian-config in RK3288 boards.

 

 

Also, I think I know which changes would need to be made in order to make it work in RK3328 and RK3399. But I don't have any board with thise SoC's. I am not planning to buy any RK3328; maybe a RK3399, when they release one for the average user who doesn't need to attach three LVDS at a time.

[TonyMac32]

55 minutes ago, JMCC said:

when they release one for the average user who doesn't need to attach three LVDS at a time

yeah.

 

58 minutes ago, JMCC said:

Probably needs some specific entries in Xorg config file.

I'm sure, I can probably find some examples for that.  The permissions are in the build system now, so item 0 should be unneeded.

[pipozovato]

On 17/02/2018 at 4:11 PM, JMCC said:

Yes, sorry, I corrected the link in the post. Should be this: https://mega.nz/#!UuRThALT!FcgA72sKKk179giSYzPooJQy6SbDYFpy4NJ8Gkp95Xc

Hi many thank for all.

I was out for work, I install all libs, I restart and now the system is asking me for the tinkerboard login and a password...

I type them and I got the prompt and I stay on command line, the desktop is not starting .... :-(

I'm sure I missed à step, I will reinstall the actual image and try another time, but as I told you I'm a beginner in Armbian, so I think I'm going to wait the new image.

If you can tell me when the new image will be ready, I will be very kind!

Regards 

:-) Pietro

 

 

[JMCC]

1 hour ago, pipozovato said:

Hi many thank for all.

I was out for work, I install all libs, I restart and now the system is asking me for the tinkerboard login and a password...

I type them and I got the prompt and I stay on command line, the desktop is not starting .... :-(

I'm sure I missed à step, I will reinstall the actual image and try another time, but as I told you I'm a beginner in Armbian, so I think I'm going to wait the new image.

If you can tell me when the new image will be ready, I will be very kind!

Regards 

:-) Pietro

 

 

Hello Pietro,

What is happening to you is normal. The tutorial does not tell you to reboot at that time, because if you do so, X will not start. That is because the old 1.8 Xserver bundled with ubuntu will crash while trying to start glamor. You only need to do the following step (number 3.), and everything will work fine.

[jkljkl1197]

I can't enable the graphic aceleration don't know why it's not working. I want it for Webrtc natively on chromium to get better performance.
http://ix.io/O51

Thx guy's

maybe https://github.com/rockchip-linux/gstreamer-rockchip can help?

[TonyMac32]

11 minutes ago, jkljkl1197 said:

I can't enable the graphic aceleration don't know why it's not working. I want it for Webrtc natively on chromium to get netter performance.
http://ix.io/O51

Thx guy's

maybe https://github.com/rockchip-linux/gstreamer-rockchip can help?

your kernel: Linux version 4.4.112-rockchip (root@xeon) (gcc version 5.5.0 (Linaro GCC 5.5-2017.10) ) #7 SMP Thu Jan 25 01:04:58 CET 2018

 

Did you build this one?  And from which source?  the kernel config could be the problem.  I followed these instructions using my uploaded kernel packages with no issue.

 

I also see we need to go in and kill off that excessive debug message in this kernel on the WiFi as well...

[jkljkl1197]

Where i can find a good img file to get work, graphic acceleration, Webrtc natively on firefox or chromium, generic camera (microsoft HD) on a working kernel where a display adapter hdmi -> vga work, because i don't know why but on some image i can't display nothing no console noting the screen show to me "67.5Khz/60Hz" (same for vty1-2...) to solve this, I need to edit the xorg config to get it work, but without the ip as been set to it, I cannot acces console to edit the file.

[JMCC]

@jkljkl1197 Sorry, if you told us which board you have, I can't find the post where you said it. I assume it is a Tinker Board.  Well, in that case, I think you have two options: Armbian and the official TinkerOS Debian.

 

About Armbian, if you are not able to build an image yourself with the new kernel, you can either download the old image and then install the packages referenced in the OP of this thread (including the kernel packages, from another thread linked also there), or you can also wait until the new image is released.

 

About TinkerOS, it will give you some of those things already configured without you needing to do so. But Armbian is more stable, has slightly better performance, and several tweaks and scripts that you won't find in TinkerOS.

 

I recommend you to try both, and see which one suits your needs better. This is what the world of FOSS and SBC's is about: experimenting, trying, learning, tinkering... ;-)

[jkljkl1197]

Thx @JMCC , I compile a new image in Hyper-V now.
Ubuntu is better, about support for graphic and all drivers or i compile debian insted ?
and it's a tinkerboard.

[JMCC]

9 hours ago, jkljkl1197 said:

Ubuntu is better, about support for graphic and all drivers or i compile debian insted ?

As of now, Debian Stretch image is still under development (though very functional already). So I would recommend you to build Ubuntu. Maybe when Stretch image is released officially, it will be better to use it for the Tinkerboard, since all the Rockchip-linux packages are built against Stretch. But for now, I recommend Ubuntu.

[pipozovato]

On 19/02/2018 at 7:15 PM, JMCC said:

Hello Pietro,

What is happening to you is normal. The tutorial does not tell you to reboot at that time, because if you do so, X will not start. That is because the old 1.8 Xserver bundled with ubuntu will crash while trying to start glamor. You only need to do the following step (number 3.), and everything will work fine.

Hi Again! away for business during few days :-)

 

So I restart from the beginning, new image Armbian_5.38_Tinkerboard_Ubuntu_xenial_next_4.14.14_desktop_Mainline, 
Then sudo dpkg-reconfigure keyboard-configuration to configure french keyboard.
$ sudo apt install xserver-xorg-hwe-16.04 xserver-xorg-video-fbdev-hwe-16.04

I install all files under libs folder, then disable Display compositing, install all files under xserver folder, backup of the 01-armbian-defaults.conf file insert the new one.
When I type restart X, I go an error : 
This program 'restart' is curently not installed. you can install it by typing : apt install upstart
I do apt install upstart
after I type restart X ... error :
Restart: unable to connect to upstart: failed to connect to socket /com/unbuntu/upstart: connection refused.

I surely did an error but ..... where I don't know :-)

So I restart the board and ...
Unbuntu 16.04.3 lts Tinkerboard tty1
The system asked me my user name and code, and I'm still on terminal ...

If you understand whats append. Maybe my image is corrupted? Do I need to download it again ?

 

:-) Pietro

 

[pro777]

On 17.02.2018 at 2:25 AM, JMCC said:

With the new default kernel based on rockchip-linux, boards with RK3288 SoC have now full kernel support for interacting with the GPU and hardware video decoder. Rockchip-linux also provides libraries and userspace software to make everything work. 

I fulfilled 4 tutorial item. Unfortunately, there was a startup error glmark2-es2:

Spoiler

user@armbian:~$ glmark2-es2
ERROR: The DDK is not compatible with any of the Mali GPUs on the system.
The DDK was built for 0x750 r0p0 status range [1..1], but none of the GPUs matched:
Error: eglInitialize() failed with error: 0x3001
ERROR: The DDK is not compatible with any of the Mali GPUs on the system.
The DDK was built for 0x750 r0p0 status range [1..1], but none of the GPUs matched:
Error: eglInitialize() failed with error: 0x3001
Error: main: Could not initialize canvas

Libmali packages are installed:

Spoiler

user@armbian:~$ aptitude search libmali
i   libmali-rk-dev                                   - The mali library for Rockchip RK3288.                     
i   libmali-rk-midgard-t76x-r14p0-r0p0               - The mali library for Rockchip RK3288(not included RK3288W)

user@armbian:~$ aptitude show libmali-rk-midgard-t76x-r14p0-r0p0
Package: libmali-rk-midgard-t76x-r14p0-r0p0
State: installed
Automatically installed: no
Multi-Arch: same
Version: 1.6-1
Priority: optional
Section: libs
Maintainer: Randy Li <randy.li@rock-chips.com>
Architecture: armhf
Uncompressed Size: 17.6 M
Depends: libc6 (>= 2.15), libdrm2 (>= 2.3.1), libgcc1 (>= 1:3.5), libstdc++6 (>= 4.8), libx11-6, libx11-xcb1,
         libxcb-dri2-0 (>= 1.8), libxcb1
Replaces: libegl1-mesa, libegl1-x11, libgbm1, libgles2-mesa
Provides: libegl1, libegl1-mesa, libgbm1, libgles2-mesa, libwayland-egl1-mesa
Description: The mali library for Rockchip RK3288(not included RK3288W).

user@armbian:~$ dmesg | grep mali
[    2.150304] mali ffa30000.gpu: Failed to get regulator
[    2.156148] mali ffa30000.gpu: Power control initialization failed
[    3.750212] W : [File] : drivers/gpu/arm/midgard/platform/rk/mali_kbase_config_rk.c; [Line] : 113; [Func] : kbase_platform_rk_init(); power-off-delay-ms not available.
[    3.750410] mali ffa30000.gpu: GPU identified as 0x0750 r0p0 status 1
[    3.750833] I : [File] : drivers/gpu/arm/midgard/backend/gpu/mali_kbase_devfreq.c; [Line] : 284; [Func] : kbase_devfreq_init(); success initing power_model_simple.
[    3.751071] mali ffa30000.gpu: Probed as mali0
[    4.574880] I : [File] : drivers/gpu/arm/mali400/mali/linux/mali_kernel_linux.c; [Line] : 415; [Func] : mali_module_init(); svn_rev_string_from_arm of this mali_ko is '-7b43537', rk_ko_ver is '5', built at '21:01:40', on 'Feb  1 2018'.

What can be wrong?

[pro777]

26 minutes ago, pro777 said:

I fulfilled 4 tutorial item. Unfortunately, there was a startup error glmark2-es2:

  Reveal hidden contents

user@armbian:~$ glmark2-es2
ERROR: The DDK is not compatible with any of the Mali GPUs on the system.
The DDK was built for 0x750 r0p0 status range [1..1], but none of the GPUs matched:
Error: eglInitialize() failed with error: 0x3001
ERROR: The DDK is not compatible with any of the Mali GPUs on the system.
The DDK was built for 0x750 r0p0 status range [1..1], but none of the GPUs matched:
Error: eglInitialize() failed with error: 0x3001
Error: main: Could not initialize canvas

 

 

Wow, I run the command:
sudo chmod 666 /dev/mali0.
Everything worked!
However, after reboot, the device's rights are returned. How is it better to solve the problem?

[JMCC]

@pipozovato You downloaded the wrong image. It should be https://dl.armbian.com/tinkerboard/Ubuntu_xenial_default_desktop.7z, with 4.4 kernel. And, until the new image is released, you must also install the testing packages with the updated 4.4 kernel, from the post referenced at the beginning.

 

@pro777 When the new image is released, permissions will be set properly by default. Until then, you can put in /etc/rc.local these three lines:

sudo chmod 666 /dev/hevc-service
sudo chmod 666 /dev/mali0
sudo chmod 666 /dev/media*

You guys must remember that all these features are still under testing. The stable image won't need all the fixes.

[JMCC]

Updated to include also Open CL testing with a GPU miner. No, guys, you're not going to get rich with your little Tinker, but it is funny to see her join all those mining rigs and ASICs at the mining pools.

[jkljkl1197]

it's working for me too nice job !!

[TonyMac32]

Official image now includes the proper permissions and kernel.

[bedalus]

I bought myself a wired USB keyboard so I could try this tutorial. I've got the new March 4.4 kernel xenial desktop image, and I followed the instructions. I got to the point where you restart x, so I issued

sudo systemctl restart nodm.service

and got booted back to the full-screen root terminal. Is this the wrong command? I issued 'reboot' and got a black screen, then a full power cycle, and got a black screen. Any ideas?

 

Ninja edit: I just cycled my TV through it's HDMI modes, and the picture is back. Not sure what the difference between mode 1 and mode 2 on the TV is, but I now have a signal. Is there supposed to be a 4k setting in the xorg conf file, e.g 3840 x 2160 or 4096 x 2160?

[JMCC]

54 minutes ago, bedalus said:

Is there supposed to be a 4k setting in the xorg conf file, e.g 3840 x 2160 or 4096 x 2160?

I can't answer, since I don't have a 4k display to test with. You can add the mode to the xorg.conf file, and let us know what happens.

[bedalus]

I don't think I can help diagnose the problem. After I changed the TV mode, the picture was there, and the display options recognised the TV model and provided the option for 3840x2160. I selected that and the resolution worked fine. I've now added that to the xorg conf and rebooted. The desktop appeared in the new resolution, but I'm not sure which thing I did that fixed it. Sorry, this was pretty useless for debugging the problem.

 

I've noticed that in this this resolution, when I drag windows around they leave trails and lag behind the mouse. Is this normal due to the limited ability of the chip?

[TonyMac32]

53 minutes ago, bedalus said:

Sorry, this was pretty useless for debugging the problem.

 

I am all too familiar with this kind of magic.  ;-)  My 4k Samsung monitor defaulted immediately to 4k, the only issue I ever had was it crashing when trying to change modes.  Rockchip appears to have fixed that, as the issue is now gone.