Tutorial: 3D, video acceleration and OpenCL in RK3288 boards with new 4.4 (default) kernel

[TonyMac32]

well, I'll give it another go, at the moment I'm disposing of some RPi hardware by making a jukebox for my 5 year old.

[JMCC]

I'm working on a script to install all the stuff at once, but I'm not sure when it will be finished. In the meantime,in case anyone wants to try MPV with GBM (dsiplay) and RKMPP (decoding) acceleration, here are the instructions:

1. After installing the base libs, configs and MPV according to the first post, download this package and install it.
2. From then, you can launch mpv like this, in order to have full acceleration:

LD_LIBRARY_PATH=/opt/libmali-gbm:$LD_LIBRARY_PATH mpv --hwdec=rkmpp --vo=gpu --gpu-api=opengl --gpu-context=drm <file>

The deb package above will install the GBM version of libmali. I packaged it to install under /opt, so it did not interfere with system libs and we can still have X11 acceleration with the libmali-X11 version.

 

MPV can still be run normally, through X11-EGL, with the known performance limitations (see first post). But if you launch it with the command above, it will completely ignore the X server, and use directly GBM/KMS. That means it will only run in fullscreen, with no mouse support. You can control it with the keyboard and (I suppose) with a remote control or HDMI CEC. MPV on-screen display works normally,  so when you press the appropriate keys info is displayed on the screen (see MPV manpage for a list of keyboard controls). When you press "q" or "Q", it will exit and you will have your X11 session back as you left it.

 

In MPV's implementation of RKMPP, it is bound to GBM display, so the only way to use hardware video decoding is with this launcher. Performance is outstanding: Silk-smooth 4K playback, with minimal CPU usage.

[pro777]

Hi, JMCC,

When playing video files with the MPV in Armbian, as mentioned above in your message, often there is a video playback error:

user@armbian:~$ LD_LIBRARY_PATH=/opt/libmali-gbm:$LD_LIBRARY_PATH mpv --hwdec=rkmpp --vo=gpu --gpu-api=opengl --gpu-context=drm /media/user/1803-416A/HEVC/Затерянные\ в\ космосе.1998.HEVC.mkv
Playing: /media/user/1803-416A/HEVC/Затерянные в космосе.1998.HEVC.mkv
 (+) Video --vid=1 (*) 'HEVC' (hevc 1920x800 23.976fps)
 (+) Audio --aid=1 --alang=rus (*) 'Kinomanija' (ac3 6ch 48000Hz)
     Audio --aid=2 --alang=rus 'Dub' (ac3 2ch 48000Hz)
     Audio --aid=3 --alang=rus 'R5' (ac3 6ch 48000Hz)
     Audio --aid=4 --alang=eng 'Original' (ac3 6ch 48000Hz)
     Subs  --sid=1 --slang=rus 'Full' (subrip)
     Subs  --sid=2 --slang=eng 'Full' (subrip)
File tags:
 Title: Lost in Space
[vo/gpu] VT_GETMODE failed: Inappropriate ioctl for device
[vo/gpu/opengl] Failed to set up VT switcher. Terminal switching will be unavailable.
mpi: mpp version: 598cae3 author: Jacob Chen DEBIAN: update rules for release_20171218-2
mpp_rt: NOT found ion allocator
mpp_rt: found drm allocator
H265D_PARSER: No start code is found.
Using hardware decoding (rkmpp).
AO: [pulse] 48000Hz 5.1(side) 6ch float
VO: [gpu] 1920x800 drm_prime
[vo/gpu] Using HW-overlay mode. No GL filtering is performed on the video!
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 02:10:14 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 02:10:14 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 02:10:14 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 02:10:14 (0%) A-V:  0.000

Exiting... (Quit)

Instead of the video image - an empty black screen, although there is sound.

I installed ffmpeg and mpv deb-packages and you collected the package GBM version of libmali in Debian firmware from Rockchip - there are no such problems, everything is beautifully played. I thought maybe Ffmpeg was assembled in Debian and Armbian Ubuntu with different keys. Checked, the difference, seemingly insignificant, ffmpeg in Debian compiled with additional keys:

 --enable-librsvg --enable-libxml2 --cross-prefix=arm-linux-gnueabihf- --arch=armhf --target-os=linux

I also checked that the libraries libsdl2, libxml2 and librsvg in Armbian are installed.

Any idea what might be the problem?

[JMCC]

5 hours ago, pro777 said:

Any idea what might be the problem?

This tutorial is still here for educational purposes only. I created a script that will automatically install and configure everything. It is meant to be installed in a fresh Armbian image, either stable or development. You can download it here:

Now, you mention that you have a video playback error "often". So it means that it does not always happen, right? There is a reason for it.

 

First, let me explain briefly the difference between the approach you can read in Rockchip's repo and ours. According to their docs (https://github.com/rockchip-linux/rk-rootfs-build/tree/master/packages/armhf/others), they tell you to download the GBM libmali and replace with it the system one (which is the X11 version). Then they tell you to stop the X server, and run MPV. It will just use the system libmali, and display video thorugh GBM.  But you won't be able to use X11 acceleration until you restore the original X11 libmali.

 

We, instead, install the GBM libmali under /opt, and therefore there is no need to get rid of the system X11 libmali: both can be installed at the same time. But I am discovering now that, if both are not only installed, but also used at the same time, you get conflicts between them, taking over some resources and making it impossible for the other version of the library to use them. That is why you experience these black screens only sometimes: it happens only when you have done something that caused the X11 library to take possession of the resources the GBM library needs, or vice versa.

 

So far, I haven't been able to debug which are those resources causing the conflict, nor if there is a way to have the libraries release them. In the meantime, I recommend you two possible workarounds:

1. When you hit any of those problems, reboot and the resource will be free, so you will be able to play the video.
2. Stop the X server before using a GBM accelerated player. You do it with these steps:
   1. Change to a console virtual terminal, e.g. by pressing Ctrl+Alt+F1
   2. Log in
   3. Issue "sudo service nodm stop" or "sudo service lightdm stop", depending on the DM you are using

Please follow up in any new finding you do about this issue.

[TonyMac32]

@Myy I stumbled across this today:  https://openedev.amarulasolutions.com/display/ODKERN/VCODEC+on+RK3288 

[Myy]

I wonder who put so much logs in the VPU code ? It's just unreadable ! ...

 

Jokes aside, if you see "init failed" in the logs, this means that the VPU driver could not load so... Yeah playing videos will be done with the CPU.

 

I'm currently trying to understand the different ways to make user-space memory usable with DMA operations. Technically the hardware have no idea of what is "user-memory" (as demonstrated by various Firewire/USB DMA security bypass issues on Linux/Mac/Windows), so it's more about kernel securities and IOMMU configuration.

 

That aside, I was included in a recent discussion with the Chromium developers and the Rockchip developers and it seems that the Chromium ones are using the v4l2 driver instead, which is more cleanly documented.
However the V4L2 API changed during the 4.x era. Most of the video formats related defines and structures were moved to the user-space library, so the driver can't be ported "as-is". Also, this driver lacks some formats.

Still, it's much cleaner and easier to understand : https://github.com/rockchip-linux/kernel/tree/release-4.4/drivers/media/platform/rockchip-vpu

[JMCC]

5 hours ago, Myy said:

were moved to the user-space library

Does this user-space library relate in some way to the deprecated rockchip-va library? (http://rockchip.wikidot.com/rockchip-va-driver)

[Myy]

It seems to be that one. I don't know why they chose to build their MPP thingy instead, though.

 

That said, up to some 4.x version of the kernel, the V4L2 kernel API had access to a lot of structures and macros that related to video formats (H264, H265, VP8, ...). These structures and macros has been moved the V4L2 user API afterwards, making such drivers a bit difficult to port.

 

[Aaron]

thank you so much @JMCC

 

Have been trying to get this to work for a few days.  Now I have sgminer recognising OpenCL on my tinkerboard! (shame about lack of cryptonight support on ARM).

 

Running 16.04 xenial armbian build: Armbian_5.41_Tinkerboard_Ubuntu_xenial_default_4.4.119_desktop.7z

 

Just hope there's a currency this thing will still mine.

 

Once again, thank you for putting all the time into providing this tutorial for us.  We'd never get there without people like you!!

 

Regards,

 

Aaron.