Jump to content

RK3399 Legacy Multimedia Framework

JMCC

By JMCC
in Rockchip

 Share

Recommended Posts

JMCC

JMCC

Posted
Posted

IT'S FINALLY HERE...

background-business-design-star-internet-concept-1626698-pxhere_com.jpg.719bd6b4d02c30003ba20713d391aede.jpg

THE OFFICIAL ROCKCHIP-LEGACY MULTIMEDIA INTEGRATION

 

After two years of using a separate script to enable the multimedia features in RK3399 Legacy Kernel, the whole framework has been incorporated to the official Armbian packaging system. The choice distro for this integration is Debian Buster (see FAQ at the end of this post about the reasons).

 

I. Installation

  1. Download a Armbian Buster Legacy Desktop image for your board, and install it with the standard Armbian method.
  2. Install the complete multimedia solution with 
    sudo apt update && sudo apt upgrade

sudo apt install media-buster-legacy-rk3399 --install-recommends

    The switch "--install-recommends" will add the whole Kodi binary addons collection (retro-gaming cores, music visualizations, screensavers, additional media decoders/encoders, vfs, etc.), plus the GLES-to-OpenGL wrapper "gl4es".

 

II. Features

  1. Accelerated GLES/EGL X desktop: No action needed.
  2. Accelerated Chromium, with WebGL and video display acceleration: No action needed
  3. Desktop video player capable of smooth 4K HEVC-HDR: Use the "Rockchip Gst Player" from the Multimedia menu, or choose it with right-click on the media file. Command-line 4K playing is also possible with "gst-play-1.0 --videosink=kmssink".

  4. RKMPP-accelerated MPV: Use normally for standard operation (windowed with mouse-operated GUI). For fullscreen, keyboard-operated mode, use the command line switch "--gpu-context=drm" (this will allow you to play smooth 4K).
            - See instructions below, in the next post, for playing YouTube videos up to 4K with this MPV.

  5. ISP Camera with real-time h.264/1080p HW encoding: Using the Gstreamer Plugin. Check this wiki for instructions on how to use it. Most of it applies to Armbian, except for the selection of ov5647/imx219 camera, which must be done using DT overlays.
  6. OpenCL 1.2 support: It will be fully functional, no further action needed. You can download some tests and examples from this link.
  7. Kodi 18.9 Leia with full RKMPP+GBM acceleration, 4K-HDR capable: You can start it from LightDM menu as your user account: 
    kodi-gbm-info.jpg.271beb6d036378968e74faa1f841dcbd.jpg

    Alternatively, you can also run it as a system service with these command lines: 
    sudo systemctl disable lightdm
sudo systemctl enable kodi-gbm
sudo reboot
  8. Full collection of Kodi binary add-ons: Includes retrogaming cores, media encoders and decoders, PVR, screensavers, vfs and audio visualizations. They are all installed with the package "kodi-addons-full", but are disabled by default. They need to be enabled individually within the Kodi GUI.
  9. OpenGL 2.1 support through the gl4es wrapper: It is installed with the package "gl4es", with no further action needed.

 

III. Sources

This is the list of the sources used for the packages:

  Reveal hidden contents

 

IV. FAQ

  • ¿Why did you use Debian Buster as a base for this implementation?
    It was the most appropriate for several reasons. Upstream Rockchip-Linux developers use Debian buster, so the software could be ported with less modifications than if we chose a different distro. Besides, it is a completely stable distro, unlike Bullseye, which is a moving target as of today. It also has Chromium as a package, unlike Focal that uses snap instead. For last, it has a good backports repo, with several libs that would otherwise need to be compiled and maintained if we chose, for example, Focal.
     
  • ¿Why Legacy instead of Mainline?
    This is an implementation based on the vendor's BSP kernel. It has been tested and is reliable, which many people will prefer rather than having a bleeding-edge, less stable implementation. In addition to that, Mainline upstream multimedia support is still a WIP, and lacks many features that are only present on Legacy kernels.
     
  • ¿Will you add new features to this implementation?
    No, this implementation will only receive bug fixes if necessary. From now on, all multimedia work will be focused on Mainline and recent distros (like Focal or Bullseye). All new features will go there.
11
Werner

Werner

Posted
Posted

Haha if I knew that you were working on this I could have saved three hours earlier today :lol::lol:

 

Nice job. Will feed this into the Station P1 soon.

1
TRS-80

TRS-80

Posted
Posted (edited)

Those masks though...  Why JMCC, just why?!  :D

 

  On 12/17/2020 at 3:04 PM, JMCC said:

(Reserved for future use)

Expand  

 

Why do I feel like I am at XDA Developers forum?  :D

 

All jokes aside, this is great news!  Even though I am not desktop user[0], it's exciting to see all this desktop progress happening lately.  Thanks for all your work @JMCC!  As you say I know this has been years in the making.

 

:thumbup:  :beer:

 

[0] I may become one soon, because of things like this and general state of desktop in Armbian.

Edited by TRS-80
correct info
0
alsc

alsc

Posted
Posted

Thank you very much for your work. I've been following your other thread once your tool's been one of the only ways I found to use Kodi on a RockPro64 in a server, without have to necessarily use LibreELEC.

 

I really appreciate what you've been doing!

1
MotoFckr9k

MotoFckr9k

Posted
Posted
  On 12/17/2020 at 3:02 PM, JMCC said:
III. Sources

This is the list of the sources used for the packages:

  Reveal hidden contents

 

Expand  

 

Any reason you used teacpux's fork instead of https://github.com/paolosabatino/xf86-video-armsoc which is newer in terms of commits?

0
JMCC

JMCC

Posted
  • Author
Posted
  On 12/18/2020 at 12:49 PM, MotoFckr9k said:

Any reason you used teacpux's fork instead of https://github.com/paolosabatino/xf86-video-armsoc which is newer in terms of commits?

Expand  

Just to clarify: teacupx is my Github nickname.

 

I examined paolosabatino's commits, and the only one affecting Rockchip that is not present in my fork, is forcing SW cursor. Since I have not experienced any instability due to using HW cursor in my integration, and on the other hand SW cursor introduces an annoying blink when using accelerated EGL layers, I chose to stay with my fork using HW cursor.

 

@jock Any insights on this?

1
jock

jock

Posted
Posted

Hello, my armsoc fork has some bits here and there to allow working fullscreen acceleration with Mali proprietary drivers on mainline kernel using dumb buffers instead of private Ioctls for rockchip, allwinner and amlogic.

On rockchip it works fine also with the legacy 4.4 kernel too.

Hardware cursor works fine, just switch to HWCURSOR_API_PLANE the attribute here

 

The reason why it is currently set to software cursor is to allow the cursor work on rk322x.

rk322x has only two hardware planes and one plane is forcefully used as primary, so just one hardware plane is left for use.

Current rk322x-legacy armbian variant has a patch that allocates the spare hardware plane as overlay, so if I set armsoc to use hardware cursor, the cursor just disappears because there is no hardware cursor plane!

 

A smarter logic in armsoc would be to run over all the DRM planes looking for the hardware cursor one and use it. Otherwise, in case no hardware cursor plane is present, just use the software cursor.

 

To say it all, almost all rockchip SoCs (including rk3288, rk3339, rk3328 and rk322x) have a special dedicated hardware plane capable of up to 128x128 pixels just to be used for the cursor, so you don't have to sacrifice a general purpose hardware plane. I made an incomplete WIP patch for and it partially works, but has issues that cannot be fixed without changing some surrounding code :(

1
NicoD

NicoD

Posted
Posted

I just tried this on M4V2 with Buster legacy, and it broke dpkg. 

E: dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem.

"sudo dpkg --configure -a" restarts the desktop. But doesn't fix dpkg. Also not with "dpkg --configure install

Boy I'm having a bad time. Nothing seems to work the last days.
I had to download a Buster Legacy backup from June to have a working system. Now this. 
But that's not all, no usb drives are able to mount on the M4V2. Not with Focal neither with Buster. Works on the Station P1. 
 

 

  On 12/17/2020 at 3:02 PM, JMCC said:

Besides, it is a completely stable distro,

Expand  

:wacko:
I wanted to make a video about it. But there are clearly way too much problems. Downloadable Buster Legacy image doesn't boot, dpkg breaks when installing media tools, usb drives don't work.
I don't get why/how this all happens. I only know my head hurts. I guess it's bedtime :) 

0
JMCC

JMCC

Posted
  • Author
Posted

I see. I normally do install and upgrades "the safe way", from console or ssh. But many people will do it from the desktop. I will release a fix for that and other bugs, G.w.

Enviado desde mi moto g(6) plus mediante Tapatalk

2
NicoD

NicoD

Posted
Posted
  On 12/19/2020 at 5:28 AM, Werner said:

Same but fix is easy. Run the script from console, not from terminal.

Expand  

 

  On 12/19/2020 at 7:13 AM, JMCC said:

I see. I normally do install and upgrades "the safe way"

Expand  

Works now. Boy I'm happy. I rarely use console. Never had issue's with the terminal. Only when I've got crashes I use console to reboot. 

Now I'm going to relax and watch a movie with this :) 
Already done something else today, sandpapered the contact surface of the heatsink of the RockPiX. It was bubbled and painted black. Overheated to 80C and throttle above 75C. Blender was 40m

Now with sandpapered heatsink 60C max no throttle and blender in less than 35m. Now I start my M4V2 and easy fix. 
This is a way better day than yesterday :) Thank you!

2
NicoD

NicoD

Posted
Posted

@JMCC Video playback with Chromium doesn't seem to be VPU accelerated. 
It does show it is, but it's not. 

Graphics Feature Status
Canvas: Hardware accelerated
Flash: Hardware accelerated
Flash Stage3D: Hardware accelerated
Flash Stage3D Baseline profile: Hardware accelerated
Compositing: Hardware accelerated
Multiple Raster Threads: Force enabled
Out-of-process Rasterization: Disabled
OpenGL: Enabled
Hardware Protected Video Decode: Hardware accelerated
Rasterization: Software only. Hardware acceleration disabled
Skia Renderer: Enabled
Video Decode: Hardware accelerated
Vulkan: Disabled
WebGL: Hardware accelerated
WebGL2: Hardware accelerated

Even 720p25fps doesn't play well fullscreen. While in Bionic with the media script I can watch 1080p30fps without frameloss. image.png.38df881ee894b5f21e938b872427b109.png

Video playback with gst is perfect.

0
JMCC

JMCC

Posted
  • Author
Posted
  On 12/20/2020 at 1:03 AM, NicoD said:

Video playback with Chromium doesn't seem to be VPU accelerated.

Expand  

Well, it wasn't either VPU accelerated in Bionic, just GPU accelerated -- that is, the video display layer is GPU accelerated, but the frame decoding is done via software.

 

IIRC, the Bionic script had the extension "h264ify" enabled by default, which I could not do in this version of Buster. Just search the Chrome App Store for "h264ify", install it and it should make the difference (h264 is less cpu-demanding than vp9).

 

  On 12/19/2020 at 11:52 PM, NicoD said:

Now I'm going to relax and watch a movie with this

Expand  

You can also enable the ScummVM addon in Kodi, and play Monkey Island ;)

1
NicoD

NicoD

Posted
Posted
  On 12/20/2020 at 1:15 AM, JMCC said:

Just search the Chrome App Store for "h264ify", install it and it should make the difference (h264 is less cpu-demanding than vp9).

Expand  

Doesn't seem to make a difference.
I also tried youtube addons on Kodi, but am unable to login.

 

  On 12/20/2020 at 1:15 AM, JMCC said:

and play Monkey Island 

Expand  

That's a too hard game, makes me feel more stupid :) A friend of mine liked it. I never liked those kind of games.
Enough great DOS games that work on it. For me that's enough. Cheers.

PS: Firefox plays a lot better. Certainly with 1080p display res it plays 1080p30fps video great. I'll also try Vivaldi. That does best normally without acceleration.

0
Arvo

Arvo

Posted
Posted

Great job, JMMC! My M4V2 now is usable even in 3440x1440 resolution. :) It still forgets about it each time, when monitor is switched off, but at least moving of browser or Thunar window is much smother and faster as on legacy Bionic with script. Even PVR addon works in KODI, although VLC in this case is more preferable IPTV player.

Thanks, I hope dpkg will be easily solvable. Console workaround is ok, but old habits are hard to overcome :unsure:

0
JMCC

JMCC

Posted
  • Author
Posted

New packages uploaded to the repos (it may take some hours to synchronize).

  • Bug fixes:
    • X Desktop no longer restarted on package install, so it can be installed from a desktop session (NOTE: Since the old package is still going to restart X when uninstalled, it is recommended to do this update from console).
    • MPV now is not failing when it cannot create EGL surface (it will just default to standard x11).
    • CEC is now working on several boards (tested in RockPro64 and Rockpi 4B).
  • New features:
4
webbbn

webbbn

Posted
Posted

This is great! Unfortunately, I'm not having any luck so far.

 

I built my own image (I know, that's not the standard method, but I assumed it would be the same), and it doesn't have the media-buster-legacy-rk3399 package available. Next I tried downloading the standard M4V2 image (Armbian_20.11.3_Nanopim4v2_buster_legacy_4.4.213_desktop.img.xz), and it gets stuck on bluetooth while booting and get's stuck there.

 

I'm going to try the neo4 image now. I assume the Armbian_20.11.3_Nanopineo4_buster_legacy_4.4.213.img.xz image is a desktop image?

 

In any case, I'm very excided about this. I have gone through the process of building a few systems with the script, but would much prefer to have the capability in the standard image without having to go through the extra steps.

 

Is it possible to e.g. have MPV display directly on the framebuffer, accelerated with another app (OSD) running in another, overlying framebuffer on rk3399?

 

I need/want to be able to build my own image, so is there some repo I need to add to the standard build, or should that have worked with a normal build? I may have some build option set incorrectly.

 

Edit: The neo4 is getting stuck on starting bluetooth as well. Is there any way to skip it?

 

Got past the bluetooth error on startup. It apparently had gotten far enough to allow network logins, so I was able to login, update, and install the multimedia packages, and it's looking good now. I was able to smoothly play a 1080p video that I had lying around with mpv. Time to start playing!

 

 

Can you give a hint on how to enable the camera? I don't see any overlays that seem to relate to the camera. Is there another package that I have to install to install additional overlays?

 

I'm trying to get libmpv to work, and I keep getting the error "EGL_NOT_INITIALIZED". I even get that when I install and run qtcreator. Does that make sense to anyone?

1
JMCC

JMCC

Posted
  • Author
Posted
  On 12/22/2020 at 1:05 AM, webbbn said:

I need/want to be able to build my own image

Expand  

It is very easy: https://docs.armbian.com/Developer-Guide_Build-Preparation/#how-to-start

 

  On 12/22/2020 at 1:05 AM, webbbn said:

The neo4 is getting stuck on starting bluetooth as well

Expand  

If you build the image with the method above, everything should be fine. There is a bug in the images available for download, we are working on the fix.

 

  On 12/22/2020 at 2:50 AM, webbbn said:

Can you give a hint on how to enable the camera? I don't see any overlays

Expand  

You're right, the overlays are not available for your kernel. I will look into it, and post here when they are in place. In the meantime, you can also take them from Raxda repos.

 

  On 12/22/2020 at 5:23 AM, webbbn said:

I'm trying to get libmpv to work, and I keep getting the error "EGL_NOT_INITIALIZED".

Expand  

Which libmpv are you using? Please post the output of "glxinfo" and "es2gears".

 

Try to reinstall the mali library, and post the results: 

apt install --reinstall libmali-rk-midgard-t86x-r14p0-r0p0

 

0
Werner

Werner

Posted
Posted
  On 12/22/2020 at 8:13 AM, JMCC said:

Which libmpv are you using? Please post the output of "glxinfo" and "es2gears".

Expand  

I get the same errors on Station P1 after installation of the script. Before I had classic software render via LLVM.

 

  Reveal hidden contents

Reinstalling mali as suggested fixes this:

 

  Reveal hidden contents

 

0
webbbn

webbbn

Posted
Posted
  Quote

It is very easy: https://docs.armbian.com/Developer-Guide_Build-Preparation/#how-to-start

Expand  

Thanks. I build Armbian regularly, but, for some reason the image that I built did not have the media-buster-legacy-rk3399 package. I'll try again, and if it doesn't work I'll post my build options.

 

  Quote

You're right, the overlays are not available for your kernel. I will look into it, and post here when they are in place. In the meantime, you can also take them from Raxda repos.

Expand  

Thanks! I'll take a look at the repo.

 


 

  Quote

 

Which libmpv are you using? Please post the output of "glxinfo" and "es2gears".

 

Try to reinstall the mali library, and post the results: 

apt install --reinstall libmali-rk-midgard-t86x-r14p0-r0p0

 


 

Expand  

That did fix it for me. I had to reboot (logging out/in might have been enough), but it is working now.

0
Werner

Werner

Posted
Posted
  On 12/21/2020 at 10:42 AM, JMCC said:

New packages uploaded to the repos (it may take some hours to synchronize).

  • Bug fixes:
    • X Desktop no longer restarted on package install, so it can be installed from a desktop session (NOTE: Since the old package is still going to restart X when uninstalled, it is recommended to do this update from console).
    • MPV now is not failing when it cannot create EGL surface (it will just default to standard x11).
    • CEC is now working on several boards (tested in RockPro64 and Rockpi 4B).
  • New features:
Expand  

I have libcec updated, however media-buster-legacy-rk3399 is kept back. I assume the other necessary packages are not there yet?

0
JMCC

JMCC

Posted
  • Author
Posted

Bugfix release. For those of you upgrading, since the upgrade will install new packages, you need to issue the command like:

apt install media-buster-legacy-rk3399

instead of the regular "apt upgrade", which would retain the packages to avoid new installations.
 
This bugfix release should fix the problem of needing to reinstall the mali library: now it should work OK the first time.

0
Werner

Werner

Posted
Posted

dist-upgrade does not work either. I had to "force" by issuing apt-get install media-buster-legacy-rk3399

This also removed mpv and installed mpv classic.

1
webbbn

webbbn

Posted
Posted

Just to follow up. I was able to build an image. My problem previously was that my script was building a focal image, which, of course, would not work. A program of mine that uses libmpv is working very well.

 

I have not been able to get the camera working. I'm pretty sure that I had /dev/video* device entries previously, but now I don't have anything, so maybe the camera died for some reason. It could also be that I don't know what I'm doing. :-)

 

I wasn't able to a dts file that seemed like an exact fit for the camera, so that could be a problem as well. The camera that I have is this one: https://www.friendlyarm.com/index.php?route=product/product&path=78&product_id=247. It's not high priority, so, rather than thrash around some more, I'll just wait and see if anyone else gets it working.

0
madao

madao

Posted
Posted

Hi

I would like to set up a broadcast using jellyfin.

I installed it following the instructions. I configured hwaccel as instructed.

I don't know, maybe ffmpeg is not configured correctly, but decoding is only software. all cores are 100% loaded, the temperature is about 60, the video lags

0
JMCC

JMCC

Posted
  • Author
Posted
  On 12/24/2020 at 5:18 PM, madao said:

Hi

I would like to set up a broadcast using jellyfin.

I installed it following the instructions. I configured hwaccel as instructed.

I don't know, maybe ffmpeg is not configured correctly, but decoding is only software. all cores are 100% loaded, the temperature is about 60, the video lags

Expand  

Rockchips have no ffmpeg HW encoding support, only decoding. And Jellyfin won't support even your RK3399 HW decoding. The only ARM boards where you can enable HW support in Jellyfin are RPi and Odroid XU4 (and its derivated HC1/HC2).

0
madao

madao

Posted
Posted
  On 12/24/2020 at 5:28 PM, JMCC said:

Rockchips have no ffmpeg HW encoding support, only decoding. And Jellyfin won't support even your RK3399 HW decoding. The only ARM boards where you can enable HW support in Jellyfin are RPi and Odroid XU4 (and its derivated HC1/HC2).

Expand  

Jellyfin supports VAAPI decoding, isn't that the case for rk3399?

0

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
 Share
Go to topic listing
×
×
  • Create New...

Important Information

Terms of Use - Privacy Policy - Guidelines

  I accept