Myy

MPP/RKMPP is the RocKchip Media Process Platform.
A set of libraries, made by Rockchip, to communicate with their VPU driver. The thing is done in such a way that the "driver" basically only handle a few things like memory management.
The actual registers of the hardware are known by MPP and are setup by this library, then sent to the driver which almost blindly write the registers values into the hardware, or read them back and send them back to MPP.
Which mean that, even if you have the sources of the Rockchip VPU driver, you need the sources of MPP to understand how the hardware is actually programmed, based on the format you want to decode/encode.
This is the kind of setup which make you wonder, who's the real "driver" ?
http://opensource.rock-chips.com/wiki_Mpp
 
FFMPEG is one the most famous multimedia processing library and tool. This thing can combine audio/video from different sources and combine/convert them into a LOT of formats.
It comes as a library AND as a binary, which is one of the swiss-army knife for Audio-Video processing.
https://ffmpeg.org/
 
MPV is a Media Player, fork of Mplayer2, which use FFMPEG as a backend. It currently have a RKMPP backend to decode video frames using the RKMPP libraries.
https://mpv.io/
 
H264 is a video format.
https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC
 
The I-frames in H264 are reference (key) frames, from which other kind of frames (B/P frames) will be generated. The I-frame is basically the full frame, while the B/P frames are basically "patches" applied to I-frames to get the new picture.
The "patches" being generally smaller than the I frame, you get one way to "compress" the video (upon various others used simultaneously).
https://en.wikipedia.org/wiki/Inter_frame

After fighting to get FFMPEG compiled, MPV compiled (with Debian putting ffmpeg includes inside /usr/include/arm-what-ever-arch-hf/ and /usr/include/arm-what-ever-arch-hf/ having priority over /usr/include !) and then patching MPV to get the OpenGL display working correctly (stop using eglGetDisplay when you can use eglGetPlatformDisplayEXT !), I got the same error as you had...
 
And then I realized that the error states that the buffer allocation failed...
 
So I tried as root and it led to RKMPP failing to get the right frames in time. And then it crashed... badly.
 
I'll retry with Gstreamer, since Rockchip seems to love gstreamer. And if that doesn't work, then the VPU is still in a shit state... But at least, they can communicate with it so I guess it's something...

@Myy I missed your comment on the initial commit, I need to quite waiting until midnight to do the more interesting stuff. 
 
I'm being presented with 5 options under the rockchip_mpp kconfig:  I've got MPP_VDEC_DEVICE, vpu decovder V1 and V2, vpu encoder V1 and V2. Are they all needed?

Be sure that ROCKCHIP_MPP is enabled in the kernel configuration. It's not automatically set up.
The driver is available in "Device Drivers -> Staging drivers -> Rockcihp MPP".

Generally you get some warning, like for the Realtek staging driver, telling you that the code isn't that great, etc...

Okay, the driver is loading now... So now, I have to pull out a setup that can use "mpp service"... @JMCC Is your setup useable with MPP service ?
 
I'll try to rewrite the patches and upload them, along with a complete build, in order to let people play with it, see if they can get something out of it.
 
There's still a *BIG* fat warning : [    6.565010] Failed to attached device ff9a0000.video-codec to IOMMU_mapping
Now... The nodes have been split in two and ff9a0000.video-codec is the Video Encoder. The Video decoder is at ff9a0400.video-codec so... maybe it's working fine ?
It's still highly probable that it will just crash in a very bad way.
 
Anyway, I'll need some testers that are used to the whole MPP stuff.

There's a few problems with the UART driver on 5.0-rc2 kernels, since proper checks are not done correctly and lead to a Null Pointer dereference.
 
Here's a patch to avoid this issue :
 
From 7e43ae8446b420907f00b43308ad0b99b6fe4e51 Mon Sep 17 00:00:00 2001 From: "Miouyouyou (Myy)" <myy@miouyouyou.fr> Date: Wed, 16 Jan 2019 23:58:52 +0100 Subject: [PATCH] drivers: tty: serial: Bail out if no UART is detected Before the 5.0, serial8250_register_8250_port consisted of one BIG if(uart && uart->port.type != PORT_8250_CIR) block, which prevented NULL dereference if uart, a pointer to an "uart_8250_port" detected through "serial8250_find_match_or_unused", were to be NULL. However, a recent patch added a few bits of code just after that, and that code manipulates the "uart" pointer without checking if it's NULL or not. This patch changes the mechanism and bail out early if no UART structure pointer is provided serial8250_find_match_or_unused. A goto is used instead of returning directly, since we're inside a mutex and must release it correctly. Signed-off-by: Miouyouyou (Myy) <myy@miouyouyou.fr> --- drivers/tty/serial/8250/8250_core.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/drivers/tty/serial/8250/8250_core.c b/drivers/tty/serial/8250/8250_core.c index 189ab1212..11120c2d9 100644 --- a/drivers/tty/serial/8250/8250_core.c +++ b/drivers/tty/serial/8250/8250_core.c @@ -981,7 +981,12 @@ int serial8250_register_8250_port(struct uart_8250_port *up) mutex_lock(&serial_mutex); uart = serial8250_find_match_or_unused(&up->port); - if (uart && uart->port.type != PORT_8250_CIR) { + if (!uart) { + printk(KERN_INFO "One UART port failed to register correctly\n"); + goto no_uart; + } + + if (uart->port.type != PORT_8250_CIR) { if (uart->port.dev) uart_remove_one_port(&serial8250_reg, &uart->port); @@ -1081,6 +1086,7 @@ int serial8250_register_8250_port(struct uart_8250_port *up) uart->overrun_backoff_time_ms = 0; } +no_uart: mutex_unlock(&serial_mutex); return ret; -- 2.16.4  

Thanks to ASUS, I got my hands on one of these after seeing what appeared to be a giant heat sink fin integrated into the top of the case.  This case may be of interest to non-Tinker owners as well, it is not designed like the equivalent Pi cases with a fixed aluminum stud touching the SoC.  Instead it has a small aluminum block that has an adhesive side, and a thermal pad side, and is clamped down onto the processor by putting the two halves together.  This allows some freedom on the location of the SoC relative to the lid.
 
First off, same nice packaging the Tinker owners are familiar with:
                         
 
The case itself is quite heavy, and a nice color/texture, although the finish is most likely not 100% on this one, as it's pre-production
 
The reason for the weight becomes immediately obvious when pulling the two halves apart:

All I can say about this is, if the thermal pad/adhesive aluminum block fit properly, there is a lot of thermal mass here, and I'm perfectly alright with ASUS calling this fanless.  The extrusion is very thick, over 8mm in places.  Now for the bottom, a comparatively much thinner stamped part, the embossing does it's work to strengthen the base adequately.
 
Something important to notice in this picture:  The Tinker sits on aluminum studs, and does not bolt down.  The heat sink block holds it in place.  I have been told that the two additional holes you see here to the left side of the base are for a VESA mount adapter:
https://www.asus.com/Destkop-Accessories/VivoPC_VESA_Mounting_Kit/
 
I can't verify (no hardware), but the holes are 85mm apart and threaded.
 
Board fits nicely:
                                 
 
Now, putting it together only involves 1 thumb screw once you've gotten the aluminum block bit sorted out (a little bit of a balancing act, but not really a problem.  This would be my only feedback where I think a different option would have been better:  The thumbscrew is located at a position so as to be on center with the SoC.  This makes sure the whole stack is making contact, but it also creates a pivot point which rattles when you move the case around.  Not a problem for 90% of people, to be honest.  In my humble opinion, 2 thumb screws, one to each side, making it a bit more rigid once assembled.  Oh, I also pulled out some rubber feet and put them on it, none were provided in the box, and I like the grippy feet.
 

 
 
 
My unofficial testing shows the case very easily outperforming the tiny heat sink thermally, so in that respect it wins.  Aesthetically it is a very nice looking product, of course I'd say that should be expected.  I'll follow with something a bit more empirical later on.

Yeah, VPU doesn't exist in mainline.  There are some patches for educational purposes out there, but nothing constituting a truly functional driver.  Mali needs patched in, similar to what @Myy does with RK3288.

I've been able to boot a mainline kernel with a barely working DTS file that can, still, boot up to X11 and provide a working serial console.
USB is "just broken" though, since the DTS file doesn't follow the schematics at all.
 
https://gitlab.com/snippets/1792126
 
Note that I'm currently using an external MMC file for the moment, since it's the easiest way for me. Replacing broken kernels on the eMMC is a pain, I can't get Gadget mode working with the DesignWare DWC3 connector on U-Boot at the moment (I just tried to enable the option, though, not tinker with the DTB and such...).
So my only way to test and replace kernels quickly is to use a Micro SDCard to boot the system on the OrangePi, and use an USB MicroSD reader on my main PC to replace broken kernels/DTB files.
 
So, tomorrow will be : Schematics to DTS time... Yay...

I was offered this board to see what I can do with it. Just like the other boards I got (MiQi, Tinkerboard, ...)
 
Concerning "plans", I won't have any until I at least get a way to install a Linux image with a Rockchip kernel that "kind of work" correctly, in an "automated fashion" (scripted and useable with rkdeveloptool and Windows tools, so that you can reflash it at any moment if things go wrong).
Then I'll start mimicking their drivers on latest kernels and see if there's any major barrier, though I will have to deal with a few other issues before that.
 
I still see that the VPU is neither documented anywhere on their schematics, so there's at least that. If they could at least document which channel the VPU uses with the DMA subsystem, it might help a bit.
 
For the GPU side, given how fast the Panfrost project is progressing, it might be possible to overcome Mali binary drivers issues with these drivers. The Rockchip team was never able to provide Vulkan-ready binary drivers for their Mali-T7xx and Mali-T8xx chips, and the only one ARM provided with Vulkan support was with fbdev support only. There's of course drivers able to use DRM/KMS but you're "limited" to OpenGL ES 2.x / 3.x and you need an old GPL kernel driver that's starting to pile up patches to get it work with latest mainline kernels.
 
Now, given how the NanoPC T4 4.4 kernel was able to provide : HDMI output, (kind of shaky but working) Ethernet, eMMC and SDCard support, there's chances that "Orange Pi" specific issues might be fewer than expected. What remains, in terms of connections, are : SATA ports, USB, USBC, Audio, Wifi and Bluetooth.
Bluetooth might just be like the Tinkerboard : Wake up the Wifi/Bluetooth chip, flip a few GPIO and then find a way to send the local "binary blob firmware" to the chip.
 
There's also the MIPI connectors, but I got nothing to connect to these ports. Same for the battery.
 
Anyway, I'll be a bit busy this week but I might be able to advance a bit this week-end.

I was offered this board to see what I can do with it. Just like the other boards I got (MiQi, Tinkerboard, ...)
 
Concerning "plans", I won't have any until I at least get a way to install a Linux image with a Rockchip kernel that "kind of work" correctly, in an "automated fashion" (scripted and useable with rkdeveloptool and Windows tools, so that you can reflash it at any moment if things go wrong).
Then I'll start mimicking their drivers on latest kernels and see if there's any major barrier, though I will have to deal with a few other issues before that.
 
I still see that the VPU is neither documented anywhere on their schematics, so there's at least that. If they could at least document which channel the VPU uses with the DMA subsystem, it might help a bit.
 
For the GPU side, given how fast the Panfrost project is progressing, it might be possible to overcome Mali binary drivers issues with these drivers. The Rockchip team was never able to provide Vulkan-ready binary drivers for their Mali-T7xx and Mali-T8xx chips, and the only one ARM provided with Vulkan support was with fbdev support only. There's of course drivers able to use DRM/KMS but you're "limited" to OpenGL ES 2.x / 3.x and you need an old GPL kernel driver that's starting to pile up patches to get it work with latest mainline kernels.
 
Now, given how the NanoPC T4 4.4 kernel was able to provide : HDMI output, (kind of shaky but working) Ethernet, eMMC and SDCard support, there's chances that "Orange Pi" specific issues might be fewer than expected. What remains, in terms of connections, are : SATA ports, USB, USBC, Audio, Wifi and Bluetooth.
Bluetooth might just be like the Tinkerboard : Wake up the Wifi/Bluetooth chip, flip a few GPIO and then find a way to send the local "binary blob firmware" to the chip.
 
There's also the MIPI connectors, but I got nothing to connect to these ports. Same for the battery.
 
Anyway, I'll be a bit busy this week but I might be able to advance a bit this week-end.

I've adapted a set of patches from Urja Rannikko, which move the HDMI frequencies definition into the DTS file, with some preconfigured rates if none are defined in the DTS.
 
The original patches set is here : https://www.spinics.net/lists/arm-kernel/msg673156.html
The big patch I used as a basis is here : https://github.com/urjaman/arch-c201/blob/master/linux-c201/0020-RK3288-HDMI-clock-hacks-combined.patch
I included these patches by splitting them again : https://github.com/Miouyouyou/RockMyy/commit/57ff073fffb5678e75959e566704376f6f52ac17
 
These patches were suggested to me here : https://github.com/Miouyouyou/RockMyy/issues/3
 
Anyway, after some quick tests, these patches remove the dreaded bar of purple pixels at the left of my HDMI screen, when plugged to Tinkerboard systems and MiQi systems. Note that these bars only appear on my screen when using "good" (enough) USB power supply.
Still, with these patches, they are gone. However... I only tested these patches on my 60Hz 1080p screen, which is pretty much the de-facto standard screen.
I'd like to see people test these patches on other HDMI configurations which are known to have some issues with Tinkerboard, MiQi and others RK3288 systems.
 
So, yeah, if you'd like to test some kernel patches and are not afraid to see your screen go black after a reboot, have a try.
 
Note : This is completely unrelated with the CEC issue.

I've adapted a set of patches from Urja Rannikko, which move the HDMI frequencies definition into the DTS file, with some preconfigured rates if none are defined in the DTS.
 
The original patches set is here : https://www.spinics.net/lists/arm-kernel/msg673156.html
The big patch I used as a basis is here : https://github.com/urjaman/arch-c201/blob/master/linux-c201/0020-RK3288-HDMI-clock-hacks-combined.patch
I included these patches by splitting them again : https://github.com/Miouyouyou/RockMyy/commit/57ff073fffb5678e75959e566704376f6f52ac17
 
These patches were suggested to me here : https://github.com/Miouyouyou/RockMyy/issues/3
 
Anyway, after some quick tests, these patches remove the dreaded bar of purple pixels at the left of my HDMI screen, when plugged to Tinkerboard systems and MiQi systems. Note that these bars only appear on my screen when using "good" (enough) USB power supply.
Still, with these patches, they are gone. However... I only tested these patches on my 60Hz 1080p screen, which is pretty much the de-facto standard screen.
I'd like to see people test these patches on other HDMI configurations which are known to have some issues with Tinkerboard, MiQi and others RK3288 systems.
 
So, yeah, if you'd like to test some kernel patches and are not afraid to see your screen go black after a reboot, have a try.
 
Note : This is completely unrelated with the CEC issue.

Yes, but the Tinker moves it from 2 to 3 to keep common with what ASUS is doing.  I think the UART 2 might be blocking an IO range that periogerals could use, but pure speculation.  
 
 

Oldies but goodies: a made a quick demo with my smartphone (sorry for the bad quality) of Quake2 running on rk3288 using GL4ES. The game works really well, it is totally playable and there are no issues of any sort 
 
 

That's the question we're working on.  I think we can handle all from one codebase, at least as long as there aren't too many Tinker Reboot workarounds on other boards (I have that "contained" so it doesn't break anything)
 
From kernel to kernel the MAli drivers need tweaking, and while @Myy does excellent work with this, I thought it easier to keep Next on an an LTS kernel, then if people want bleeding edge they can compile a Dev image.
 
Also, welcome back!

The Mali driver can use devfreq, and has various ways to be told how to vary the frequency of the GPU.
Now, I've been informed recently that a patch is necessary on mainline kernels to avoid some Mali Devfreq related warnings and panics.
A modified version of the patch is available here : https://github.com/Miouyouyou/RockMyy/blob/master/patches/Midgard/r19p0-01rel0/0010-GPU-Mali-Midgard-remove-rcu_read_lock-references.patch
The original is here : https://github.com/mihailescu2m/linux/commit/bbe73c3c1143e5991bdcaee3afaecf5c31af0647
 
That said, if you want to push the Mali GPU to higher limits, you can do something like this :
cd /sys/class/misc/mali0/device/devfreq/devfreq0/ echo `cat max_freq` > min_freq echo 10 > polling_interval This will force the GPU to be at its highest clock rate constantly (supposedly, I have no way to actually verify that).
 
Now, let's be clear, if you launch a GPU intensive task with these settings, while not having a beefy stable power supply, the board might just shut down or not provide any performance gain. Now that's mostly the case for USB power supplied board.
 
Still, these settings provide good performances gain on some benchmarks, very little on others. Frequency alone won't help you overcome bad optimizations. Also sometimes performances issues become CPU bound as the GPU gets faster.
 

What if you do :
mv /var/lib/apt/lists/apt.armbian.com_dists_xenial_InRelease{,.bak} apt update ?
 
 
If that fails miserably, just do :
mv /var/lib/apt/lists/apt.armbian.com_dists_xenial_InRelease{.bak,}  

I'm currently watching a conversation between Tomasz Figa (Member of the Chromium team) and a developer named Baruch Siach about how to port the V4L2 driver to the mainline kernel.
Meanwhile, now that I got a roughly good idea on how DMA and IOMMU interoperate, and a documentation of the VPU registers (that was not easy due to the split of the drivers informations between the Rockchip VPU driver and their MPP codebase),
I'm currently :
trying to get a patched version of MPP working on 4.4 kernel working on a spare MiQi, using @JMCC scripts; get a snapshot of a H264 frame sent to the VPU, with all the VPU registers parameters, with that patched MPP; reproduce this in my last driver and see how it fares.  
Now, I had to a few troubles, between the fact that my current job takes (too much) time, a few bugs had to be dealt with, I had to buy a second screen because using my main screen and unplugging the HDMI output of my Radeon Vega is a NO-NO due to some vega driver bug (I have to report this bug...)... And I still need to get a few things setup to make the whole Tinkerboard/MiQi/RK3399 juggling painless.
 
The reason I have to peek MPP memory buffers instead of producing a H264 naked frame by myself is that : I currently don't know how to produce a H264 by myself (there's a few tutorials here and there...), and there's a lot of parameters in this standard that require a very good understanding of how video work in general, and some extensive knowledge of the norm. I mean, just computing the dimensions of a frame requires some special math that is not intuitive.
 
Still, if I can send a frame to VPU and have it decoded, than I will be able to focus on trying to get with working with the V4L2 API, which seems to be setup for GPU<->VPU communication, reusing the code Tomasz Figa wrote for the Chromium team.

https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?h=v4.14.52&amp;id=16d7ceb04b554207aa68dd27c1bc11f8933813fd

Ah, EOVERFLOW... You might need that kind of patch applied to the Mali drivers : https://github.com/Miouyouyou/RockMyy/blob/master/patches/Midgard/r19p0-01rel0/0009-GPU-ARM-Midgard-Adapt-to-the-new-mmap-call-checks.patch
 
They might have imported Linus Torvalds patch to mmap into the 4.14 series.

Ah, EOVERFLOW... You might need that kind of patch applied to the Mali drivers : https://github.com/Miouyouyou/RockMyy/blob/master/patches/Midgard/r19p0-01rel0/0009-GPU-ARM-Midgard-Adapt-to-the-new-mmap-call-checks.patch
 
They might have imported Linus Torvalds patch to mmap into the 4.14 series.

Prebuilt image can be found here

I don't think this is "required" per se.
 
To reserve memory for the CMA allocator, you need to pass the cma=Size parameter to the kernel, at boot time.
Edit your boot file accordingly and add something like cma=32M .
Now this will reserve 32M of RAM for the CMA allocator. Meaning that, if it's not used, you'll just lose 32M of RAM for nothing.

I'm Kever Yang from Rockchip BSP team, and I'm in charge of open source for Rockchip SoCs, you should able to see my patches to upstream U-Boot and kernel. Thanks very much to @tkaiser for share this thread to me, and I hope I can do some explanation here so that help developers understand what we are doing.
 
The github(https://github.com/rockchip-linux/) is owned by Rockchip,  and repos on github consider to be a 'Linux SDK' of some of Rockchip SoCs(mainly for RK3328, RK3288, RK3399 now), we have a wiki to provide basic document for it(http://opensource.rock-chips.com/), the github is now maintained by one of our product team. We hope this github can be a good reference to developers who interested in Rockchip platform.
 
I have to declare that Rockchip never try to stop community developers to report issue to Rockchip by github or by mail, but it's not correct to close issues without any comments, I'm sorry about what we have done and I promise it would never happen again. For some issues, we may not able to fix it, but people should get a saying if we have to close it. Rockchip always open to community, and, yes, we hope to get advice about github maintain.
 
Here is the information about Rockchip source code:
- Rockchip have only one common BSP maintained by BSP team, including U-Boot, kernel, rkbin(ddr, miniloader, trust), this is used in all Rockchip products and for many SoCs, so it including 4.4 base/LTS patches, upstream backport for some modules, vendor/rockchip solution for some modules because the requirement for production;
- Rockchip product team will test for product SDK release, but usually only for one SoC per SDK, and it will fix at that point before next version SDK. 
- Rockchip kernel is now 4.4, and will update to next LTS;    
  We try to use upstream source code as much as possible for all the modules, you can find this out by compare to our last version kernel 3.10, but still way to go.
- Rockchip kernel is used for Android and Linux OS;
- Github source code is a special 'Linux SDK', not like other 'fixed after release for one SoC' SDK, it keeps update and used for more than one SoCs, you can consider it to be a mirror of Rockchip internal BSP
     * that's why we not able to merge the pull request, we have to review all the source code internally;
     * the github release branch is  not maintained directly by BSP team, so it may not have maintained good enough now, I will sync with the maintainer, we will have better test, and better release flow;
- Rockchip don't have official community version BSP which only have everything from upstream, and won't going to have one in near future;
- Issue report(feature defeat, performance and etc) are always welcome, we want provide a better common kernel.
    We will fix issues really need to fix, people don't want to get a kernel always have the same issue.
- We focus on evb first for new SoCs, and some popular board will add later; this may be one of the conflict with the what community wants.
 
    About the common kernel for community, here is my suggestion:
- Community can chose one kernel version fork from Rockchip as common kernel, now the kernel4.4 already has good support for RK3288/RK3399/RK3328;
- Developing, apply patches for new board support, switch some of module to upstream driver, and so on;
- Upstream the new board support or feature support as much as you can, then rockchip kernel can support it when update to new kernel;
- If some feature/performance issue need to fix recently, community people can contact Rockchip/me, we will try to fix it if we can;
- maybe a mailing list is needed for better communication?