4 4
Myy

The VPU driver

Recommended Posts

OK, got it compiled into 4.13.3:  http://sprunge.us/aBaA

 

Both HEVC-service and VPU-service available in /dev/

 

[Edit]  I forgot they were using Stretch, tried to install the Rockchip Mali driver and nuked my xserver.  Tried to install their xserver but Xenial does not support a couple libraries they now need.  Building a Stretch image to give it a try.  I'll have to try this on 4.14 next as I'd rather not stuff (this level of) experimental code into Next.

Share this post


Link to post
Share on other sites

I've ported the new code provided by Ayaka but I still need to test it. Since I'm using a different MiQi, I have to recompile mpp (done) and mpv.

 

The last one is posing several issues since it wants a pretty recent version of FFMPEG. I'll see whether I compile a new local version of ffmpeg of if I upgrade from Ubuntu Xenial to Ubuntu Artful to resolve this issue.

Share this post


Link to post
Share on other sites

Hi,

 

Please can you tell me the  current state of this work?

 

I wish to us a Tinker board, for Shinobi DVR  (cctv), and Rstp stream cameras.  The H264 streams are handled by ffmeg in the Shinobi  application.

 

many thanks

 

regards

 

Patrick

Share this post


Link to post
Share on other sites

So, I'm still trying to get this "thing" working on mainline kernels and my last issue is that the setup is done, I launch a decode pass, the IRQ handler is triggered but the output buffer remains untouched.

However, the thing I don't have any error messages. I see that some VPU registers changed after the (failed) decode pass, but most of the changed registers are completely undocumented.

 

And by "undocumented", I mean, even RKMPP and the Chromium V4L2 do not document these registers.

 

Meanwhile, I saw that the old drivers use the IOMMU DMA API, so I tried to enable the IOMMU_DMA API on Rockchip systems but that led to the Video Output MMU breaking, along with the DRM drivers firing BUG_ON after BUG_ON.

I was still able to boot the system by unplugging the screen, but the VPU driver issue remained identical.

Loads, runs, trigger the IRQ, no output.

 

I've even setup an IOMMU Fault handler, just in case, but it's never called. I'm currently trying to look at how to setup a DMA fault handler, maybe that's the issue here.

Meanwhile, I tried to disable an old patch I kept around in my kernels and... that led to the VPU MMU clocks not starting anymore... Yay...

 

As a last resort, I'm asking for various hints on the Rockchip Linux and IOMMU Linux Mailing lists. Still... I don't expect much.

 

So I'm close but still not there.

Maybe it's a simple bug in the code that I'm not seeing.

Share this post


Link to post
Share on other sites
On 8/18/2018 at 9:12 AM, Myy said:

So, I'm still trying to get this "thing" working on mainline kernels and my last issue is that the setup is done, I launch a decode pass, the IRQ handler is triggered but the output buffer remains untouched.

However, the thing I don't have any error messages. I see that some VPU registers changed after the (failed) decode pass, but most of the changed registers are completely undocumented.

 

 

 

Did they send you any hint?

Share this post


Link to post
Share on other sites

No. However, they sent some patches to the LKML with an implementation that seems to work with mainline kernels.
So, I'm currently trying to get a kernel 5.0 working in order to test the VPU on it, with these drivers, and see how it goes.

Share this post


Link to post
Share on other sites
15 hours ago, Myy said:

No. However, they sent some patches to the LKML with an implementation that seems to work with mainline kernels.
So, I'm currently trying to get a kernel 5.0 working in order to test the VPU on it, with these drivers, and see how it goes.

Sounds good!

Does it mean that there is more people pushing for VPU support on the tinkerboard? Or they made a generic vpu implementation?
Don't forget to post any progress :pp

Share this post


Link to post
Share on other sites

@Myy,

Any chance you will be working on RK3399 5.0.rcX ?  (GPU / VPU). I have been looking at your work for NanoPi-T4 but the label ** NOT TESTED ** was a bit discouraging.

I can try to help testing or to provide some feedback.

Share this post


Link to post
Share on other sites
8 hours ago, Myy said:

Well, I've been able to get a 5.0-rc2 working now. So I'll try the VPU patches this week and see if I can get decode anything with them.

 

Other people seems to be eager too, it seems.

Wow! i hate not being able to understand anything of those patches, looks like tons of work.

I bet you rockchip will copy your changes and publish as theirs some time.

 

So someone also tried porting google's vpu work on rk3288? is that compatible with debian?

 

Share this post


Link to post
Share on other sites

I will give the RK3399 a try. Note that ayaka patches on the mailing list provided DTS bindings for RK3399, so those might be what you're looking for.

https://lkml.org/lkml/2019/1/5/124

 

About Chromebook specific kernels, you might be able to adapt the patches to mainline kernels and get them working. But the patched kernels are generally frozen way beyond mainline, so it might be better to just use Rockchip 4.4 kernel at that point.

 

Now, the patch doesn't provide DTS bindings for RK3288 so I'll have to work on this, this week.

Share this post


Link to post
Share on other sites

I tried to integrate the driver and I was able to compile it, after a few modifications, however while I can load it manually, it seems to fail (silently) to initialize the hardware, so I guess I need to review the DTS nodes (and the code).

 

However, the driver still need modifications.

1. #include <soc/rockchip/pm_domains.h> does not exist in mainline kernels.

2. rockchip_pmu_idle_request does not exist in mainline kernels. However, I already have an old patch that adds it. That said, I'm not sure the added code is valid.

Using my patch, you need to add #include <linux/rockchip_pmu.h> to each mpp_dev_ file... I guess I'll rewrite my patch to generate <soc/rockchip/pm_domains.h> instead and avoid additional modifications.

 

So, yeah, it compiles, it loads, it does nothing. I'll have a look at the last part.

Share this post


Link to post
Share on other sites

Okay, the driver is loading now... So now, I have to pull out a setup that can use "mpp service"... @JMCC Is your setup useable with MPP service ?

 

I'll try to rewrite the patches and upload them, along with a complete build, in order to let people play with it, see if they can get something out of it.

 

There's still a *BIG* fat warning [    6.565010] Failed to attached device ff9a0000.video-codec to IOMMU_mapping

Now... The nodes have been split in two and ff9a0000.video-codec is the Video Encoder. The Video decoder is at ff9a0400.video-codec so... maybe it's working fine ?

It's still highly probable that it will just crash in a very bad way.

 

Anyway, I'll need some testers that are used to the whole MPP stuff.

Share this post


Link to post
Share on other sites
11 hours ago, Myy said:

And here's a build : https://miouyouyou.fr/RockMyy/builds/RockMyy-5.0.0-rc2-HighFive-VPU-Beta.tar.xz

 

Note that, with these builds, local SSH drop continuously so... if you don't have a keyboard/mouse plugged to the Tinkerboard, you might want to pass on it.

Also, you have to understand how to install an unofficial kernel.

Just wait if you don't.

So much progress in a single day!

 

I'm downloading the build and i will try to use it, got mouse+keyboard so gonna be fine i hope :)

 

Im not sure about how to install an unofficial kernel but i will try by unzipping the build on root a let it replace everything it asks.

Edit: i tried it and i guess failed as it does not longer boot.

I replaced /boot/, /lib/ & /usr/ folder from one partition and i also applied the same files from /boot/ folder to the boot partition.

When i give it power, does nothing, black screen.

 

I noticed /boot/ folder replaces all files except "rk3288-miniarm.dtb" which is the one set in extlinux.conf. Switching from that file to rk3288-tinker.dtb throws a Kernel Panic.

 

Maybe i'm not using the right base image (TinkerOS 2.0.8) i will try over an armbian one.

Edit2: Now i tried over Armbian 5.70 - Ubuntu bionic 4.4.166 and i get the black screen too.

Edit3: Now i tried over Armbian 5.70 - Ubuntu bionic 4.19 and.... IT BOOTS!

 

The problem wih 4.19 is that Media Script only supports 4.4 so i can't install it :(

Share this post


Link to post
Share on other sites

If you get a kernel panic, just try to get a photo or a retranscription of what's happening.

 

I'd highly recommend you to get an USB<->RS232 adapter that you can plug on your computer and the Tinkerboard, in order to get a serial output when kernel crashes. That way, you can launch "picocom" on your computer and do something like :

picocom -b 115200 /dev/ttyUSB0

 

And then copy-paste the big error message that appear on the serial console.

 

To quit picocom, you'll need to do CTRL+A then CTRL+Q

Share this post


Link to post
Share on other sites
15 hours ago, petatester said:

Now i tried over Armbian 5.70 - Ubuntu bionic 4.4.166 and i get the black screen too.

Edit3: Now i tried over Armbian 5.70 - Ubuntu bionic 4.19 and.... IT BOOTS!

 

The 4.4-4.18 images are looking for "miniarm" because we had working images before the mainline device tree was useful.  4.19+ inages the u-boot is looking for tinker dtb, but also has a miniarm DTB to maintain compatibility for the upgrade path.  TL/DR: So if you had no miniarm DTB the boot would fail.  

 

Ah, also, the old boot scripts (pre 4.19) hate the tinker dtb because of overlays.  I turned a bunch of gpio things off so the logic was sane for the overlays.

Share this post


Link to post
Share on other sites

OK, I have the driver compiled into the dev kernel, enabled, loads on boot:

 

root@tinkerboard:~# lsmod
Module                  Size  Used by
overlay                90112  1
fuse                   98304  3
zstd                   16384  4
gpio_keys              20480  0
zram                   24576  5
rockchip_vpu           32768  0
rockchip_rga           20480  0
rk_crypto              24576  0
videobuf2_dma_sg       20480  1 rockchip_rga
videobuf2_dma_contig    20480  1 rockchip_vpu
videobuf2_vmalloc      16384  1 rockchip_vpu
v4l2_mem2mem           20480  2 rockchip_vpu,rockchip_rga
videobuf2_memops       16384  3 videobuf2_dma_sg,videobuf2_dma_contig,videobuf2_vmalloc
snd_soc_hdmi_codec     16384  1
videobuf2_v4l2         24576  3 rockchip_vpu,v4l2_mem2mem,rockchip_rga
videobuf2_common       40960  4 rockchip_vpu,v4l2_mem2mem,videobuf2_v4l2,rockchip_rga
syscon_reboot_mode     16384  0
dw_wdt                 16384  0
reboot_mode            16384  1 syscon_reboot_mode
r8723bs               561152  0
rockchip_thermal       24576  0
cpufreq_dt             16384  0
thermal_sys            57344  2 rockchip_thermal,cpufreq_dt
sch_fq_codel           20480  6
ip_tables              24576  0
mali_kbase            335872  2
dw_hdmi_i2s_audio      16384  0
realtek                16384  1

@JMCC if I'm not mistaken this isn't going to work with the rockchip software, as it's looking for the vpu_service and hevc_service, correct?  being a V4L2 setup probably changes the dynamic a bit...

Share this post


Link to post
Share on other sites
19 minutes ago, TonyMac32 said:

being a V4L2 setup probably changes the dynamic a bit...

Yes, it certainly does. I don't know if the media script will work with this or not, depends whether the defaults of the programs I compiled were to enable or disable v4l2-m2m (I think at least ffmpeg in mpv has it enabled). I won't have access to my board in some days, so I cannot test now.

Share this post


Link to post
Share on other sites

Ah, the rockchip_vpu driver is the driver written by Ezequiel Garcia, which only supports JPEG encoding at the moment.

 

The driver is named rockchip_mpp. You might want to disable the Rockchip VPU in the staging drivers, and enable Rockchip MPP which is the option at the bottom of the staging section.

If you can't see it, maybe the integration within the Kconfig or Makefile isn't done correctly (´・ω・` ).

Share this post


Link to post
Share on other sites

@Myy I missed your comment on the initial commit, I need to quite waiting until midnight to do the more interesting stuff.  :unsure:

 

I'm being presented with 5 options under the rockchip_mpp kconfig:  I've got MPP_VDEC_DEVICE, vpu decovder V1 and V2, vpu encoder V1 and V2. Are they all needed?

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
4 4