bedalus Posted September 13, 2017 Posted September 13, 2017 Wow, this is such a great contribution for RK3288. It must be immensely satisfying to remake this driver and see it working!
TonyMac32 Posted September 22, 2017 Posted September 22, 2017 On 9/10/2017 at 4:46 PM, Myy said: Slight-butchering-to-test-the-VPU-driver I will be looking at this tomorrow with Armbian, the 4.14 stuff and the various other odds and ends got me off course. I will try it on the intended kernel first, but then I'll see if it goes with 4.14.
TonyMac32 Posted September 25, 2017 Posted September 25, 2017 OK, got it compiled into 4.13.3: http://sprunge.us/aBaA Both HEVC-service and VPU-service available in /dev/ [Edit] I forgot they were using Stretch, tried to install the Rockchip Mali driver and nuked my xserver. Tried to install their xserver but Xenial does not support a couple libraries they now need. Building a Stretch image to give it a try. I'll have to try this on 4.14 next as I'd rather not stuff (this level of) experimental code into Next. 1
qaplus Posted October 9, 2017 Posted October 9, 2017 I'm a linux newbie. Is there a way to try this VPU enabled kernel without compiling it from scratch?
TonyMac32 Posted October 9, 2017 Posted October 9, 2017 I'm afraid not, and if there were you would still have to compile the Rockchip MPP and video player.
Myy Posted November 26, 2017 Author Posted November 26, 2017 I've ported the new code provided by Ayaka but I still need to test it. Since I'm using a different MiQi, I have to recompile mpp (done) and mpv. The last one is posing several issues since it wants a pretty recent version of FFMPEG. I'll see whether I compile a new local version of ffmpeg of if I upgrade from Ubuntu Xenial to Ubuntu Artful to resolve this issue. 1
Myy Posted December 1, 2017 Author Posted December 1, 2017 Well, after upgrading to artful, I discovered that the official FFMPEG project pulled a patch provided by LongChair which enable MPP support in FFMPEG directly... So I now need to recompile FFMPEG... On Ubuntu... I'll try to recompile ffmpeg with just h264 support for now. 1
superpat Posted April 13, 2018 Posted April 13, 2018 Hi, Please can you tell me the current state of this work? I wish to us a Tinker board, for Shinobi DVR (cctv), and Rstp stream cameras. The H264 streams are handled by ffmeg in the Shinobi application. many thanks regards Patrick
Myy Posted August 18, 2018 Author Posted August 18, 2018 So, I'm still trying to get this "thing" working on mainline kernels and my last issue is that the setup is done, I launch a decode pass, the IRQ handler is triggered but the output buffer remains untouched. However, the thing I don't have any error messages. I see that some VPU registers changed after the (failed) decode pass, but most of the changed registers are completely undocumented. And by "undocumented", I mean, even RKMPP and the Chromium V4L2 do not document these registers. Meanwhile, I saw that the old drivers use the IOMMU DMA API, so I tried to enable the IOMMU_DMA API on Rockchip systems but that led to the Video Output MMU breaking, along with the DRM drivers firing BUG_ON after BUG_ON. I was still able to boot the system by unplugging the screen, but the VPU driver issue remained identical. Loads, runs, trigger the IRQ, no output. I've even setup an IOMMU Fault handler, just in case, but it's never called. I'm currently trying to look at how to setup a DMA fault handler, maybe that's the issue here. Meanwhile, I tried to disable an old patch I kept around in my kernels and... that led to the VPU MMU clocks not starting anymore... Yay... As a last resort, I'm asking for various hints on the Rockchip Linux and IOMMU Linux Mailing lists. Still... I don't expect much. So I'm close but still not there. Maybe it's a simple bug in the code that I'm not seeing.
petatester Posted January 17, 2019 Posted January 17, 2019 On 8/18/2018 at 9:12 AM, Myy said: So, I'm still trying to get this "thing" working on mainline kernels and my last issue is that the setup is done, I launch a decode pass, the IRQ handler is triggered but the output buffer remains untouched. However, the thing I don't have any error messages. I see that some VPU registers changed after the (failed) decode pass, but most of the changed registers are completely undocumented. Did they send you any hint?
Myy Posted January 17, 2019 Author Posted January 17, 2019 No. However, they sent some patches to the LKML with an implementation that seems to work with mainline kernels. So, I'm currently trying to get a kernel 5.0 working in order to test the VPU on it, with these drivers, and see how it goes.
petatester Posted January 18, 2019 Posted January 18, 2019 15 hours ago, Myy said: No. However, they sent some patches to the LKML with an implementation that seems to work with mainline kernels. So, I'm currently trying to get a kernel 5.0 working in order to test the VPU on it, with these drivers, and see how it goes. Sounds good! Does it mean that there is more people pushing for VPU support on the tinkerboard? Or they made a generic vpu implementation? Don't forget to post any progress :pp
Myy Posted January 20, 2019 Author Posted January 20, 2019 Well, I've been able to get a 5.0-rc2 working now. So I'll try the VPU patches this week and see if I can get decode anything with them. Other people seems to be eager too, it seems.
@lex Posted January 20, 2019 Posted January 20, 2019 @Myy, Any chance you will be working on RK3399 5.0.rcX ? (GPU / VPU). I have been looking at your work for NanoPi-T4 but the label ** NOT TESTED ** was a bit discouraging. I can try to help testing or to provide some feedback.
petatester Posted January 21, 2019 Posted January 21, 2019 8 hours ago, Myy said: Well, I've been able to get a 5.0-rc2 working now. So I'll try the VPU patches this week and see if I can get decode anything with them. Other people seems to be eager too, it seems. Wow! i hate not being able to understand anything of those patches, looks like tons of work. I bet you rockchip will copy your changes and publish as theirs some time. So someone also tried porting google's vpu work on rk3288? is that compatible with debian?
Myy Posted January 21, 2019 Author Posted January 21, 2019 I will give the RK3399 a try. Note that ayaka patches on the mailing list provided DTS bindings for RK3399, so those might be what you're looking for. https://lkml.org/lkml/2019/1/5/124 About Chromebook specific kernels, you might be able to adapt the patches to mainline kernels and get them working. But the patched kernels are generally frozen way beyond mainline, so it might be better to just use Rockchip 4.4 kernel at that point. Now, the patch doesn't provide DTS bindings for RK3288 so I'll have to work on this, this week.
Myy Posted January 22, 2019 Author Posted January 22, 2019 I tried to integrate the driver and I was able to compile it, after a few modifications, however while I can load it manually, it seems to fail (silently) to initialize the hardware, so I guess I need to review the DTS nodes (and the code). However, the driver still need modifications. 1. #include <soc/rockchip/pm_domains.h> does not exist in mainline kernels. 2. rockchip_pmu_idle_request does not exist in mainline kernels. However, I already have an old patch that adds it. That said, I'm not sure the added code is valid. Using my patch, you need to add #include <linux/rockchip_pmu.h> to each mpp_dev_ file... I guess I'll rewrite my patch to generate <soc/rockchip/pm_domains.h> instead and avoid additional modifications. So, yeah, it compiles, it loads, it does nothing. I'll have a look at the last part.
Myy Posted January 22, 2019 Author Posted January 22, 2019 Okay, the driver is loading now... So now, I have to pull out a setup that can use "mpp service"... @JMCC Is your setup useable with MPP service ? I'll try to rewrite the patches and upload them, along with a complete build, in order to let people play with it, see if they can get something out of it. There's still a *BIG* fat warning : [ 6.565010] Failed to attached device ff9a0000.video-codec to IOMMU_mapping Now... The nodes have been split in two and ff9a0000.video-codec is the Video Encoder. The Video decoder is at ff9a0400.video-codec so... maybe it's working fine ? It's still highly probable that it will just crash in a very bad way. Anyway, I'll need some testers that are used to the whole MPP stuff. 1
Myy Posted January 22, 2019 Author Posted January 22, 2019 Here's the build script with the additional patches : https://github.com/Miouyouyou/RockMyy/tree/RandyLi-Patches-Test https://gitlab.com/Miouyouyou/RockMyy/tree/RandyLi-Patches-Test https://notabug.org/Miouyouyou/RockMyy/src/RandyLi-Patches-Test I'll upload a complete build in a few minutes. Now, the additional patches are : Randy Li patches : https://github.com/Miouyouyou/RockMyy/blob/RandyLi-Patches-Test/patches/kernel/v5.0/0101-staging-video-rockchip-video-codec-for-vendor-API.patch https://github.com/Miouyouyou/RockMyy/blob/RandyLi-Patches-Test/patches/kernel/v5.0/0102-staging-video-rockchip-fixup-for-upstream.patch https://github.com/Miouyouyou/RockMyy/blob/RandyLi-Patches-Test/patches/kernel/v5.0/0103-staging-video-rockchip-add-video-codec.patch Additional patches to make it compile : https://github.com/Miouyouyou/RockMyy/blob/RandyLi-Patches-Test/patches/kernel/v5.0/0006-soc-rockchip-power-domain-export-idle-request.patch RK3288 DTS patches : https://github.com/Miouyouyou/RockMyy/blob/RandyLi-Patches-Test/patches/kernel/v5.0/DTS/0025-ARM-DTSI-Setup-the-VEPU-and-VDPU-nodes-for-the-MPP-S.patch https://github.com/Miouyouyou/RockMyy/blob/RandyLi-Patches-Test/patches/kernel/v5.0/DTS/0026-ARM-DTS-Enable-the-VEPU-VDPU-and-VPU-MMU-on-the-Tink.patch
Myy Posted January 22, 2019 Author Posted January 22, 2019 And here's a build : https://miouyouyou.fr/RockMyy/builds/RockMyy-5.0.0-rc2-HighFive-VPU-Beta.tar.xz Note that, with these builds, local SSH drop continuously so... if you don't have a keyboard/mouse plugged to the Tinkerboard, you might want to pass on it. Also, you have to understand how to install an unofficial kernel. Just wait if you don't.
JMCC Posted January 23, 2019 Posted January 23, 2019 8 hours ago, Myy said: @JMCC Is your setup useable with MPP service ? Yes, that's how HW video accel is performed with my script.
petatester Posted January 23, 2019 Posted January 23, 2019 11 hours ago, Myy said: And here's a build : https://miouyouyou.fr/RockMyy/builds/RockMyy-5.0.0-rc2-HighFive-VPU-Beta.tar.xz Note that, with these builds, local SSH drop continuously so... if you don't have a keyboard/mouse plugged to the Tinkerboard, you might want to pass on it. Also, you have to understand how to install an unofficial kernel. Just wait if you don't. So much progress in a single day! I'm downloading the build and i will try to use it, got mouse+keyboard so gonna be fine i hope Im not sure about how to install an unofficial kernel but i will try by unzipping the build on root a let it replace everything it asks. Edit: i tried it and i guess failed as it does not longer boot. I replaced /boot/, /lib/ & /usr/ folder from one partition and i also applied the same files from /boot/ folder to the boot partition. When i give it power, does nothing, black screen. I noticed /boot/ folder replaces all files except "rk3288-miniarm.dtb" which is the one set in extlinux.conf. Switching from that file to rk3288-tinker.dtb throws a Kernel Panic. Maybe i'm not using the right base image (TinkerOS 2.0.8) i will try over an armbian one. Edit2: Now i tried over Armbian 5.70 - Ubuntu bionic 4.4.166 and i get the black screen too. Edit3: Now i tried over Armbian 5.70 - Ubuntu bionic 4.19 and.... IT BOOTS! The problem wih 4.19 is that Media Script only supports 4.4 so i can't install it
Myy Posted January 23, 2019 Author Posted January 23, 2019 If you get a kernel panic, just try to get a photo or a retranscription of what's happening. I'd highly recommend you to get an USB<->RS232 adapter that you can plug on your computer and the Tinkerboard, in order to get a serial output when kernel crashes. That way, you can launch "picocom" on your computer and do something like : picocom -b 115200 /dev/ttyUSB0 And then copy-paste the big error message that appear on the serial console. To quit picocom, you'll need to do CTRL+A then CTRL+Q
TonyMac32 Posted January 23, 2019 Posted January 23, 2019 15 hours ago, petatester said: Now i tried over Armbian 5.70 - Ubuntu bionic 4.4.166 and i get the black screen too. Edit3: Now i tried over Armbian 5.70 - Ubuntu bionic 4.19 and.... IT BOOTS! The 4.4-4.18 images are looking for "miniarm" because we had working images before the mainline device tree was useful. 4.19+ inages the u-boot is looking for tinker dtb, but also has a miniarm DTB to maintain compatibility for the upgrade path. TL/DR: So if you had no miniarm DTB the boot would fail. Ah, also, the old boot scripts (pre 4.19) hate the tinker dtb because of overlays. I turned a bunch of gpio things off so the logic was sane for the overlays.
TonyMac32 Posted January 26, 2019 Posted January 26, 2019 OK, I have the driver compiled into the dev kernel, enabled, loads on boot: root@tinkerboard:~# lsmod Module Size Used by overlay 90112 1 fuse 98304 3 zstd 16384 4 gpio_keys 20480 0 zram 24576 5 rockchip_vpu 32768 0 rockchip_rga 20480 0 rk_crypto 24576 0 videobuf2_dma_sg 20480 1 rockchip_rga videobuf2_dma_contig 20480 1 rockchip_vpu videobuf2_vmalloc 16384 1 rockchip_vpu v4l2_mem2mem 20480 2 rockchip_vpu,rockchip_rga videobuf2_memops 16384 3 videobuf2_dma_sg,videobuf2_dma_contig,videobuf2_vmalloc snd_soc_hdmi_codec 16384 1 videobuf2_v4l2 24576 3 rockchip_vpu,v4l2_mem2mem,rockchip_rga videobuf2_common 40960 4 rockchip_vpu,v4l2_mem2mem,videobuf2_v4l2,rockchip_rga syscon_reboot_mode 16384 0 dw_wdt 16384 0 reboot_mode 16384 1 syscon_reboot_mode r8723bs 561152 0 rockchip_thermal 24576 0 cpufreq_dt 16384 0 thermal_sys 57344 2 rockchip_thermal,cpufreq_dt sch_fq_codel 20480 6 ip_tables 24576 0 mali_kbase 335872 2 dw_hdmi_i2s_audio 16384 0 realtek 16384 1 @JMCC if I'm not mistaken this isn't going to work with the rockchip software, as it's looking for the vpu_service and hevc_service, correct? being a V4L2 setup probably changes the dynamic a bit...
JMCC Posted January 26, 2019 Posted January 26, 2019 19 minutes ago, TonyMac32 said: being a V4L2 setup probably changes the dynamic a bit... Yes, it certainly does. I don't know if the media script will work with this or not, depends whether the defaults of the programs I compiled were to enable or disable v4l2-m2m (I think at least ffmpeg in mpv has it enabled). I won't have access to my board in some days, so I cannot test now.
TonyMac32 Posted January 26, 2019 Posted January 26, 2019 No worries, I'll playSent from my Pixel using Tapatalk
Myy Posted January 26, 2019 Author Posted January 26, 2019 Ah, the rockchip_vpu driver is the driver written by Ezequiel Garcia, which only supports JPEG encoding at the moment. The driver is named rockchip_mpp. You might want to disable the Rockchip VPU in the staging drivers, and enable Rockchip MPP which is the option at the bottom of the staging section. If you can't see it, maybe the integration within the Kconfig or Makefile isn't done correctly (´・ω・` ).
TonyMac32 Posted January 26, 2019 Posted January 26, 2019 @Myy I missed your comment on the initial commit, I need to quite waiting until midnight to do the more interesting stuff. I'm being presented with 5 options under the rockchip_mpp kconfig: I've got MPP_VDEC_DEVICE, vpu decovder V1 and V2, vpu encoder V1 and V2. Are they all needed? 1
Myy Posted January 26, 2019 Author Posted January 26, 2019 Put them all as modules and the DTB parser will load the appropriate ones automatically.
Recommended Posts