mali node missing from dtb's on H5?

[CNLohr]

I got the mali stuff working on my H3 on my orange pi one to great success for use with the HTC Vive, but I'm trying to get it working on my H5 on my Orange Pi Zero Plus 2. 

 

I can't seem to get any work done as far as installing the mali modules/etc since I can't find the mali node in any of the dtb's.  Also, because I am trying to use the normal armbian system, instead of installing my own sunxi-next kernel. 

 

I've switched to nightly, but can't figure out how to edit my own changes to the dts's that are built to add add the extra mali node (not that I have any idea where the mali450 is on the H5!).  Any help would be greatly appreciated.

 

I can't wait to fire this thing up with my HTC vive and see what my new even orager VR experience is like.

 

Charles

[jernej]

Here you have H5 mali DT node patch. Kernel side driver can be found on @Icenowy github page here. However, the biggest question is where to find appropriate userspace drivers if you care about licensing stuff. For private builds you can take them from anywhere you can find them, but it won't be redistributable. Don't forget that mali kernel driver version has to match to userspace version.

 

About how to add your own patches to build system - you can find some documentation here.

[CNLohr]

Just because there aren't mali userspace drivers yet doesn't mean the nodes and open source kernel sides shouldn't be mainlined.   But I appreciate the information, thank you for adding all three links, since I still don't understand the patching system for armbian.  If I can get it working, I will post back here with complete instructions.

[CNLohr]

Sorry, making one last post so I get notified of replies, I can't figure out how to turn it on for a topic.  Just a note if anyone minds updating the documentation:  It's unclear where the absolute path is for where the patches should go.

[chwe]

3 minutes ago, CNLohr said:

Just because there aren't mali userspace drivers yet doesn't mean the nodes and open source kernel sides shouldn't be mainlined.

Hmm, the more things you implement, the more things have to be maintained.. Means more opportunities that something can fail. It might be on a low priority as long as we don't get appropriate user-space libraries. 

 

But armbian makes it easy to patch the kernel to your own needs.  

set CREATE_PATCHES="no"  to yes (in config-default.conf) start to build the image for your board and make your changes in u-boot /kernel (can be found in cache) as soon as the buildscript says you should make the changes. In output/patch you can find the patch with your changes. If you want to use this one for new builds you can move it to userpatches/kernel and for every new build you make. 

[zador.blood.stained]

28 minutes ago, CNLohr said:

Just because there aren't mali userspace drivers yet doesn't mean the nodes and open source kernel sides shouldn't be mainlined.

Since there needs to be an exact match between the userspace and kernel driver versions a kernel-side driver will not be mainlined unless the userspace side becomes open-source too. Especially since the kernel side part may not meet the kernel code quality and integration criteria.

[CNLohr]

Re: not mainlining the module: That makes a fair bit of sense.  I was just referring to the dt.

 

I have another question about the dtb.  It seems that there is a relatively straightforward process for boot-patching DTs.  Is it feasible to create an overlay using this mechanism: https://docs.armbian.com/User-Guide_Allwinner_overlays/ with this patch file?  https://github.com/jernejsk/LibreELEC.tv/blob/aw_h5_init/projects/Allwinner/devices/H5/patches/linux/20-add-mali-node.patch if it is, then, it seems that I may be able to get everything working with minimal effort.

 

Charles

[zador.blood.stained]

1 hour ago, CNLohr said:

I have another question about the dtb.  It seems that there is a relatively straightforward process for boot-patching DTs.  Is it feasible to create an overlay using this mechanism: https://docs.armbian.com/User-Guide_Allwinner_overlays/ with this patch file?  https://github.com/jernejsk/LibreELEC.tv/blob/aw_h5_init/projects/Allwinner/devices/H5/patches/linux/20-add-mali-node.patch if it is, then, it seems that I may be able to get everything working with minimal effort.

Yes, creating and loading an overlay is the easiest way to add modifications to the DT, but in this case you will need to either replace all macro constants like GIC_SPI, IRQ_TYPE_LEVEL_HIGH, CLK_BUS_GPU by their numerical values or add #include directives for all necessary headers and ensure that compiler will find those headers, which is a bit tricky if overlays are compiled out of tree.

[CNLohr]

I can't seem to get the system to find the mali node.  Perhaps there is something wrong with my patch or patch process.

Here is my patch:

https://gist.github.com/cnlohr/8164f52c4551d10aaa2c94ad6156c7ae

 

Then, I am adding it with

armbian-add-overlay 20-add-mali-node.dts

And, reboot.  No errors. It find it and says good next time. 

 

20967 bytes read in 567 ms (35.2 KiB/s)
385 bytes read in 507 ms (0 Bytes/s)
Applying kernel provided DT overlay sun50i-h5-usbhost2.dtbo
385 bytes read in 497 ms (0 Bytes/s)
Applying kernel provided DT overlay sun50i-h5-usbhost3.dtbo
989 bytes read in 277 ms (2.9 KiB/s)
Applying user provided DT overlay 20-add-mali-node.dtbo
4179 bytes read in 477 ms (7.8 KiB/s)
Applying kernel provided DT fixup script (sun50i-h5-fixup.scr)
## Executing script at 44000000

 

No hint of mali in /dev or in dmesg. 

Insmodding sunxi-mali,  via the following procedure

# apt-get install linux-headers-next-sunxi64
# git clone https://github.com/mripard/sunxi-mali
# cd sunxi-mali
# export CROSS_COMPILE=
# export KDIR=/usr/src/linux-headers-4.14.37-sunxi64/
# export INSTALL_MOD_PATH=/lib/modules/4.14.37-sunxi64
# ./build.sh -r r6p2 -b
# ./build.sh -r r6p2 -i
# insmod /root/git/sunxi-mali/r6p2/src/devicedrv/mali/mali.ko

yields

[ 3498.705058] mali: loading out-of-tree module taints kernel.
[ 3498.709246] Couldn't find the mali node

 

Any idea where to go from here or how to diagnose why my dts patch isn't being used (despite

[CNLohr]

More notes.  I've tried https://github.com/Icenowy/sunxi-mali instead of https://github.com/mripard/sunxi-mali and can't get r7p0's kernel module to patch and compile compile. I've tried stubbing out stuff, but when I got the cma I couldn't figure out how to get it to compile against my nightly kernel.

 

Additionally, I've used `dtc -I fs /sys/firmware/devicetree/base` to see my current dtb configuration and it looks right as long as the h3 and h5 constants I subbed are the same.

 

                i2c1 = "/soc/i2c@1c2b000";
                hdmi_phy = "/soc/hdmi-phy@1ef0000";
                mali = "/soc/gpu@1280000", "";
                hdmi_in = "/soc/hdmi@1ee0000/ports/port@0";

...

                usb@1c1a400 {
                        compatible = "allwinner,sun8i-h3-ohci", "generic-ohci";
                        clocks = <0x3 0x21 0x3 0x25 0x3 0x5c>;
                        resets = <0x3 0x12 0x3 0x16>;
                        status = "disabled";
                        interrupts = <0x0 0x49 0x4>;
                        phandle = <0x2f>;
                        reg = <0x1c1a400 0x100>;
                };

                gpu@1280000 {
                        clocks = <0x3 0x31 0x3 0x72>;
                        resets = <0x3 0x23>;
                        clock-names = "bus", "core";
                        interrupts = <0x0 0x60 0x4 0x0 0x61 0x4 0x0 0x62 0x4 0x0 0x63 0x4 0x0 0x64 0x4 0x0 0x65 0x4 0x0 0x66 0x4 0x0 0x67 0x4 0x0 0x68 0x4 0x0 0x69 0x4 0x0 0x6a 0x4 0x0 0x6b 0x4>;
                        assigned-clock-rates = <0x16e36000>;
                        assigned-clocks = <0x3 0x72>;
                        phandle = <0x54>;
                        reg = <0x1e80000 0x30000>;
                        linux,phandle = <0x54>;
                        interrupt-names = "gp", "gpmmu", "pmu", "pp", "pp0", "ppmmu0", "pp1", "ppmmu1", "pp2", "ppmmu2", "pp3", "ppmmu3";
                };

                mmc@1c11000 {
                        cap-mmc-hw-reset;
                        compatible = "allwinner,sun50i-h5-emmc", "allwinner,sun50i-a64-emmc";
                        clocks = <0x3 0x18 0x3 0x4d>;
                        resets = <0x3 0x9>;
                        clock-names = "ahb", "mmc";

 

[zador.blood.stained]

5 hours ago, CNLohr said:

https://github.com/mripard/sunxi-mali

This driver is only for Mali400: https://github.com/mripard/sunxi-mali/blob/master/patches/0005-mali-Add-sunxi-platform.patch#L100

 

2 hours ago, CNLohr said:

I've tried https://github.com/Icenowy/sunxi-mali instead of https://github.com/mripard/sunxi-mali and can't get r7p0's kernel module to patch and compile compile.

You may have to copy patches from mripard's tree into Icenowy's one, specifically this patch: https://github.com/mripard/sunxi-mali/blob/master/patches/0012-mali-support-building-against-4.14.patch

 

[CNLohr]

I was able to get the r8p0 code running (though I did not follow a guide so I could have messed other stuff up), but still no node found.  Is there any way to know if the node is actually present and my problem is with the kernel module or if the problem is still in dtb land?

 

I will try applying the patch a little later.

[zador.blood.stained]

30 minutes ago, CNLohr said:

I was able to get the r8p0 code running (though I did not follow a guide so I could have messed other stuff up), but still no node found.  Is there any way to know if the node is actually present and my problem is with the kernel module or if the problem is still in dtb land?

If you did not apply any patches (from Maxime's or Icenowy's sunxi-mali repositories) to the upstream r8p0 driver it's not surprising that it does not work.

The driver will look for the "compatible" properties in DT, if nothing matches then it won't do anything when you load it.

 

You can dump the running DT with dtc if you want to check it

dtc -I fs /proc/device-tree

 

[CNLohr]

I will do those operations.  There's also a new concerning things that has just arisen.  The folks over on sunxi-mali have pointed out that the userspace blob for r7 and r8 is unavailable?  https://github.com/mripard/sunxi-mali/issues/40#issuecomment-386231407  It may mean that there is absolutely no way of using the GPU on the H5?  Is it time to go back to Allwinner and be like "what the heck, guys?"  If there is absolutely no way of using the GPU on the H5, why put it there?

 

Charles

[zador.blood.stained]

1 hour ago, CNLohr said:

The folks over on sunxi-mali have pointed out that the userspace blob for r7 and r8 is unavailable?  https://github.com/mripard/sunxi-mali/issues/40#issuecomment-386231407 

It is not available from Allwinner under a redistributable license that would allow to put it in OS images or in a Github repository. As I see it the blobs are not hardware specific and you could use a blob found anywhere for other platforms as long as they were built for the same GPU core and revision.

 

1 hour ago, CNLohr said:

It may mean that there is absolutely no way of using the GPU on the H5?

It is a difficult question that depends on the definition of "using", and there are different blob types (fbdev, xorg, wayland) and different architectures (armhf and arm64). You could probably find a blob for one of those combinations.

 

1 hour ago, CNLohr said:

Is it time to go back to Allwinner and be like "what the heck, guys?"  If there is absolutely no way of using the GPU on the H5, why put it there?

AW guys don't care (at least ones that could release a properly licensed mali blob). You can use GPU on Android, and that is all they care about.

[jernej]

Of course there is a way. Fortunately, Rockchip prepared a lot of Mali blobs with permissive license (they can be used on AW SoCs), among them all possible mali450 versions: https://github.com/rockchip-linux/libmali/tree/rockchip

 

Usually, mali blobs are distributed by SoC vendor. I'm pretty sure that AW can prepare those blobs. However, someone have to ask for them. That would be SBC vendor job. I'm not sure if any SBC vendor redistributed mali450 blobs, I didn't check since I found RK blobs with right license.

 

Be aware that even if you manage to put everything together, there is a chance that it doesn't work properly. I'm working on H5 mali450 currently (for LibreELEC) and I have some rendering issues with GBM version. Investigation is ongoing.

[CNLohr]

Ooph.  This is rough.  I'm currently hoping to use fbdev since that keeps things a loooot simpler on my end because I can re-use the H3 code.

 

I'm not exactly sure where I should go from here.  It looks like the rockchip supports r7 and r14, not sure if I should pursue either of the kernel drivers for them?

 

I did make this post... we can always dream: http://www.orangepi.org/orangepibbsen/forum.php?mod=viewthread&tid=3546&extra=

[CNLohr]

WOOT WOOT! One more step.  Got r7p0 compiled and running by applying mripard to icenowy's sunxi-mali.  It found the GPU.  Had a few bumps, but really only had to remove __GFP_REPEAT.  Hopefully that's not needed...

 

[  366.671234] cc1: page allocation stalls for 10848ms, order:0, mode:0x14080c0(GFP_KERNEL|__GFP_ZERO), nodemask=(null)
[  366.671263] cc1 cpuset=/ mems_allowed=0
[  366.671280] CPU: 3 PID: 3545 Comm: cc1 Not tainted 4.14.37-sunxi64 #59
[  366.671283] Hardware name: OrangePi Zero Plus2 (DT)
[  366.671287] Call trace:
[  366.671306] [<ffff0000080891d8>] dump_backtrace+0x0/0x3f0
[  366.671313] [<ffff0000080895dc>] show_stack+0x14/0x20
[  366.671323] [<ffff000008965710>] dump_stack+0x9c/0xbc
[  366.671333] [<ffff000008172d30>] warn_alloc+0x120/0x1b8
[  366.671341] [<ffff000008173744>] __alloc_pages_nodemask+0x914/0xc10
[  366.671348] [<ffff0000081c6984>] alloc_pages_current+0x7c/0xe0
[  366.671358] [<ffff00000819ca04>] __pte_alloc+0x24/0x108
[  366.671365] [<ffff0000081a1270>] __handle_mm_fault+0xb28/0xc60
[  366.671370] [<ffff0000081a1470>] handle_mm_fault+0xc8/0x168
[  366.671377] [<ffff000008097634>] do_page_fault+0x1ac/0x3c0
[  366.671382] [<ffff000008097884>] do_translation_fault+0x3c/0x48
[  366.671388] [<ffff000008080b14>] do_mem_abort+0x54/0xd0
[  366.671393] Exception stack(0xffff00000d7d3ec0 to 0xffff00000d7d4000)
[  366.671400] 3ec0: 000000003a3fe880 0000000000000000 0000000000010811 0000000000001f71
[  366.671406] 3ee0: 000000003a4007f0 0000000000001f71 0000000000000005 0000000000001f70
[  366.671412] 3f00: 0000000000000003 000000003a364d70 7fff316f7fff3154 0000000000000000
[  366.671417] 3f20: 000000007fff316f 0000000000000000 0000000000000001 062b080810100000
[  366.671424] 3f40: 000000000128ac58 0000ffffac3bab68 0000000001115240 000000003a3fe890
[  366.671430] 3f60: 0000000000001f60 0000ffffac48d4f8 0000ffffac48b0d0 0000ffffac463390
[  366.671436] 3f80: 0000ffffac48b9f0 0000ffffac48b9f0 0000ffffac48b9f0 0000ffffac48b998
[  366.671442] 3fa0: 0000000000001f70 0000ffffca40e810 0000ffffac3babbc 0000ffffca40e810
[  366.671448] 3fc0: 0000ffffac3b9198 0000000060000000 0000000000000009 00000000ffffffff
[  366.671454] 3fe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[  366.671460] [<ffff000008082cb8>] el0_da+0x20/0x24
[  366.671463] Mem-Info:
[  366.671478] active_anon:54381 inactive_anon:54601 isolated_anon:0
                active_file:265 inactive_file:345 isolated_file:96
                unevictable:0 dirty:0 writeback:0 unstable:0
                slab_reclaimable:2084 slab_unreclaimable:4584
                mapped:247 shmem:2071 pagetables:1937 bounce:0
                free:615 free_pcp:23 free_cma:134
[  366.671491] Node 0 active_anon:217524kB inactive_anon:218404kB active_file:1060kB inactive_file:1456kB unevictable:0kB isolated(anon):0kB isolated(file):256kB mapped:988kB dirty:0kB writeback:0kB shmem:8284kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
[  366.671494] Node 0 DMA free:2460kB min:2392kB low:2988kB high:3584kB active_anon:217524kB inactive_anon:218404kB active_file:1148kB inactive_file:1560kB unevictable:0kB writepending:0kB present:524288kB managed:494212kB mlocked:0kB kernel_stack:4560kB pagetables:7748kB bounce:0kB free_pcp:92kB local_pcp:28kB free_cma:536kB
[  366.671509] lowmem_reserve[]: 0 0 0
[  366.671517] Node 0 DMA: 113*4kB (UMEHC) 36*8kB (EHC) 24*16kB (UEHC) 17*32kB (UEHC) 11*64kB (UEHC) 1*128kB (H) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2500kB
[  366.671562] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[  366.671565] 3073 total pagecache pages
[  366.671570] 249 pages in swap cache
[  366.671574] Swap cache stats: add 2613, delete 2364, find 701/899
[  366.671576] Free swap  = 127484kB
[  366.671578] Total swap = 131068kB
[  366.671581] 131072 pages RAM
[  366.671583] 0 pages HighMem/MovableOnly
[  366.671585] 7519 pages reserved
[  366.671587] 32768 pages cma reserved
[  907.725103] mali: loading out-of-tree module taints kernel.
[  907.733352] Mali<2>: 
[  907.733362] Inserting Mali v900 device driver. 
[  907.733366] Mali<2>: 
[  907.733369] Compiled: May  4 2018, time: 02:11:43.
[  907.733371] Mali<2>: 
[  907.733373] Driver revision: r7p0-00rel1-d06b414
[  907.733375] Mali<2>: 
[  907.733377] mali_module_init() registering driver
[  907.733531] Mali: 
[  907.733533] Mali device driver loaded

 

 

Now, I'm up to trying this binary blob: https://github.com/rockchip-linux/libmali/blob/rockchip/lib/arm-linux-gnueabihf/libmali-utgard-400-r7p0-fbdev.so

 

Strangely, I am getting a crash when I try starting.

#0  0x0000000000000000 in ?? ()
#1  0x0000ffffbf53a78c in _egl_get_display () from /usr/local/lib/libMali.so
#2  0x0000ffffbf53b3b8 in eglGetDisplay () from /usr/local/lib/libMali.so
#3  0x0000aaaaaaab1ab4 in CNFGSetup (WindowName=WindowName@entry=0xaaaaaaab5cc8 "Spread Test", w=w@entry=2160, 
    h=h@entry=1200) at rawdraw/CNFGEGLDriver.c:313

 

I'm not sure how to debug it from here.  Any ideas?  Do you have any pointers or recommendations?  Were could I get eglegears or something like that that I could be careful with?

[CNLohr]

Anyone know what the distinction between libmali-utgard-450-r7p0-fbdev.so and libmali-utgard-450-r7p0.so  It looks like strace says it's looking for "/usr/lib/libGLESv2.so", but, no matter what combination I use it seems to die in the same place.

[Moklev]

4 hours ago, CNLohr said:

Anyone know what the distinction between libmali-utgard-450-r7p0-fbdev.so and libmali-utgard-450-r7p0.so  It looks like strace says it's looking for "/usr/lib/libGLESv2.so", but, no matter what combination I use it seems to die in the same place.

 

Maybe libmali-utgard-450-r7p0-fbdev.so is the Linux Framebuffer driver and libmali-utgard-450-r7p0.so the DRM x11 driver?

However, a closed source for an obsolete and ridicolous GPU like a Mali 4x0 is pathetic. What is the current status of open source drivers like Lima or Panfrost?

[jernej]

4 hours ago, CNLohr said:

libmali-utgard-450-r7p0.so

 

It has string

Quote

VARIANT=mali450-gles11-gles20-neon-linux-x11-drm-dma_buf-no_Werror

inside, so I guess it's x11 version. However, I'm not sure what drm in this case would mean.

 

7 minutes ago, Moklev said:

What is the current status of open source drivers like Lima or Panfrost?

Not there yet (for normal, everyday use). However, progresss is nice.

[CNLohr]

9 hours ago, Moklev said:

However, a closed source for an obsolete and ridicolous GPU like a Mali 4x0 is pathetic. What is the current status of open source drivers like Lima or Panfrost?

Do you have any recommendations for SBCs with a better GPU and not a lot of $$?  The Mali400 MP2 was sufficient for VR, I figured the 450 MP4 would be overkill.

 

Until Now, I didn't realize Panfrost or Lima even was really a thing.   If all I need is to render some triangles, are they suitable for such tests?  I can't seem to find any clear instructions on their installation.  Vertex shading would be a huge boon, but, I can tolerate it without them.  Speed is super critical though.

 

Still want to point out - my current question is how to install the Rockchip Mali Userspace blobs.  I can't seem to figure out any configuration that produces a functioning Mali driver.

 

 

 

[CNLohr]

I didn't check that the r7p0 driver actually made the node... It's missing.  There is no /dev/mali.

 

But, it does say the following:

[   31.338001] mali: loading out-of-tree module taints kernel.
[   31.347176] Mali<2>: 
[   31.347186] Inserting Mali v900 device driver. 
[   31.347190] Mali<2>: 
[   31.347193] Compiled: May  4 2018, time: 02:11:43.
[   31.347195] Mali<2>: 
[   31.347197] Driver revision: r7p0-00rel1-d06b414
[   31.347199] Mali<2>: 
[   31.347201] mali_module_init() registering driver
[   31.347382] Mali: 
[   31.347385] Mali device driver loaded

How would this happen and there be no mali node in /dev/mali?

[Moklev]

On 5/4/2018 at 7:21 PM, CNLohr said:

Do you have any recommendations for SBCs with a better GPU and not a lot of $$?  The Mali400 MP2 was sufficient for VR, I figured the 450 MP4 would be overkill.

 

An Asus Tinkerboard (Mali-T764) or a Rock64 (Mali-450MP4)?

With a quite good support:

http://opensource.rock-chips.com/wiki_Status_Matrix

 

On 5/4/2018 at 7:21 PM, CNLohr said:

Still want to point out - my current question is how to install the Rockchip Mali Userspace blobs.  I can't seem to figure out any configuration that produces a functioning Mali driver.

 

Try to build r6 driver by Free Electrons:

https://github.com/mripard/sunxi-mali

 

Kernel module:

git clone https://github.com/mripard/sunxi-mali.git
cd sunxi-mali
export CROSS_COMPILE=$TOOLCHAIN_PREFIX
export KDIR=$KERNEL_BUILD_DIR
export INSTALL_MOD_PATH=$TARGET_DIR
./build.sh -r r6p2 -b
./build.sh -r r6p2 -i

 

... and the userspace driver:

git clone https://github.com/free-electrons/mali-blobs.git
cd mali-blobs
cp -a r6p2/fbdev/lib/lib_fb_dev/lib* $TARGET_DIR/usr/lib

 

Pay attention:

"In order to build the kernel module, you'll need a functional DRM driver. If you have that already, you'll need the options CONFIG_CMA and CONFIG_DMA_CMA enabled in your kernel configuration."