2 2
qstaq

[Fixed] Changing Amlogic GPU RAM allocation through the DTB?

Recommended Posts

I have a number of headless AML S905 / S912 1GB Ram TV boxes running a bastardised Armbian for systems monitoring. I have a 200MB zram compressed block device setup to store temporary data which gets processed locally then synced to a command server every hour because the eMMC's on these cheap boxes do not last very long under sustained write scenarios. I would like to increase the zram filesystem size to 500MB so I only have to sync daily instead of hourly but because 25% of the RAM is allocated to the GPU/VPU I dont have enough available ram

 

All the devices I have seen seem to allocate approx 250MB for the GPU which is obviously wasted in a headless scenario. I would like to reduce this to 32MB or 64MB which would give enough system RAM left over for a 500MB zram allocation

 

Can anyone point me in the right direction of what I need to add or modify in the DTS files? 

 

So far my digging has me here:

Im guessing this is or can be accomplished through modifying the DTB's for these devices but having looked through the DTS source files for various AML S9xx and RK33xx boxes I cant find anywhere where the GPU RAM allocation is defined. Im a complete noob when it comes to device trees so I have been studying @balbes150 Amlogic_s905-kernel github repo

 

Using a Nexbox A95X as an example I can see that the dtb image gets built from the source code of the following dts files:

 

https://github.com/150balbes/Amlogic_s905-kernel/tree/master/arch/arm64/boot/dts/amlogic

meson-gxl-s905x-nexbox-a95x.dts
meson-gxl-s905x.dtsi
meson-gxl.dtsi
meson-gx.dtsi
meson-gxl-mali.dtsi

 

Potentially Interesting looking entries:

In the meson-gxl-s905x-nexbox-a95x.dts file I found the following, but modifying it seems to just reduce total RAM available to the SoC

memory@0 {
    device_type = "memory";
    reg = <0x0 0x0 0x0 0x80000000>;
};

 

In meson-gx.dtsi I found vdec and vpu declarations. They seem to declare 5 separate memory regions but nothing seems to match the size of what has been reserved

vdec: video-decoder@c8820000 {
    compatible = "amlogic,gx-vdec";
    reg = <0x0 0xc8820000 0x0 0x10000>,
          <0x0 0xc110a580 0x0 0xe4>;
    reg-names = "dos", "esparser";

    interrupts = <GIC_SPI 44 IRQ_TYPE_EDGE_RISING>,
         	    <GIC_SPI 32 IRQ_TYPE_EDGE_RISING>;
    interrupt-names = "vdec", "esparser";

    amlogic,ao-sysctrl = <&sysctrl_AO>;
    amlogic,canvas = <&canvas>;
};
vpu: vpu@d0100000 {
    compatible = "amlogic,meson-gx-vpu";
    reg = <0x0 0xd0100000 0x0 0x100000>,
          <0x0 0xc883c000 0x0 0x1000>,
          <0x0 0xc8838000 0x0 0x1000>;
    reg-names = "vpu", "hhi", "dmc";
    interrupts = <GIC_SPI 3 IRQ_TYPE_EDGE_RISING>;
    #address-cells = <1>;
    #size-cells = <0>;
    amlogic,canvas = <&canvas>;

    /* CVBS VDAC output port */
    cvbs_vdac_port: port@0 {
        reg = <0>;
    };

    /* HDMI-TX output port */
    hdmi_tx_port: port@1 {
        reg = <1>;
            hdmi_tx_out: endpoint {
            remote-endpoint = <&hdmi_tx_in>;
        };
    };
};

 

 

Share this post


Link to post
Share on other sites

A more interesting section in meson-gx.dtsi:

reserved-memory {
	#address-cells = <2>;
	#size-cells = <2>;
	ranges;

	/* 16 MiB reserved for Hardware ROM Firmware */
	hwrom_reserved: hwrom@0 {
		reg = <0x0 0x0 0x0 0x1000000>;
		no-map;
	};

	/* 2 MiB reserved for ARM Trusted Firmware (BL31) */
	secmon_reserved: secmon@10000000 {
		reg = <0x0 0x10000000 0x0 0x200000>;
		no-map;
	};

	/* Alternate 3 MiB reserved for ARM Trusted Firmware (BL31) */
	secmon_reserved_alt: secmon@5000000 {
		reg = <0x0 0x05000000 0x0 0x300000>;
		no-map;
	};

	/* 32 MiB reserved for ARM Trusted Firmware (BL32) */
	secmon_reserved_bl32: secmon@5300000 {
		reg = <0x0 0x05300000 0x0 0x2000000>;
		no-map;
	};

	linux,cma {
		compatible = "shared-dma-pool";
		reusable;
		size = <0x0 0x38000000>;
		alignment = <0x0 0x400000>;
		linux,cma-default;
	};
};

I presume this is reserving 53MB for ROM and ATF?

 

The linux cma section seems to be declaring 896MB of shared RAM but I dont understand what the alignment declaration does and none of these numbers match up with what seems to be allocated for the GPU

Share this post


Link to post
Share on other sites

i was looking into this a while ago for a s905w tv box and found out, that a lot of memory seems to be blocked already by legacy u-boot etc. (about 120m) and you cannot get it back by changes on linux kernel side. what helps is to chainload mainline u-boot (just build the libretech-cc u-boot from 2019.07 and use the resulting u-boot.bin):

 

fatload mmc 0 0x01000000 u-boot.bin
go 0x01000000

 

in legacy u-boot and then you can use regular boot.scr or extlinux to boot further from there using the chainloaded u-boot just booted.

 

good luck and best wishes - hexdump

Share this post


Link to post
Share on other sites
On 8/8/2019 at 3:32 AM, qstaq said:

I presume this is reserving 53MB for ROM and ATF?

The reg property is used to declare where the device have their memory-mapped region to interact with the operating system. You can imagine that devices (like the ethernet, the gpu, etc...) have one or more "windows" in the virtual address space of the CPU. Normally these windows are fixed in the address space, and the device tree declares where those windows are, letting the operating system drivers know where to access the devices.

 

In case of reserved-memory section, I see no real drivers involved there, so I guess that the memory is already reserved by bootloaders (the amlogic proprietary bootloader, mainly) and the declarations there inform the kernel not to use those locations because they are reserved.

You could remove the reserved sections, probably the kernel will show that more memory is available, but you may also expect malfunctions because some application may write in those reserved locations causing unpredictable behaviour!

 

Also check the size of the reserved CMA block (dmesg | grep cma). It should be possible to resize it via extraargs in armbianEnv.txt or recompiling the kernel. 8 megabytes there should suffice.

 

edit: I checked on a S905 device, the CMA can be changed only editing the device tree and should not be less than 32M, otherwise X fails to start on my configuration.

Share this post


Link to post
Share on other sites

Thanks @jock

 

I have created a new dtb with the following change:

 

linux,cma 
	{
	compatible = "shared-dma-pool";
	reusable; size = <0x0 0x2000000>;
	alignment = <0x0 0x400000>;
	linux,cma-default;
    };

This should reduce the cma size to 32MB (0x2000000) if I understand correctly?

 

However even with that change I still see "cma: Reserved 256 MiB" in dmesg, not 32MB as I would expect. I have also tried various different "cma=x" values on the kernel command line but the argument seems to be ignored

 

Have I misunderstood something?

Share this post


Link to post
Share on other sites
10 minutes ago, qstaq said:

Thanks @jock

 

I have created a new dtb with the following change:

 


linux,cma 
	{
	compatible = "shared-dma-pool";
	reusable; size = <0x0 0x2000000>;
	alignment = <0x0 0x400000>;
	linux,cma-default;
    };

This should reduce the cma size to 32MB (0x2000000) if I understand correctly?

 

However even with that change I still see "cma: Reserved 256 MiB" in dmesg, not 32MB as I would expect. I have also tried various different "cma=x" values on the kernel command line but the argument seems to be ignored

 

Have I misunderstood something?

Yes it should work that way. At least, it worked on my S905 machine. Didn't you forget to recompile the dts back to dtb, did you?

I'm not sure, but the reusable flag there should allow the kernel to use the same CMA region for its own needs, so it should act totally as a shared dynamic memory. You can anyway take a look to the official documentation becuase I may miss something: https://www.kernel.org/doc/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt

 

 

Share this post


Link to post
Share on other sites

Im de-compiling the existing dbt with:

 

dtc -I dtb -O dts meson-gxl-s905x-nexbox-a95x.dtb > custom.dts

then modifying custom.dts and re-compiling with:

 

dtc -O dtb -o custom.dtb custom.dts

Then changing uEnv.ini to point at custom.dts 

 

Edit: Working Now :)

 

OK so setting the FDT option in uEnv.ini doesnt do anything on S905x devices, you have to change the FDT in extlinux.conf instead (balbes images)

Share this post


Link to post
Share on other sites
On 8/8/2019 at 4:48 PM, hexdump said:

i was looking into this a while ago for a s905w tv box and found out, that a lot of memory seems to be blocked already by legacy u-boot etc. (about 120m) and you cannot get it back by changes on linux kernel side. what helps is to chainload mainline u-boot (just build the libretech-cc u-boot from 2019.07 and use the resulting u-boot.bin):

 

fatload mmc 0 0x01000000 u-boot.bin
go 0x01000000

 

in legacy u-boot and then you can use regular boot.scr or extlinux to boot further from there using the chainloaded u-boot just booted.

 

good luck and best wishes - hexdump

Hi @hexdump

 

I have built the libretech-cc u-boot and copied it to my /boot fs. How do I instruct legacy u-boot, installed on an sd card, to chainload the libretech u-boot?

 

Thanks for the help

Share this post


Link to post
Share on other sites

by just putting the above lines into your s905_autoboot.cmd and creating a fresh s905_autoboot from it. then it will on boot start the mainline u-boot instead of the kernel. and you need a working  extlinux/extlinux.conf file then which tells the mainline what to do.

 

good luck - hexdump

Share this post


Link to post
Share on other sites
On 8/8/2019 at 11:48 PM, hexdump said:

i was looking into this a while ago for a s905w tv box and found out, that a lot of memory seems to be blocked already by legacy u-boot etc. (about 120m) and you cannot get it back by changes on linux kernel side. what helps is to chainload mainline u-boot (just build the libretech-cc u-boot from 2019.07 and use the resulting u-boot.bin):

 

fatload mmc 0 0x01000000 u-boot.bin
go 0x01000000

 

in legacy u-boot and then you can use regular boot.scr or extlinux to boot further from there using the chainloaded u-boot just booted.

 

good luck and best wishes - hexdump

 

I'm trying to get this to work but the board will always reboot itself with this error:

gxl_p230_v1#fatload usb 0 0x01000000 u-boot.bin

reading u-boot.bin
851968 bytes read in 43 ms (18.9 MiB/s)
gxl_p230_v1#go 0x01000000

## Starting application at 0x01000000 ...
"Synchronous Abort" handler, esr 0x02000000
ELR:     1000000
LR:      77ee4824
x0 : 0000000000000001 x1 : 0000000073ede1b8
x2 : 0000000073ede1b8 x3 : 0000000001000000
x4 : 0000000000000020 x5 : 000000008c137742
x6 : 0000000077f41000 x7 : 000000000000000f
x8 : 0000000077f5f748 x9 : 0000000000000000
x10: 00000000003e0000 x11: 0000000000010000
x12: 000000000000000d x13: 0000000000000000
x14: 0000000000000000 x15: 0000000000000000
x16: 0000000000000000 x17: 0000000000000000
x18: 0000000073ec8e28 x19: 0000000073ede1b8
x20: 0000000000000002 x21: 0000000001000000
x22: 0000000000000002 x23: 0000000077f71ef8
x24: 0000000000000000 x25: 0000000073ee4800
x26: 0000000000000000 x27: 0000000000000000
x28: 0000000000000000 x29: 0000000073ec8b80

Resetting CPU ...

resetting ...

@hexdump The board is a s905d TV box, and I built the u-boot binaries with this tutorial: http://wiki.loverpi.com/faq:sbc:libre-aml-s805x-howto-compile-u-boot
 

The "Synchronous Abort" handler always appear whenever I try to boot just about anything. I'm trying to get Fedora on this box, and possibly even UEFI booting, but the stock u-boot just won't let me test anything. Any suggestions on how I should proceed?

Share this post


Link to post
Share on other sites

@utgf - maybe try the attached u-boot.bin (after gunzipping it) - if that does not work, then maybe do a printenv on the legacy u-boot prompt - maybe it uses another address (your should find some start address in there)

 

best wishes - hexdump

u-boot.bin-libretech-cc.gz

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
2 2