2 2
qstaq

[Fixed] Changing Amlogic GPU RAM allocation through the DTB?

Recommended Posts

I have a number of headless AML S905 / S912 1GB Ram TV boxes running a bastardised Armbian for systems monitoring. I have a 200MB zram compressed block device setup to store temporary data which gets processed locally then synced to a command server every hour because the eMMC's on these cheap boxes do not last very long under sustained write scenarios. I would like to increase the zram filesystem size to 500MB so I only have to sync daily instead of hourly but because 25% of the RAM is allocated to the GPU/VPU I dont have enough available ram

 

All the devices I have seen seem to allocate approx 250MB for the GPU which is obviously wasted in a headless scenario. I would like to reduce this to 32MB or 64MB which would give enough system RAM left over for a 500MB zram allocation

 

Can anyone point me in the right direction of what I need to add or modify in the DTS files? 

 

So far my digging has me here:

Im guessing this is or can be accomplished through modifying the DTB's for these devices but having looked through the DTS source files for various AML S9xx and RK33xx boxes I cant find anywhere where the GPU RAM allocation is defined. Im a complete noob when it comes to device trees so I have been studying @balbes150 Amlogic_s905-kernel github repo

 

Using a Nexbox A95X as an example I can see that the dtb image gets built from the source code of the following dts files:

 

https://github.com/150balbes/Amlogic_s905-kernel/tree/master/arch/arm64/boot/dts/amlogic

meson-gxl-s905x-nexbox-a95x.dts
meson-gxl-s905x.dtsi
meson-gxl.dtsi
meson-gx.dtsi
meson-gxl-mali.dtsi

 

Potentially Interesting looking entries:

In the meson-gxl-s905x-nexbox-a95x.dts file I found the following, but modifying it seems to just reduce total RAM available to the SoC

memory@0 {
    device_type = "memory";
    reg = <0x0 0x0 0x0 0x80000000>;
};

 

In meson-gx.dtsi I found vdec and vpu declarations. They seem to declare 5 separate memory regions but nothing seems to match the size of what has been reserved

vdec: video-decoder@c8820000 {
    compatible = "amlogic,gx-vdec";
    reg = <0x0 0xc8820000 0x0 0x10000>,
          <0x0 0xc110a580 0x0 0xe4>;
    reg-names = "dos", "esparser";

    interrupts = <GIC_SPI 44 IRQ_TYPE_EDGE_RISING>,
         	    <GIC_SPI 32 IRQ_TYPE_EDGE_RISING>;
    interrupt-names = "vdec", "esparser";

    amlogic,ao-sysctrl = <&sysctrl_AO>;
    amlogic,canvas = <&canvas>;
};
vpu: vpu@d0100000 {
    compatible = "amlogic,meson-gx-vpu";
    reg = <0x0 0xd0100000 0x0 0x100000>,
          <0x0 0xc883c000 0x0 0x1000>,
          <0x0 0xc8838000 0x0 0x1000>;
    reg-names = "vpu", "hhi", "dmc";
    interrupts = <GIC_SPI 3 IRQ_TYPE_EDGE_RISING>;
    #address-cells = <1>;
    #size-cells = <0>;
    amlogic,canvas = <&canvas>;

    /* CVBS VDAC output port */
    cvbs_vdac_port: port@0 {
        reg = <0>;
    };

    /* HDMI-TX output port */
    hdmi_tx_port: port@1 {
        reg = <1>;
            hdmi_tx_out: endpoint {
            remote-endpoint = <&hdmi_tx_in>;
        };
    };
};

 

 

Share this post


Link to post
Share on other sites

A more interesting section in meson-gx.dtsi:

reserved-memory {
	#address-cells = <2>;
	#size-cells = <2>;
	ranges;

	/* 16 MiB reserved for Hardware ROM Firmware */
	hwrom_reserved: hwrom@0 {
		reg = <0x0 0x0 0x0 0x1000000>;
		no-map;
	};

	/* 2 MiB reserved for ARM Trusted Firmware (BL31) */
	secmon_reserved: secmon@10000000 {
		reg = <0x0 0x10000000 0x0 0x200000>;
		no-map;
	};

	/* Alternate 3 MiB reserved for ARM Trusted Firmware (BL31) */
	secmon_reserved_alt: secmon@5000000 {
		reg = <0x0 0x05000000 0x0 0x300000>;
		no-map;
	};

	/* 32 MiB reserved for ARM Trusted Firmware (BL32) */
	secmon_reserved_bl32: secmon@5300000 {
		reg = <0x0 0x05300000 0x0 0x2000000>;
		no-map;
	};

	linux,cma {
		compatible = "shared-dma-pool";
		reusable;
		size = <0x0 0x38000000>;
		alignment = <0x0 0x400000>;
		linux,cma-default;
	};
};

I presume this is reserving 53MB for ROM and ATF?

 

The linux cma section seems to be declaring 896MB of shared RAM but I dont understand what the alignment declaration does and none of these numbers match up with what seems to be allocated for the GPU

Share this post


Link to post
Share on other sites

i was looking into this a while ago for a s905w tv box and found out, that a lot of memory seems to be blocked already by legacy u-boot etc. (about 120m) and you cannot get it back by changes on linux kernel side. what helps is to chainload mainline u-boot (just build the libretech-cc u-boot from 2019.07 and use the resulting u-boot.bin):

 

fatload mmc 0 0x01000000 u-boot.bin
go 0x01000000

 

in legacy u-boot and then you can use regular boot.scr or extlinux to boot further from there using the chainloaded u-boot just booted.

 

good luck and best wishes - hexdump

Share this post


Link to post
Share on other sites
On 8/8/2019 at 3:32 AM, qstaq said:

I presume this is reserving 53MB for ROM and ATF?

The reg property is used to declare where the device have their memory-mapped region to interact with the operating system. You can imagine that devices (like the ethernet, the gpu, etc...) have one or more "windows" in the virtual address space of the CPU. Normally these windows are fixed in the address space, and the device tree declares where those windows are, letting the operating system drivers know where to access the devices.

 

In case of reserved-memory section, I see no real drivers involved there, so I guess that the memory is already reserved by bootloaders (the amlogic proprietary bootloader, mainly) and the declarations there inform the kernel not to use those locations because they are reserved.

You could remove the reserved sections, probably the kernel will show that more memory is available, but you may also expect malfunctions because some application may write in those reserved locations causing unpredictable behaviour!

 

Also check the size of the reserved CMA block (dmesg | grep cma). It should be possible to resize it via extraargs in armbianEnv.txt or recompiling the kernel. 8 megabytes there should suffice.

 

edit: I checked on a S905 device, the CMA can be changed only editing the device tree and should not be less than 32M, otherwise X fails to start on my configuration.

Share this post


Link to post
Share on other sites

Thanks @jock

 

I have created a new dtb with the following change:

 

linux,cma 
	{
	compatible = "shared-dma-pool";
	reusable; size = <0x0 0x2000000>;
	alignment = <0x0 0x400000>;
	linux,cma-default;
    };

This should reduce the cma size to 32MB (0x2000000) if I understand correctly?

 

However even with that change I still see "cma: Reserved 256 MiB" in dmesg, not 32MB as I would expect. I have also tried various different "cma=x" values on the kernel command line but the argument seems to be ignored

 

Have I misunderstood something?

Share this post


Link to post
Share on other sites
10 minutes ago, qstaq said:

Thanks @jock

 

I have created a new dtb with the following change:

 


linux,cma 
	{
	compatible = "shared-dma-pool";
	reusable; size = <0x0 0x2000000>;
	alignment = <0x0 0x400000>;
	linux,cma-default;
    };

This should reduce the cma size to 32MB (0x2000000) if I understand correctly?

 

However even with that change I still see "cma: Reserved 256 MiB" in dmesg, not 32MB as I would expect. I have also tried various different "cma=x" values on the kernel command line but the argument seems to be ignored

 

Have I misunderstood something?

Yes it should work that way. At least, it worked on my S905 machine. Didn't you forget to recompile the dts back to dtb, did you?

I'm not sure, but the reusable flag there should allow the kernel to use the same CMA region for its own needs, so it should act totally as a shared dynamic memory. You can anyway take a look to the official documentation becuase I may miss something: https://www.kernel.org/doc/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt

 

 

Share this post


Link to post
Share on other sites

Im de-compiling the existing dbt with:

 

dtc -I dtb -O dts meson-gxl-s905x-nexbox-a95x.dtb > custom.dts

then modifying custom.dts and re-compiling with:

 

dtc -O dtb -o custom.dtb custom.dts

Then changing uEnv.ini to point at custom.dts 

 

Edit: Working Now :)

 

OK so setting the FDT option in uEnv.ini doesnt do anything on S905x devices, you have to change the FDT in extlinux.conf instead (balbes images)

Share this post


Link to post
Share on other sites
On 8/8/2019 at 4:48 PM, hexdump said:

i was looking into this a while ago for a s905w tv box and found out, that a lot of memory seems to be blocked already by legacy u-boot etc. (about 120m) and you cannot get it back by changes on linux kernel side. what helps is to chainload mainline u-boot (just build the libretech-cc u-boot from 2019.07 and use the resulting u-boot.bin):

 

fatload mmc 0 0x01000000 u-boot.bin
go 0x01000000

 

in legacy u-boot and then you can use regular boot.scr or extlinux to boot further from there using the chainloaded u-boot just booted.

 

good luck and best wishes - hexdump

Hi @hexdump

 

I have built the libretech-cc u-boot and copied it to my /boot fs. How do I instruct legacy u-boot, installed on an sd card, to chainload the libretech u-boot?

 

Thanks for the help

Share this post


Link to post
Share on other sites

by just putting the above lines into your s905_autoboot.cmd and creating a fresh s905_autoboot from it. then it will on boot start the mainline u-boot instead of the kernel. and you need a working  extlinux/extlinux.conf file then which tells the mainline what to do.

 

good luck - hexdump

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
2 2