Kernel 5.x.x and rk3399

[Merblud]

There are several questions on the topic:

1. Between the moment of transfer of control to the kernel from the u-boot and the moment of start of the kernel operation it takes 1.5 -2 minutes. I assume that during this period of time there is decompression of the code and placement in memory. But why so long? The 4.4.x kernel starts instantly. When I first turned on a nanopc-t4 with a 5.x.x (4.20) kernel, I already thought the kernel was corrupted. And quite accidentally waited for the green LED to blink.
2. Why is there still no sound in fdt for nanopc-t4? I corrected fdt, but I can 't put it in kernel.org. It is possible that my patch goes against the established structure, but now I use sound. Maybe there are some subtle nuances that I 'm not aware of?
3. Why can 't the kernel turn off power? The 4.4.x kernel was fine. It 's a flaw in fdt, in the kernel, something else?

Thanks.

[drice]

4 hours ago, Merblud said:

Between the moment of transfer of control to the kernel from the u-boot and the moment of start of the kernel operation it takes 1.5 -2 minutes. I assume that during this period of time there is decompression of the code and placement in memory. But why so long? The 4.4.x kernel starts instantly. When I first turned on a nanopc-t4 with a 5.x.x (4.20) kernel, I already thought the kernel was corrupted. And quite accidentally waited for the green LED to blink.

 

It appears to be decompression of initramfs taking 31 seconds:

 

[    6.147142] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    6.148094] PCI: CLS 0 bytes, default 64
[    6.158734] Trying to unpack rootfs image as initramfs...
[   35.413752] Freeing initrd memory: 7632K
[   35.462578] hw perfevents: enabled with armv8_cortex_a53 PMU driver, 7 counters available
[   35.482492] hw perfevents: enabled with armv8_cortex_a72 PMU driver, 7 counters available

I suspect this is due to the CPU starting at a very slow clock speed. The speed of these algorithms should be much faster:

 

[    4.104906] raid6: neonx8   gen()    26 MB/s
[    4.172463] raid6: neonx8   xor()    21 MB/s
[    4.242370] raid6: neonx4   gen()    27 MB/s
[    4.309700] raid6: neonx4   xor()    21 MB/s
[    4.379941] raid6: neonx2   gen()    20 MB/s
[    4.446522] raid6: neonx2   xor()    19 MB/s
[    4.517607] raid6: neonx1   gen()    12 MB/s
[    4.583751] raid6: neonx1   xor()    14 MB/s
[    4.653424] raid6: int64x8  gen()    14 MB/s
[    4.721122] raid6: int64x8  xor()     9 MB/s
[    4.790970] raid6: int64x4  gen()    15 MB/s
[    4.860366] raid6: int64x4  xor()    10 MB/s
[    4.934680] raid6: int64x2  gen()    10 MB/s
[    5.003469] raid6: int64x2  xor()     8 MB/s
[    5.079205] raid6: int64x1  gen()     6 MB/s
[    5.151255] raid6: int64x1  xor()     6 MB/s
[    5.151948] raid6: using algorithm neonx4 gen() 27 MB/s

I don't know how to check or control the initial speed of the CPU when the system is booting.

[piter75]

20 hours ago, Merblud said:

Between the moment of transfer of control to the kernel from the u-boot and the moment of start of the kernel operation it takes 1.5 -2 minutes. I assume that during this period of time there is decompression of the code and placement in memory. But why so long? The 4.4.x kernel starts instantly..

 

IMHO, as @drice suggested it has to do with the big CPUs starting at low frequency:

cpufreq: cpufreq_online: CPU4: Running at unlisted freq: 12000 KHz
cpufreq: cpufreq_online: CPU4: Unlisted initial frequency changed to: 408000 KHz

It was emphasised by this commit: https://github.com/torvalds/linux/commit/97df3aa76b4a384e29668f374a8b1034a31aa215 which makes a preference to use to use big cores instead of LiTTLE ones.

 

There are some patches to make it work better:

https://gitlab.manjaro.org/manjaro-arm/packages/core/linux-aarch64-rc/commit/129dd4053ddca867b0255bd7e4cff109b66bf61a

https://gitlab.collabora.com/rockpi/linux/commit/e55cd521ed67943cc3edcd8b78884ad53741da8c

 

I have used the latter in my local build on Rock Pi 4 and it works well - boot times in 5.3-rc4 are reduced.

I did not make a PR for it as it is a hack really but I sure can to make the booting experience smoother.

 

20 hours ago, Merblud said:

Why can 't the kernel turn off power? The 4.4.x kernel was fine. It 's a flaw in fdt, in the kernel, something else?

 

This should be fixed in 5.2 and reapplied for 5.3-rc4 with the following patch: https://github.com/armbian/build/blob/master/patch/kernel/rockchip64-dev/RK808-use-syscore-for-poweroff.patch

It works on Rock Pi 4 and in principle should on all other RK3399 / RK808 based boards, including NanoPC T4.

[Merblud]

On 9/2/2019 at 7:51 PM, drice said:

It appears to be decompression of initramfs taking 31 seconds:

Thanks. The thought of decompressing of initramfs didn 't come to my mind right away. But it takes me less than 30 seconds. But I did not pay attention to it:

[    2.333097] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    2.333648] HugeTLB registered 32.0 MiB page size, pre-allocated 0 pages
[    2.334066] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    2.334454] HugeTLB registered 64.0 KiB page size, pre-allocated 0 pages
[   65.467118] alg: No test for lzo-rle (lzo-rle-generic)
[   65.474155] alg: No test for lzo-rle (lzo-rle-scomp)
[   65.653691] ACPI: Interpreter disabled.

But this kernel is not from armbian. This is my local build (5.2.8). I probably didn 't take anything into account.

[Merblud]

2 piter75

 

Thanks. I 'll try to use it all.

 

9 hours ago, piter75 said:

It was emphasised by this commit: https://github.com/torvalds/linux/commit/97df3aa76b4a384e29668f374a8b1034a31aa215 which makes a preference to use to use big cores instead of LiTTLE ones.

I understand correctly that the hard setting of the performance value in the FDT is a bad solution. The kernel calculates the performance of the cores at start-up. What's the problem? Logic does not provide that cores can be different?

[Merblud]

9 hours ago, piter75 said:

There are some patches to make it work better:

https://gitlab.manjaro.org/manjaro-arm/packages/core/linux-aarch64-rc/commit/129dd4053ddca867b0255bd7e4cff109b66bf61a

https://gitlab.collabora.com/rockpi/linux/commit/e55cd521ed67943cc3edcd8b78884ad53741da8c

 

I have used the latter in my local build on Rock Pi 4 and it works well - boot times in 5.3-rc4 are reduced.

I did not make a PR for it as it is a hack really but I sure can to make the booting experience smoother.

 

I understand correctly that here the maximum values for the performance of the cores at the start are forced to be setted? Or something else?

[piter75]

2 hours ago, Merblud said:

I understand correctly that the hard setting of the performance value in the FDT is a bad solution. The kernel calculates the performance of the cores at start-up. What's the problem? Logic does not provide that cores can be different?

Setting the "performance index / capacity" in device tree is IMO a good thing per se - it instructs the kernel's scheduler which cores are more performant and which could / should process more tasks.

The issue here is that on boot big cores' clock is not initialised properly, they perform actually much worse than LiTTLE cores and kernel is scheduling more tasks to them as it expects them to be faster.

Before 5.2 the cpus were treated as equal so although not as short as 4.4 the boot times were not annoying. The change just emphasised the issue IMHO.

 

2 hours ago, Merblud said:

I understand correctly that here the maximum values for the performance of the cores at the start are forced to be setted? Or something else?

Manjaro's patch reverted the performance segregation of the cores to bring back pre 5.2 behaviour.

Ezequiel's patch OTOH tries to initialise cpufreq driver as soon as possible during the boot to make the cores' performance closer to the kernel's assumption of it - at least that's how I understand it.

[Mathias]

Just out of curiosity (since I've also noticed this annoying behavior): the big cores' clocks should be initialized by the kernel at start or by the bootloader (uboot)? Ie is it something that should be fixed in the DTS or in uboot?

 

Mathias

[Merblud]

On 9/3/2019 at 11:57 PM, piter75 said:

Manjaro's patch reverted the performance segregation of the cores to bring back pre 5.2 behaviour.

Ezequiel's patch OTOH tries to initialise cpufreq driver as soon as possible during the boot to make the cores' performance closer to the kernel's assumption of it - at least that's how I understand it.

If the kernel is built for a multiarch configuration. Can 't these patches cause the kernel to become inoperable on other SoC (not rk3399)?

[madscientist42]

On 9/6/2019 at 1:56 AM, Merblud said:

If the kernel is built for a multiarch configuration. Can 't these patches cause the kernel to become inoperable on other SoC (not rk3399)?

Multi-arch doesn't compensate for board differences...it's for things like 32 and 64-bitness being on the host system at the same time.

As for becoming inoperable on some other SoC...  A kernel built for SoC <X> may not have drivers, configuration, etc. built for SoC <Y>....to the point of being inoperable.  You shouldn't be trying to boot random kernels on random chips.  The only reason why X86 doesn't "work" that way is because there's a LOT of plug and play.  A tuned kernel for a given system setup there, though, CAN be inoperable on another board/system as well.  The fact that it's "tuned" is your first hint there.  A detuned setup like what Rasbian ships right now boots on every Pi because they chose to build for Original/PiZeros and limited everything to as much overlapped as possible.  In other words, they picked 32-bits ARMv6 architecture which is doable on all the chips with a moderate to major performance hit as you go up through the generations of boards.  They will all boot Raspbian, period.

The same can't be said of the Yocto derived distribution I am making after hours or for the one I'm making for Motorola Solutions, Inc. during the day.  WHEN I build for a Pi, and I do for varying reasons, I'm specific.  The build for a Pi2 won't boot for a PiZero, because it's tuned for a Pi2, which is to say it's a Quad Cortex-A7 system with a slightly different peripheral layout on the SoC than the Zero has.  The same can be said about a Pi3 or a Pi4 in either 32 or 64-bit mode, with the 64-bit modes increasing performance by up to 40% over the 32-bit ones.  Because I am pulling out more performance out of the build and SoC, the thing won't boot properly or at all on the other Pis.  The SAME can be said for any of the other SoC's you care to mention in the ARM or even X86 spaces.  The more optimized the kernel is for the specific hardware, the less likely it will boot elsewhere within the CPU architecture that the SoC belongs in.

I don't see the patches themselves doing that sort of thing.  But they won't work the same on a non-big/LITTLE core system either.  This is one of those things that lets the OS do the right things when you have hotter or weaker clusters in a given Coreplex.

Edited October 19, 2019 by madscientist42
Tapped enter too soon.

[madscientist42]

On 9/5/2019 at 10:53 AM, Mathias said:

Just out of curiosity (since I've also noticed this annoying behavior): the big cores' clocks should be initialized by the kernel at start or by the bootloader (uboot)? Ie is it something that should be fixed in the DTS or in uboot?

 

Mathias

Probably.

[Hover]

@MerbludWould you please publish you sound patch here? I am trying to get rt5651 sound of nanopc t4 on mainline kernel 5.4rc, but only static noise in headphone so far.

 

Btw bluetooth is broken on 5.3 or 5.4rc, while it's working on 5.2. Any idea?

 

With panfrost and hantro vpu marching on well, rk3399 seems becoming a good choice to run mainline kernel.

 

[Merblud]

Here 's the patch (5.3.4 kernel).

diff -u a/arch/arm64/boot/dts/rockchip/rk3399-nanopi4.dtsi b/arch/arm64/boot/dts/rockchip/rk3399-nanopi4.dtsi
--- a/arch/arm64/boot/dts/rockchip/rk3399-nanopi4.dtsi	2019-10-17 23:47:33.000000000 +0300
+++ b/arch/arm64/boot/dts/rockchip/rk3399-nanopi4.dtsi	2019-10-27 22:34:55.988303874 +0300
@@ -105,6 +105,27 @@
 		};
 	};
 
+	rt5651-sound {
+		compatible = "simple-audio-card";
+		simple-audio-card,name = "realtek,rt5651-codec";
+		simple-audio-card,format = "i2s";
+		simple-audio-card,mclk-fs = <256>;
+		simple-audio-card,widgets =
+			"Microphone", "Mic Jack",
+			"Headphone", "Headphone Jack";
+		simple-audio-card,routing =
+			"Mic Jack", "MICBIAS1",
+			"IN1P", "Mic Jack",
+			"Headphone Jack", "HPOL",
+			"Headphone Jack", "HPOR";
+		simple-audio-card,cpu {
+			sound-dai = <&i2s0>;
+		};
+		simple-audio-card,codec {
+			sound-dai = <&rt5651>;
+		};
+	};
+
 	sdio_pwrseq: sdio-pwrseq {
 		compatible = "mmc-pwrseq-simple";
 		clocks = <&rk808 1>;
@@ -184,6 +205,10 @@
 	status = "okay";
 };
 
+&hdmi_sound {
+	status = "okay";
+};
+
 &i2c0 {
 	clock-frequency = <400000>;
 	i2c-scl-rising-time-ns = <160>;
@@ -432,6 +457,16 @@
 	i2c-scl-rising-time-ns = <150>;
 	i2c-scl-falling-time-ns = <30>;
 	status = "okay";
+
+	rt5651: rt5651@1a {
+		compatible = "rockchip,rt5651";
+		reg = <0x1a>;
+		clocks = <&cru SCLK_I2S_8CH_OUT>;
+		clock-names = "mclk";
+		hp-det-gpio = <&gpio4 RK_PC4 GPIO_ACTIVE_LOW>;
+		spk-con-gpio = <&gpio0 RK_PB3 GPIO_ACTIVE_HIGH>;
+		#sound-dai-cells = <0>;
+	};
 };
 
 &i2c2 {
@@ -459,6 +494,16 @@
 	status = "okay";
 };
 
+&i2s0 {
+	rockchip,playback-channels = <8>;
+	rockchip,capture-channels = <8>;
+	status = "okay";
+};
+
+&i2s2 {
+	status = "okay";
+};
+
 &io_domains {
 	bt656-supply = <&vcc_1v8>;
 	audio-supply = <&vcca1v8_codec>;
@@ -724,3 +769,9 @@
 &vopl_mmu {
 	status = "okay";
 };
+
+&spdif {
+	i2c-scl-rising-time-ns = <450>;
+	i2c-scl-falling-time-ns = <15>;
+	status = "okay";
+};

But the patch merely indicates to the kernel what equipment is present. We need another driver for the rt5651. Kernel sources have a module rt5651.c. I haven 't figured it out yet. You might need some more rk3399-specific module.

[Hover]

@MerbludThanks. But still no progress with rt5651. Maybe the module in mainline to be blamed. I know nothing about kernel sound driver, it seems beyond my touch.

 

Btw, although panfrost works with glmark2-es2 well, it bends knees to a simple draw, causing a segfault in rockchip_dri.so which called memset(). So it's still prealpha.

 

And no success to get vidconsole on mainline uboot, even worse the tpl-spl won't load u-boot.itb on sd. I have to resort to an old spl without tpl. Anyone got vidconsole in uboot?

[Marko Buršič]

I have a RockPi 4, installed Armbian Buster Server 5.3.11. same problem, more than 30s to unpack, while on 4.4 kernel I haven't noticed so long times at boot up.

[Merblud]

17 hours ago, Marko Buršič said:

I have a RockPi 4, installed Armbian Buster Server 5.3.11. same problem, more than 30s to unpack, while on 4.4 kernel I haven't noticed so long times at boot up.

If you use a uboot builded according to scheme (tpl/spl)->uboot (without rockchip miniloader), the loading is fast.

[Hover]

Glad to know the mainline u-boot progress. Give it another try. Now the tpl-spl loads u-boot.itb. It seems aclk-vop and hclk-vop rate setting are missing. But with these rate settings added only got a fake vidconsole, only registered the stdio driver without any signal sent to hdmi. There is something missing to enable the hdmi correctly although it's probed successfully.

 

Another mystery: the kernel would hang in very early stage after board was soft reset, but boot well after hard reset or poweron.

[Marko Buršič]

On 11/24/2019 at 9:47 AM, Merblud said:

If you use a uboot builded according to scheme (tpl/spl)->uboot (without rockchip miniloader), the loading is fast.

I use the u-boot that comes on image file. How to install "this" tpl/spl u-boot?

[Igor]

1 hour ago, Marko Buršič said:

How to install "this" tpl/spl u-boot?

If u-boot is build and packed by our build tools:

- install .deb package as usual (dpkg -i xxxxxxxxxxxxxxxx.deb and remove potentially conflicting u-boot package if exists)

- armbian-config -> system -> update boot loader

[Merblud]

4 hours ago, Igor said:

If u-boot is build and packed by our build tools:

- install .deb package as usual (dpkg -i xxxxxxxxxxxxxxxx.deb and remove potentially conflicting u-boot package if exists)

- armbian-config -> system -> update boot loader

Is there a package uboot in the armbian without miniloader?

[Merblud]

6 hours ago, Marko Buršič said:

I use the u-boot that comes on image file. How to install "this" tpl/spl u-boot?

I built the uboot myself according to instructions (from git). It 's not very difficult. But the latest versions of the uboot don 't work for me.

[Igor]

1 hour ago, Merblud said:

Is there a package uboot in the armbian without miniloader?

Don't recall / know / follow. It probably depends of the board.

[balbes150]

Does anyone have a 5651 sound running on T4 ?

 

 

The module 5651 is built , but it is not perceived by the system with the added settings in the DTB for 5651. But if I change the codec used in the settings from 5651 to 8316 (to check the DTB parameters), everything is detected and the 8316 module is automatically loaded.

[hev]

On 9/2/2019 at 8:05 PM, Merblud said:

There are several questions on the topic:

1. Between the moment of transfer of control to the kernel from the u-boot and the moment of start of the kernel operation it takes 1.5 -2 minutes. I assume that during this period of time there is decompression of the code and placement in memory. But why so long? The 4.4.x kernel starts instantly. When I first turned on a nanopc-t4 with a 5.x.x (4.20) kernel, I already thought the kernel was corrupted. And quite accidentally waited for the green LED to blink.

https://github.com/heiher/linux/commit/0a2aa023e6180b6d0416ed61310a57ce24d03dba

1. Why is there still no sound in fdt for nanopc-t4? I corrected fdt, but I can 't put it in kernel.org. It is possible that my patch goes against the established structure, but now I use sound. Maybe there are some subtle nuances that I 'm not aware of?
2. Why can 't the kernel turn off power? The 4.4.x kernel was fine. It 's a flaw in fdt, in the kernel, something else?

Thanks.

 

[Merblud]

It is possible to sum up the results.
A full power cut is working. I don 't know how to do this in fresh versions of the kernel. It 's possible it 's already working by default. Did not check. But I have a patch that solves this problem.

The sound works. The sound through HDMI works in the latest versions of kernel. The changes were only needed in fdt. The problem with outputting audio through the codec has also been solved.

Orange Pi 4 Kernel 5.x.x rt5651 sound and bluetooth fixed
It works. A different fdt is needed for the NanoPC-T4.

The latest versions of the u-boot load the kernel quickly.

[Marko Buršič]

My personal thoughts (Rockpi 4B):

I have been experimenting both legacy 4.4.y (rockcip64- ayufan) and current 5.4.y (linux-mainline). The legacy kicks on in 15 seconds, while the newer one takes more than 3 minutes. I wanted to ad some device via DT overlay. Well in legacy kernel it fails any attempt to modify .dtb or add a .dtbo  , further time to time it throws kernel exceptions that I haven't figured out what's behind. On the other side we have 5.4.y kernel that is slow to boot, but it can handle lots of abusing. Tried with device tree overlays, if something is wrong it reverts back or it throws exceptions, but almost never fail. 

I already disabled I2S0, because it shares same GPIOs as different device on board, I have enabled SPI1 and added two devices with help of cs-gpios patch: 1st flash on SPI1.0, and a 2nd a SPI device with cs-gpio on SPI1.1. The newer kernel is flexible, but slow to boot ("langsam aber sicher").

[Merblud]

Try a newer version of u-boot.

[piter75]

6 hours ago, Marko Buršič said:

The newer kernel is flexible, but slow to boot ("langsam aber sicher").

Can you build yourself an image with build system?

After this commit ROCK Pi 4 is switched to mainline u-boot in Armbian, which contains cpu clock fixes, and rebooting the board gets pleasant again ;p

[balbes150]

In kernel 5.7 without u-boot changes, the startup time is fixed.

[piter75]

2 hours ago, balbes150 said:

 

In kernel 5.7 without u-boot changes, the startup time is fixed.

 

I wonder what was actually changed to fix this.

Nonetheless we will keep our changes in u-boot for the "current" target which is v5.4.x

[balbes150]

1 hour ago, piter75 said:

I wonder what was actually changed to fix this.

The clock or priority of using cores may have been fixed.