usual user

While exploring my Info Center, I stumbled across this screen: If there are other users who use a Samsung SSD 980 PRO 2TB, they may also be interested in following this link: https://www.pugetsystems.com/support/guides/critical-samsung-ssd-firmware-update/

They are floating around. Some of them are in flight to land in mainline, some are WIP in heavy flux. In terms of functionality, mainline would not be able to do more than legacy so far. And since your offered legacy firmware works so far, there is currently no reason to do the work now to use the current mainline version. It is only interesting for someone who wants to add e.g. VOP2 or NVME driver support and ports the drivers from Linux kernel. For all pure users it is easier to wait until all outstanding components have landed and then just use mainline. Anyway, my SPEC file for the uboot-tools package is now ready to build M1 firmware as soon as needed. However, this will certainly take until 2023.07 at the earliest, because there are still some components under discussion and 2023.04 should be available next week.

Since @Igor don't like my source of knowledge, I'll leave it to him to answer such Armbian user queries now, because in his opinion sharing knowledge and having mainline support doesn't seem to be of any help to Armbian 😉

My first u-boot build:

Sure, with distro-boot only a suitable firmware in the firmware area and three files in a partition accessible from the firmware is required. Because I still have Petitboot in SPI flash, I used @rpardini's initial image as boot trampoline, because Petitboot requires a very special partition layout. Then the kernel binary, the DTB and extlinux.conf copied into the partition, and my operating system is entirely run by the NVME. It should be noted that the kernel binary must be uncompressed, as the legacy firmware does not even support compressed kernel binaries. I use a microSD card as boot trampoline, because the firmware only reads it, and I only mount it from the operating system read-write if I want to update the kernel binary or the DTB. But an eMMC works the same way. The only annoying thing about this method is the need to copy the kernel binary and DTB over, but this is a simple workaround until mainline firmware is available.

Thank you for collecting the diverse logs. Unfortunately, they only confirm my initial analysis. The issue occurs when the NVME initialization should take place. The first part of the PCI client initialization succeeds: [ 2.216527] pci 0000:00:00.0: [1d87:0100] type 01 class 0x060400 [ 2.217239] pci 0000:00:00.0: supports D1 [ 2.217613] pci 0000:00:00.0: PME# supported from D0 D1 D3hot [ 2.221587] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring while the next one: [ 2.222558] pci 0000:01:00.0: [1e4b:1202] type 00 class 0x010802 [ 2.223199] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00003fff 64bit] [ 2.223977] pci 0000:01:00.0: Max Payload Size set to 256 (was 128, max 512) [ 2.225062] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold [ 2.226049] pci 0000:01:00.0: 8.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x4 link at 0000:00:00.0 (capable of 31.504 Gb/s with 8.0 GT/s PCIe x4 link) triggers the issue in the event of an error. The cause remains unclear, but since two autonomous devices interact via a complex PCI protocol, further analysis can be very time-consuming. In any case, this requires a kernel developer who is familiar with the PCIe subsystem. It is known that not all conceivable hardware combinations will work, so other SBC manufacturers collect test reports to give their customers a guide which combinations can be used. This is the economically most favorable way to get a functioning system, development costs are usually way higher than the purchase of a suitable NVME SSD. But this is only a workaround solution. The fact that the error situation seems to depend on the runtime behavior does not make fixing any easier. In this case, a completely different cause may also be responsible. From the logs it can also be seen that even the firmware does not recognize the SPI flash on a case-by-case basis: Loading Environment from SPIFlash... jedec_spi_nor flash@0: unrecognized JEDEC id bytes: ff, ff, ff *** Warning - spi_flash_probe_bus_cs() failed, using default environment

To analyze this further, I need a detailed serial console log. Please select the "linux verbose" boot option by entering a "3" on the serial console while the boot options are displayed and the countdown runs. I also see that a different firmware is used and I don't know where it is loaded from as it is not the one included in the Armbian image. To identify the firmware as the culprit for the PCIe problem, it would be interesting to also see a log with the boot option "5.15.80-rockchip64" (4).

Here my final analysis. A kernel built with a completely different toolchain than Armbian's is now used. The kernel configuration varies in many respects from those of Armbian. My build is based on pure mainline sources where only these patches were applied, but these have no relevance for PCIe support. The test with my build has ruled out a shortcomming of Armbian due to their toolchain, configuration and applied patches. The difference between my rk3399 device and yours is the specific hardware and how it is wired via DTB. Since the PCIe support works for me, it can be assumed that the issue is triggered by your special hardware constellation. This suggests that a shortcomming exists in the kernel code. To analyze the problem further, someone familiar with the kernel PCIe code is now needed, but I doubt if anyone is available for free. There would have been a small chance for the DTB configuration as the cause, but since you preferred not to provide me with the full serial console log, I can not confirm this.

I don't expect you to switch to fedora, but every Linux user should be able to execute basic cli commands to copy a kernel into position. In this way, we can at least determine whether it is just a configuration issue or whether a bugfix is really needed. At the very least, you meet the necessary requirements, an affected device, and serial console access to investigate the issue further. I added my kernel build to the Armbian image for you and uploaded it here. Because I do not have an identical to your device to test, I can not guarantee that the result works at all. So download it and post the serial console log while it boots.

I run fedora as userspace but although fedora offers all mainline components available for the RockPi 4B, there is no out-of-the-box image. You need to know what is necessary to configure the system and how to set it up. Also, you have to add components that are not available in mainline yourself, such as firmware where the manufacturers think it is a good idea to formulate the distribution licenses in such a way that free software may not ship them out-of-the-box. The user space software components work fine, provided the kernel has the necessary mainline driver support. Out-of-mainline components like Wifi calibration parameter files must be added by yourself, i.e. YMMV. I'm sure my configured microSD card would work out-of-the-box for your device also, but the only thing I can offer is my kernel build, which is then integrated into the Armbian image of your choice to find out if it works for you.

The supplied logs prove that the kernel causes a "synchronous external abort" when the pcie sybsystem is initializing. The fact that both logs look similar is because both images use the same kernel and user space doesn't even come into play when the issue occurs. Since a rather old kernel is used, I would first switch to a more recent one to further identify the issue. With my rk3399 device I have been using NVME flawless for a long time. I therefore assume that there is no fundamental code problem. However, I am currently running 6.1.0, which contains improvements that were certainly not back-ported to 5.15.80. The rk3399 mainline kernel support is very advanced, so it doesn't make much sense to waste time analyzing a behaved kernel. Maybe the issue no longer exists with a kernel upgrade and that's all that's necessary.

TF-A is progressing slowly, but there seem to be still open questions. The DT rebase of u-boot has taken place. It still takes a rebase to be up to date, but this is only a small thing, because the basic structure conversion has already taken place. But I haven't seen any movement in uboot WIP trees that I'm aware of. IMHO they are waiting to see how the TF-A development turns out in the end to develop u-boot accordingly further. I will report when I get news. But the most important thing is that for the kernel only the rkvdec2 support is missing, and for the userspace only the FFmpeg v4l2-request support is missing to be usable out-of-the-box with pure mainline support for standard applications. It is thus available for all distributions without special effort for the Odroid-M1. They only have to maintain the distribution-specific modifications that they apply for other devices anyway. I'm currently playing with jernej's FFmpeg patches for v4l2-request support and VP8 acceleration seems to work. H.264 fails with this error: [h264_v4l2m2m @ 0xaaaabfc5ae10] Could not find a valid device [h264_v4l2m2m @ 0xaaaabfc5ae10] can't configure decoder But I don't know if the 5.1.2 paches basically work or how I can inspect the FFmpeg framework appropriately. With the gstreamer framework, however, everything works as expected.

Odroid-M1 support has just landed so will be available with 6.2.0-rc1 out-of-the-box. RGA support can be wired on top (v4l2-compliance.log), but the DMA deficiency for 32-bit addresses needs to be addressed for devices with more than 4GB of RAM.

For out-of-the-box mainline kernel support you need at least 6.1.0-rc1 because that's where all currently available code for the rk3568 SOC has landed. GPU support has been available for a long time, see glmark2-wayland-odroid-m1.log for reference. And Mali-G52 is official conformant for OpenGL ES 3.1. VPU support is only available for the hantro IP. So hardware-accelerated video decoding of H.264 and VP8 works up to 1080p. See fluster-run-odroid-m1.log for reference. The available drivers are in quite good shape, see v4l2-compliance.log for reference. The support for the rkvdec2 IP is not yet available, so no 4K resolution and no VP9 and HEVC hardware decoding for now. Hardware encoding is only available for JPEG in poor quality. But missing mainline encoder support applies almost for all SOCs. If you need 4K resolution, VP9 or HEVC codecs, you have to currently rely on the vendor's legacy kernel with proprietary interfaces.

Because the sysfs interface is only a wrapper for the character device interface to provide users with the legacy interface if they insist, this is to be expected. Since you did not say how the base DTB wired the pins, and I am too lazy to pick it out myself because I do not own this device, this is a very expected result. For example, I see from the information here that pin 32, 33 are occupied by default with the serial console (UART functionality). Applying overlays can reconfigure the default functionality, but not all pins are configured with GPIO functionality by default. Good vendors offer appropriate documentation for their provided software, but since the configuration depends on the active DTB, it is at best to be interpreted as a guide. Any other provider may have changed the configuration according to their ideas.

The method used to transfer an image byte by byte to a target device is not important. It does not change the function of the transferred code as long as no transmission errors occur. Due to the detailed description of the procedure and the confirmation that the system is working, it is proven that nothing has been done wrong so far. The problem only arises when the NVME device comes into play. The use of different images also proves that there seems to be a fundamental issue that seems worth investigating further. But a basic prerequisite are meaningful logs, and without them it is hopeless from the beginning. The excuse of not having a suitable USB serial adapter available to provide logs is mood. Every reasonably assorted electronics shop offers these for little money. Choose whatever your host system with which you want to use the adapter supports, make sure that the operating voltage of the RX/TX lines matches that of the SBC and the adapter supports a baud rate of 1500000 and you are ready to go. Tinkering with SBCs sooner or later requires access to the serial console for one reason or another. If you don't want that and have a turnkey system, an SBC isn't the right choice.

Providing the serial console logs of the boot process helps with error analysis and greatly increases the likelihood that the issue can be fixed.

gpioset --help

I doubt it, if mainline development continues at the same rate as before, the next WIP patches for the pending drivers will certainly be available at this time.

PINE64 does an excellent job documenting the state of development for mainline development. Although it is for the rk3566 SoC, it is one to one transferable for the rk3568 SoC. Except for the rkvdec2 support, the support is pretty much on par with the rk3399 SoC. And the good thing, everything has already landed in mainline and all userspace software that have support for the rk3399 SoC also work unmodified for the rk3568 SoC. So except for the 4K decoder support, devices with the rk3568 SoC are already quite suitable for everyday use. I am very confident that the rkvdec2 support is only a matter of time and then 4K media can be enjoyed impeccably. I would now only wish that all board manufacturers provide equivalent documentation for there devices, as PINE64 has shown here. This would simplify mainline usability immensely and one wouldn't have to deal with legacy forks, but I think that remains a pious wish.

4.19 legacy makes sense if you want to use functionalities for which there is no support in mainline for the rk3568 SoC yet. But this also requires special userspace software that can use these proprietary interfaces. 5.19 doesn't make much sense, because it requires backporting everything that has already landed in 6.1.0-rc. Especially if you use the tobetter tree to pull from because it is based on very early WIP patches that contain bugs that are already fixed in mainline. As far as I know, 6.1.0-rc contains all mainline code available for the rk3568 SoC. So there is currently nothing relevant to add. For the ODROID M1 support only the source for the rk3568-odroid-m1.dts is missing. This can be easily added by cherrypicking the relevant commits from the maintainer tree. Or you build from linux-next and have the full preview on 6.2.0-rc1. But from the point of view of the ODROID M1, this only adds the rk3568-odroid-m1.dts out-of-the-box. IMHO you should start with 6.1.0-rc for the ODROID M1 and mainline kernel support. Everything else is unnecessary work that does not provide any advantages over 6.1.0-rc. Of course, all distribution dependent additions can still be applied, but from the point of view of the ODROID M1 they are not absolutely necessary.

This error message indicates to me that the module does not support a certain functionality. Mainline releases tend to be more advanced than outdated forks with out-of-tree hacks that have never been ported to mainline. Now, if Mainline uses new eMMC features that are not supported by a module and even the recovery attempt fails, the legacy firmware may work better in this case. It does not trigger this circumstance. Since mainline development does not rely on the support of a special module, it can lead to such failures to the advantage of other modules that support these functions. You can now try to determine the cause and then have a fix incorporated into mainline if necessary, but that's out of my interest, since I don't have any eMMC modules. I provided my firmware builds just to be able to do a quick test to see if a fix has already been added to mainline, saving you the hassle of building it yourself or waiting for it to become available in Armbian. I build the firmware for me for other reasons anyway. This also saves Armbian the effort of quickly integrating a new version if there is no improvement anyway. But I think a prepared PR is still welcome to keep Armbian up to date if this does not lead to regressions. This is not entirely true. The dd command is the same for microSD and eMMC except for the "of=" parameter. Both are MMC devices and the block device data structure (MBR) is identical, only the hardware interface used is different. But this is handled by the corresponding drivers and is not important from the user's point of view. For SPI flash, that's a different story. SPI Flash does not necessarily use block device data structures and therefore the firmware range starts at offset zero. Therefore, the dd command for SPI flash looks like this: dd conv=notrunc,fsync if=u-boot-meson.bin of=/dev/mtdblock0 Alternatively, you can also use "flashcp".

I just looked at this page. To me it looks as if the colors are used to distinguish which devices they are intended for and with which OS system they are supplied. For the N2 I see only two colors to distinguish between Linux and Android. All eMMCs marked in orange appear to be intended for other devices. Could it be that this is the reason why they do not work reliably with the N2?

Since I have already uploaded 2022.10 GA at the request of someone else, it can be downloaded here if there is still a need. The u-boot GitLab is here. But I am building from the release tar ball with a fedora toolchain. In any case, you only get the u-boot.bin, which still has to be signed with the binary-only tools from the vendor u-boot tree and combined with the binary-blob artefacts to the FIP. In the case of Armbian, surely the better solution is for someone to adjust the Armbian building system to a more recent version and issue a pull request so that the change can be integrated into Armbian. He will certainly reap the thanks of many happy Armbian users. By the way, for another occasion I just verified that UEFI boot also works. See console.log for reference.

There is also 2022.07 GA available and if you insist I can also upload 2022.10 GA.