# Orange Pi Zero 3 (1.5GB) Random Crashes — Root Cause Found & Fixed (No Recompilation Required) **Board:** Orange Pi Zero 3 — 1.5GB LPDDR4 variant **SoC:** Allwinner H618 **OS:** Armbian Trixie **Kernels tested:** 6.6.75-legacy, 6.18.33-current, 6.16.8-edge **Status:** ✅ SOLVED — 72+ hours stable uptime confirmed --- ## The Problem My Orange Pi Zero 3 (1.5GB model) was randomly crashing every few hours under all three Armbian kernel branches. The symptoms were: - System freezes with no response (SSH drops, ping still works briefly) - Kernel oops logged: `kswapd0 → ext4_es_scan → paging request fault` - Random segfaults on normal binaries (`sudo`, `curl`, `apt`) - Corrupted library names in memory (e.g. `libGap-Jg.sK.0` instead of `libcrypto.so.3`) - `memtester` passed clean (2 loops, 1GB) - `badblocks` found zero bad blocks - `fsck` found no filesystem errors - Crashes happened even at full idle, with only ~200MB RAM in use The crashes occurred on **all three kernel branches** — legacy, current, and edge — ruling out a kernel-specific bug as the root cause. --- ## Investigation ### What we ruled out (in order) | Suspect | Test | Result | |---|---|---| | SD card physical failure | `badblocks -sv` | 0 bad blocks | | Filesystem corruption | `fsck -f` | Clean | | RAM hardware failure | `memtester 1G x2` | No errors | | eth-optimize.sh (ring buffer 1024, txqueuelen 10000) | Removed from crontab | Extended stability but didn't fix | | Kernel version | Tested 6.6.75, 6.18.33, 6.16.8 | All crashed | | DRAM size detection bug | `dmesg \| grep Memory:` | Consistently showed 1572864K (correct 1.5GB) | ### The kernel oops — the real clue ``` kernel: Unable to handle kernel paging request at virtual address 14ae620800000018 kernel: CPU: 0 PID: 56 Comm: kswapd0 kernel: Call trace: kernel: __es_tree_search.isra.0+0x20/0xa4 kernel: es_reclaim_extents+0x58/0xf0 kernel: ext4_es_scan+0xf0/0x29c kernel: do_shrink_slab+0x174/0x298 kernel: shrink_slab+0xb4/0x2f0 kernel: kswapd+0x18c/0x3b8 ``` The address `14ae620800000018` is in "address between user and kernel address ranges" — a classic corrupted/invalid pointer. This happens when the kernel accesses memory that physically doesn't exist or wraps around. ### DRAM address wraparound — the root cause The H618 1.5GB LPDDR4 variant has a known DRAM address wraparound issue, documented in the linux-sunxi community (notably by **ag123** in Armbian forums and **Jernej Skrabec's** u-boot size scan patches): > *"It is quite possible the addresses wrap around in the 1.5GB LPDDR4 DRAM chips and the 'test' for memory there returns a false positive."* In this specific case, the **size** is correctly detected as 1.5GB — but the **upper address region** (above 1GB physical) exhibits wraparound behavior. When the kernel accesses the upper ~512MB region (via page cache, kswapd reclaim, or slab allocation), writes to those addresses silently corrupt data in the lower region — because the addresses "wrap" back. This explains why: - `memtester` passes (it tests sequentially in a contiguous region, doesn't trigger wraparound patterns) - Size appears correct (1572864K = 1.5GB every boot) - Crashes are random in timing (depends on when page cache/kswapd reaches the upper region) - ALL kernels are affected (this is a u-boot/hardware level issue, not kernel-specific) --- ## The Fix ### Standard approach (requires u-boot recompilation) The proper fix is `CONFIG_DRAM_SUN50I_H616_TRIM_SIZE=y` in u-boot, or Jernej Skrabec's adjusted size scan procedure (columns-first scanning). This correctly detects and excludes the wraparound region, allowing full 1.5GB usage. ### Our approach — kernel `mem=` parameter (no recompilation needed) Since the wraparound occurs in the **upper address region**, we can simply tell the kernel to only use the safe lower 1GB: Edit `/boot/armbianEnv.txt` and add `mem=1024M` to `extraargs`: ```bash sudo nano /boot/armbianEnv.txt ``` Change: ``` extraargs=... existing params ... ``` To: ``` extraargs=mem=1024M ... existing params ... ``` Reboot and verify: ```bash cat /proc/cmdline | grep -o "mem=1024M" cat /proc/iomem | grep "System RAM" free -h ``` Expected output: ``` mem=1024M 40080000-7fffffff : System RAM total used free Mem: 974Mi ... ``` The `iomem` map confirms the kernel's physical RAM range is **exactly** `0x40080000–0x7fffffff` — stopping at the 1GB boundary. The upper wraparound region (`0x80000000` and above) is completely excluded from the kernel's memory map. ### Why this works The wraparound region begins at or above `0x80000000` (1GB physical offset). With `mem=1024M`: - Kernel allocator, page cache, slab, and all processes stay within `0x40000000–0x80000000` - `kswapd` never reclaims pages from the problematic upper region - The corrupted-pointer crash path is never triggered ### Tradeoffs vs. u-boot fix | | mem=1024M | u-boot TRIM fix | |---|---|---| | Requires recompilation | ❌ No | ✅ Yes | | RAM available | ~974MB | ~1.4GB | | Works on all kernels | ✅ Yes | ✅ Yes | | Fixes root cause | Workaround | Proper fix | For SDR servers, home automation, and similar light workloads where ~974MB is more than sufficient (our system uses ~130–200MB), this is a perfectly viable permanent solution. --- ## Results | Metric | Before fix | After fix | |---|---|---| | Uptime | 1–8 hours max | **72+ hours** (and counting) | | Kernel oops | Multiple per session | **Zero** | | Segfaults | Random, on any binary | **Zero** | | `sudo apt update` | Crashed frequently | Runs perfectly | | Memory corruption | Frequent bit-flips | **None observed** | System configuration at time of testing: - **Kernel:** 6.16.8-edge-sunxi64 - **Kernel param:** `mem=1024M` - **CPU governor:** ondemand - **Workload:** OpenWebRX+, SpyServer, RTL-TCP (x2), radiosonde_auto_rx, Xray proxy --- ## How to apply ```bash # Backup first sudo cp /boot/armbianEnv.txt /boot/armbianEnv.txt.bak # Add mem=1024M to extraargs sudo sed -i 's/^extraargs=/extraargs=mem=1024M /' /boot/armbianEnv.txt # Verify grep extraargs /boot/armbianEnv.txt # Reboot sudo reboot ``` After reboot, confirm: ```bash # Should show mem=1024M cat /proc/cmdline | grep -o "mem=1024M" # Should show ~974Mi total free -h # Should show System RAM ending at 0x7fffffff cat /proc/iomem | grep "System RAM" ``` --- ## References - ag123's analysis of 1.5GB wraparound: Armbian Community Forums - Jernej Skrabec's u-boot DRAM size scan fix: [linux-sunxi mailing list](https://www.mail-archive.com/u-boot@lists.denx.de/msg516769.html) - H618 LPDDR4 support patch: [u-boot mailing list](https://lore.kernel.org/all/20231111091000.26744-2-iuncuim@gmail.com/T/) - ag88's 1.5GB u-boot fix (alternative approach): GitHub --- ## Notes This fix was developed and tested on a **1.5GB LPDDR4** Orange Pi Zero 3. The 1GB, 2GB, and 4GB variants use different DRAM configurations and may not be affected by this specific issue. If you have a 1.5GB board and experience similar random crashes, try `mem=1024M` before spending time on kernel debugging or hardware replacement. It may save you days of investigation. **Tested and confirmed working by:** Özgür Çetinoğlu **Location:** Athens, Greece **Setup:** 24/7 SDR server (OpenWebRX+, radiosonde tracking) **Date:** June 2026

@Taz the sources for that repository is a clone of my build. It’s a very basic attempt to build a 6.19 kernel. You are better off using my build. If you got 6.6 booting then 6.18 should work as well.

@sven-ola About the issue you saw (re. UUID=63ee7593-e111-4547-ac2f-6bdb8519ce11..), what I have done is flash either of the two images at https://sven-ola.commando.de/privat-in/ to a 128GB MicroSD card using Win32DiskImager or Rufus on Windows, and then booted the RV2 with it, that's all. The RV2 has had two PCI devices connected at boot. This should not affect the boot sequence. I sent you all kinds of weird boot sequences I had especially with the Linux 7.10 Armbian image. I gave a few hours to try these Armbian images on the RV2, and ultimately it's not stable. The biggest issue is to boot from the M.2 SSD. Then, the 6.18 boots well from MicroSD but the 7.10 not. Finally my SFP+ NIC doesn't work well on the 6.18. I had more success with the testing in the beginning, e.g. I got the 7.10 image to work booted from the SSD. Could there be an issue with power supply, for example with the 5V to 3.3V converter on the PCB. For power supply I use a good 5V @ 5A USBC power supply. The most successful test I did was, flash a MicroSD card with the 6.18 image, and boot it on the RV2. Then from inside that environment, do dd if=Armbian-unofficial_26.05.0-trunk_Orangepirv2_trixie_edge_7.1.0-rc3_minimal.img of=/dev/nvme0n1 bs=100M status=progress; sync . Booting from that NVMe did work a few times. I installed the u-boot bundled with Armbian to the SPI using armbian-config. I'm not sure this was a good idea, at least it did not make NVMe booting work better.

Then wipe spi and retry. You can also pull a backup, simply via dd

I'm having trouble booting my Orange Pi5 off an SD Armbian_26.5.1_Orangepi5_trixie_vendor_6.1.115_minimal.img. I just get a blank screen. Older versions of Armbian do boot off an SD card, it also boots off NVME. I do have an old version of U-Boot flashed on SPI (Thanks to Joshua Riek), which I use because it is compatible with my KingSpec NVME drives, other versions of U-Boot did not recognize KingSpec NVME, including Armbian versions of U-Boot. I last tested Armbian U-Boot about 6 months ago, it failed. The Console log looks normal-ish right up to the point it just stops. Console Log --- DDR 9fffbe1e78 cym 24/02/04-10:09:20,fwver: v1.16 LPDDR4X, 2112MHz channel[0] BW=16 Col=10 Bk=8 CS0 Row=16 CS1 Row=16 CS=2 Die BW=16 Size=2048MB channel[1] BW=16 Col=10 Bk=8 CS0 Row=16 CS1 Row=16 CS=2 Die BW=16 Size=2048MB channel[2] BW=16 Col=10 Bk=8 CS0 Row=16 CS1 Row=16 CS=2 Die BW=16 Size=2048MB channel[3] BW=16 Col=10 Bk=8 CS0 Row=16 CS1 Row=16 CS=2 Die BW=16 Size=2048MB Manufacturer ID:0x1 CH0 RX Vref:28.5%, TX Vref:19.8%,20.8% CH1 RX Vref:29.3%, TX Vref:20.8%,20.8% CH2 RX Vref:28.5%, TX Vref:21.8%,21.8% CH3 RX Vref:29.7%, TX Vref:22.8%,21.8% change to F1: 528MHz change to F2: 1068MHz change to F3: 1560MHz change to F0: 2112MHz out U-Boot SPL board init U-Boot SPL 2017.09 (Aug 31 2024 - 15:22:22) unknown raw ID 41 18 20 Trying to boot from MMC2 spl: partition error Trying fit image at 0x4000 sector ## Verified-boot: 0 ## Checking atf-1 0x00040000 ... sha256(7612223b82...) + OK ## Checking u-boot 0x00800000 ... sha256(642bfeda4e...) + OK ## Checking fdt-1 0x008d6c48 ... sha256(7b2c4c6dbe...) + OK ## Checking atf-2 0x000f0000 ... sha256(b2af21b504...) + OK ## Checking atf-3 0xff100000 ... sha256(70505bb764...) + OK Jumping to U-Boot(0x00800000) via ARM Trusted Firmware(0x00040000) Total: 266.68/489.129 ms --- +