Odroid HC-1 died after apt-update/upgrade


Recommended Posts

First I have to thank all maintainers for their work. I ran this server for 1.5 years before something broke. Since armbian is maintained by volunteers I want to cheer you up and thank you for your hard work.

 

However just today my little server would die after apt-update/upgrade. I use a SATA drive together with a boot SD-card (only containing /boot). I can observe the following behavior:

 

The system leds indicate that booting is sucessful (blue LED does blink regularly, I can detect HDD usage by observing the green LED), yet SSH login that I have been using for years now is impossible. The device is also not responsive when pinging it. From my router's dashboard I can not detect a sucessful network connection, yet the NIC's orange LED would blink regularly. Sadly I do not own an UART adapter, maybe I should have bought one sooner.

 

What I have already done:

 

Mirror the SD card using dd. I am uploading the file and will post the link here for review. // Upload finished: https://mega.nz/file/yYJwEIiA#zw4lBfeJ8wXyxS1EeGDpMGERbQquEoxvNaCh2CFwTbg

Notice that the focal images seem to be corrupt. I have added focal sources since I needed more modern packages for installing ntopng. Maybe this had something to do with it.

 

I am very glad if someone has some advice, well any advice. I would hate to setup everything again ;)

Link to post
Share on other sites
Want Armbian t-shirt or a cup?

I'm in the same trouble.

 

It boot well on my SDcard then i do a rsync of the root to the encrypted ssd change the boot.ini to boot to the ssd as everytime i did.

~ # sudo sed -e 's/root=[^ ]*/root=\/dev\/sdg\/root cryptopts=source=\/dev\/sda,target=sdc,lvm=sdg /' -i /boot/boot.ini
~ # sudo update-initramfs -u

then it boot first on the initramfs well and i have access to the busybox ssh.

then I execute cryptroot_unlock it seems to start initializing well but without network, so i can't access.

 

I have access to the ssd from the initramfs using /sbin/cryptsetup open ...

i can mount and view file on the ssd from the initramfs.

 

Manualy tried to init with root_switch. it kick me then it the same networking problem.

~ # /sbin/cryptsetup open --type luks /dev/sda sdc
Enter passphrase for /dev/sda: 
~ # /sbin/lvm
lvm> vgchange -ay
  3 logical volume(s) in volume group "sdg" now active
lvm> exit
  Exiting.
~ # mkdir /newroot
mkdir: can't create directory '/newroot': File exists
~ # mount /dev/sdg/root /newroot
~ # mount --move /sys /newroot/sys
~ # mount --move /proc /newroot/proc
~ # mount --move /dev /newroot/dev
~ # exec switch_root /newroot /sbin/init

I'm on HC1 Armbian_20.08.1_Odroidxu4_buster_current_5.4.61_minimal

 

What do you need to help me to debug ?

Edited by rosbeef
typing errors
Link to post
Share on other sites

just what i do after flashing armbian on my sd.


 

# $erver
mkdir /etc/dropbear-initramfs/

apt install armbian-config vim rsync lvm2 cryptsetup dropbear-initramfs busybox nmap thin-provisioning-tools lxc nftables ebtables libxml2-utils iproute2 bridge-utils uidmap unattended-upgrades zram-tools

# €lient
cat .ssh/id_*.pub
# $erver
vi /etc/dropbear-initramfs/authorized_keys

#change ssh ports
sed -e "s/#Port 22/Port 2222/g" -i /etc/ssh/sshd_config
sed -e "s/^#CRYPTSETUP=/CRYPTSETUP=y/" -i /etc/cryptsetup-initramfs/conf-hook
sed -e "s/^#DROPBEAR_OPTIONS=/DROPBEAR_OPTIONS=\"-p 22222\"/" -i /etc/dropbear-initramfs/config

armbian-config
# change specific hardware config hc1 - reboot

------------------------------------------------
reboot or physically reboot
---------------------------------------------


test connectivity and halt
make disk copy with the disk linux tool

#statick mac based ip addr
Debian
export ETHID=$(nmcli -t -f device c);
export UUID=$(nmcli -t -f uuid c);
sudo nmcli con add ifname brW type bridge con-name brW;
sudo nmcli con add type bridge-slave ifname $ETHID master brW;
sudo nmcli con modify brW bridge.stp no;
sudo nmcli con down $UUID;
sudo nmcli con up brW;
sudo nmcli con modify brW ipv6.addr-gen-mode stable-privacy;
sudo nmcli con show;


sudo cryptsetup luksFormat --type luks2 /dev/sda

sudo cryptsetup luksOpen /dev/sda sdc

sudo pvcreate --dataalignmentoffset 1s /dev/mapper/sdc
sudo vgcreate sdg /dev/mapper/sdc
sudo lvcreate -L 16G /dev/mapper/sdg -n root
sudo lvcreate -L 200G /dev/mapper/sdg -n data
sudo lvcreate -l 100%FREE /dev/mapper/sdg -n save

sudo mkfs.ext4 /dev/sdg/root -E discard
sudo mkfs.ext4 /dev/sdg/data -E discard
sudo mkfs.ext4 /dev/sdg/save -E discard

sudo cryptsetup luksHeaderBackup /dev/sda --header-backup-file /boot/luksHeaderBackup.hex

-----------------------
Copy the root partition to SSD
-----------------------

sudo mount /dev/sdg/root /mnt
sudo rm -rf /mnt/*

sudo mkdir /mnt/data
sudo mount /dev/sdg/data /mnt/data

sudo mkdir /mnt/data/home
sudo mkdir /mnt/data/lxc
sudo mkdir /mnt/home

sudo mount --bind /mnt/data/home /mnt/home

sudo rsync -ahPHAXx --delete --exclude={/boot/*,/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found} / /mnt

sudo touch /mnt/etc/default/dropbear
echo 'NO_START=1' | sudo tee /mnt/etc/default/dropbear
# CRYPTTAB
echo 'sdc /dev/sda none luks,initramfs,discard' | sudo tee -a /mnt/etc/crypttab

# FSTAB
# comment root based on disk uuid
sudo sed -e 's/UUID=/#UUID=/' -i /mnt/etc/fstab

# mount root on root logical volume
sudo sed -e '1i/dev/sdg/root / ext4 defaults,noatime,nodiratime,commit=600,errors=remount-ro 0 1\' -i /mnt/etc/fstab

# create and mount at startup folder for boot files
sudo mkdir /mnt/media/sdcard
sudo sed -e '2i/dev/mmcblk0p1 /media/sdcard ext4 defaults,noatime,nodiratime,commit=600,errors=remount-ro 0 2\' -i /mnt/etc/fstab
sudo sed -e '3i/media/sdcard/boot /boot none default,bind 0 2\' -i /mnt/etc/fstab

# create and mount at startup folders for lxc privileged and unprivileges containers on data logical volume
sudo sed -e '4i/dev/sdg/data /data ext4 defaults,noatime,nodiratime,commit=600,errors=remount-ro 0 2\' -i /mnt/etc/fstab
sudo sed -e '5i/data/home /home none bind 0 0\' -i /mnt/etc/fstab
sudo sed -e '6i/data/lxc /var/lib/lxc none bind 0 0\' -i /mnt/etc/fstab

# remove root login permission
sudo sed -e 's/^PermitRootLogin yes/PermitRootLogin no #yes/' -i /mnt/etc/ssh/sshd_config
sudo cp /boot/boot.ini /boot/boot.ori
sudo rm /boot/boot.new

sudo sed -e 's/root=[^ ]*/root=\/dev\/sdg\/root cryptopts=source=\/dev\/sda,target=sdc,lvm=sdg /' -i /boot/boot.ini
sudo update-initramfs -u

#then reboot
sudo reboot

then ssh my server on root@ip -p 22222

on the server in initrramfs i run cryptroot_unlock

then it tell me that unlock successfull then kickme

Edited by rosbeef
Link to post
Share on other sites

I tried from a fresh image flashed on sdcard and just

sudo mkfs.ext4 /dev/sda -E discard
sudo mount /dev/sda /mnt
sudo rsync -ahPHAXx --delete --exclude={/boot/*,/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found} / /mnt
sudo sed -e 's/UUID=[^ ]*/UUID=MY_SDA_SSD_SPECIFIC_UUID"/' -i /boot/boot.ini

That does not change the problem

Link to post
Share on other sites

It seems that the mmcblk number change from 0 to 1 so my boot folder didn't mount, so automatic "sudo update-initramfs -u" doesn't work.

i started from scratch and replace

sudo sed -e '2i/dev/mmcblk0p1 /media/sdcard ext4 defaults,noatime,nodiratime,commit=600,errors=remount-ro 0 2\' -i /mnt/etc/fstab

by

sudo sed -e '2i/dev/mmcblk1p1 ...

What should be the origin of this change ?

Edited by rosbeef
Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...