Root partition thinks it's full

[pkfox]

Hi all, I've just booted my NanoPi M4 V2 after not using it for a year or so and when I log in via ssh I'm told the / usage is at 100% - I've deleted all logs  and the usual suspects but to no avail

uname gives

Linux M4V203 5.10.63-rockchip64 #21.08.2 SMP PREEMPT Wed Sep 8 10:57:23 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

I'm using a 32gb SD card

df -h gives

Filesystem      Size  Used Avail Use% Mounted on
udev            1.9G     0  1.9G   0% /dev
tmpfs           387M   41M  346M  11% /run
/dev/mmcblk1p1   29G   29G     0 100% /
tmpfs           1.9G     0  1.9G   0% /dev/shm
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           1.9G     0  1.9G   0% /sys/fs/cgroup
tmpfs           1.9G  4.0K  1.9G   1% /tmp
/dev/nvme0n1p1  458G   73M  434G   1% /mnt/Docker
/dev/nvme0n1p2  458G  280G  155G  65% /mnt/music
/dev/zram0       49M   20M   26M  44% /var/log
tmpfs           387M     0  387M   0% /run/user/0

 

any ideas ?

[going]

1 час назад, pkfox сказал:

any ideas ?

Remove the SD card and use the SD-USB adapter. Use another computer with a Linux OS.
Check the file system on the sections of the SD card.
This is a common Linux issue. There are a lot of instructions on the Internet.

[pkfox]

Thanks @going what command do I use to check the file system once I have the sdcard in an adapter ?

[going]

26 минут назад, pkfox сказал:

what command do I use to check the file system

set command:

> ls /usr/sbin | grep fsck
> sudo fsck --help
> man fsck

dmesg | tail
.....
[29283.479644]  sdf: sdf1
[29283.482102] sd 4:0:0:0: [sdf] Attached SCSI removable disk
[29283.882104] EXT4-fs (sdf1): mounted filesystem with writeback data mode. Opts: errors=remount-ro. Quota mode: none.

fs is mounted!!
> df -h
....
/dev/sdf1          7,2G        1004M  6,1G           14% /run/media/leo/armbi_root
> sudo umount /run/media/leo/armbi_root
> sudo fsck.ext4 /dev/sdf1
e2fsck 1.46.4 (18-Aug-2021)
armbi_root: clean, 28006/470112 files, 295400/1907712 blocks

it is good

 

The first three lines are for reference

dmesg | tail - Get information where the SD card was mounted

The disk with the SD card file system must be unmounted.

If the fsck team finds bad blocks, it will ask questions.

They must be answered in the affirmative. Sometimes it takes a long time.

 

> sudo fsck.ext4 -fcy /dev/sdf1
e2fsck 1.46.4 (18-Aug-2021)
Checking for bad blocks (read-only test): done                                                 
armbi_root: Updating bad block inode.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information

armbi_root: ***** FILE SYSTEM WAS MODIFIED *****
armbi_root: 28006/470112 files (0.1% non-contiguous), 295400/1907712 blocks

fsck.ext4 -fcy - it is forced to check and fix (It took about 7 minutes).

[pkfox]

Thank you very much for your detailed instructions I will try them when I get home and let you know how I get on

[pkfox]

Hi @going I ran fsck

fsck.ext4 -fcy /dev/sdb1
e2fsck 1.44.5 (15-Dec-2018)
Checking for bad blocks (read-only test): done
/dev/sdb1: Updating bad block inode.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information

/dev/sdb1: ***** FILE SYSTEM WAS MODIFIED *****
/dev/sdb1: 87173/1869120 files (0.8% non-contiguous), 7650362/7709728 blocks

so I mounted it

mount /dev/sdb1 /mnt/m4v203

and checked disk space

 df -h /mnt/m4v203/
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb1        29G   29G     0 100% /mnt/m4v20

 

 

 

[going]

1 час назад, pkfox сказал:

mount /dev/sdb1 /mnt/m4v203

and checked disk space

 df -h /mnt/m4v203/
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb1        29G   29G     0 100% /mnt/m4v20

du -hs /mnt/m4v20/home

ls -al  /mnt/m4v20/

[pkfox]

@going the card is now back in the NanoPi and still reports that it is full up - thank you for your time and patience

 

When I ssh into m4v203 I see this

Quote

Welcome to Ubuntu 18.04.6 LTS with Linux 5.10.63-rockchip64

System load:   0.26 0.53 0.54   Up time:        3:48
Memory usage:  5 % of 3863MB    IP:            10.10.1.16
CPU temp:      35°C
Usage of /:    100% of 29G

[ General system configuration (beta): armbian-config ]

 

 

Edited February 1 by pkfox
Add text

[SteeMan]

It reports it is full, probably because it is full.  The last set of commands @going suggested you run are to try to figure out what has filled up the storage.  du -s  will be your tool in looking as where the storage is being used.  Google for how others use this command for this purpose.

[going]

1 час назад, pkfox сказал:

When I ssh into m4v203 I see this

du -hs /home

ls -al  /

😄 🤣 😁

[pkfox]

Hi @going I gave up an reflashed in the end - the du -hs /home command gave me 200k all I had in /home was a folder with .bashrc in it - there is no way that card was full - thanks for trying to help me

[going]

1 час назад, pkfox сказал:

I gave up an reflashed in the end - the du -hs /home command gave me 200k all I had in /home was a folder with .bashrc in it - there is no way that card was full - thanks for trying to help me

Sometimes there may be large files in the root (/*).
sometimes a lot of files can accumulate in the /var folder, for example, in the apt cache.
Just go through them all.
Run the 'mc' command. Go to the root of the file system. - find "show size" in the menu.
Find the folder with the maximum size. Go to it. Repeat "show size"

 

Edited February 2 by going
add screenshot_mc

[pkfox]

Pretty sure I did all that (without mc) and couldn't find anything of note - I've had this before and ended up reflashing - i would like to get to the bottom of it though.