eMMc drive filled to 90% overnight and no idea why.

[Cpz]

My Helios has been running well for the most part stable for the past few weeks however last night i noticed when looking at my file systems in omv that my 14 GiB eMMC drive which houses my ambian install is at 13.13GiB and looks to be growing.  Normally this sits around the 5 GiB.  I have checked my docker install for anything suspicious but my plex docker is only taking up about 400mb. 

looking at my system logs i am getting a repeated this error which is new and might be related however i am not really sure what program is casing it.

 

Jun 10 17:32:27 localhost rsyslogd:  [origin software="rsyslogd" swVersion="8.1901.0" x-pid="1322" x-info="https://www.rsyslog.com"] rsyslogd was HUPed
Jun 10 17:32:28 localhost smbd[2572]: [2021/06/10 17:32:28.177454,  0] ../source3/param/loadparm.c:3362(process_usershare_file)
Jun 10 17:32:28 localhost smbd[2572]:   process_usershare_file: stat of /var/lib/samba/usershares/downloads failed. No such file or directory
Jun 10 17:32:30 localhost smbd[2572]: [2021/06/10 17:32:30.195229,  0] ../source3/param/loadparm.c:3362(process_usershare_file)
Jun 10 17:32:30 localhost smbd[2572]:   process_usershare_file: stat of /var/lib/samba/usershares/downloads failed. No such file or directory

 

 

I would really appreciate any help on how i can find out what is populating my drive so i can correct.

 

 Thank you. 

[Heisath]

Use a tool like ncdu to figure out which folder(s) are getting so big. Then look into them.

[Cpz]

21 hours ago, Heisath said:

Use a tool like ncdu to figure out which folder(s) are getting so big. Then look into them.

Thank you for your suggestion

 

 for that have installed ut and tried running it using 

ncdu -e

ncdu -x

 

however cant find anything apart from system files is there any other commands that i should try.  

[Cpz]

Just to add to this I am able to locate the emmc drive through df command but i um unable to access it.. please see below

 

Filesystem       1K-blocks       Used  Available Use% Mounted on
udev               1901936          0    1901936   0% /dev
tmpfs               395612       7400     388212   2% /run
/dev/mmcblk2p1    14843200   13393004    1268104  92% /
tmpfs              1978044          0    1978044   0% /dev/shm
tmpfs                 5120          0       5120   0% /run/lock
tmpfs              1978044          0    1978044   0% /sys/fs/cgroup
tmpfs              1978044          4    1978040   1% /tmp
folder2ram         1978044     103360    1874684   6% /var/log
folder2ram         1978044          0    1978044   0% /var/tmp
folder2ram         1978044        772    1977272   1% /var/lib/openmediavault/rrd
folder2ram         1978044        728    1977316   1% /var/spool
folder2ram         1978044      16748    1961296   1% /var/lib/rrdcached
folder2ram         1978044          4    1978040   1% /var/lib/monit
folder2ram         1978044       1604    1976440   1% /var/cache/samba
/dev/md0       11627192048 1934573924 9692601740  17% /srv/dev-disk-by-uuid-72c7802e-29b2-406a-81b6-0323b18e6ef3
tmpfs               395608          0     395608   0% /run/user/0
root@helios64:~# /dev/mmcblk2p1
-bash: /dev/mmcblk2p1: Permission denied
 

[Werner]

You are trying to execute a block device as a command which obviously does not work. The device is mounted as your root directory /

So Try ncdu -x /

[Heisath]

might need to do 

sudo ncdu -x /

 

Then you can navigate through a tree-view, and figure out where the size comes from. 

 

As a general step-by-step guide:

- use "df -h" (or something like that), to figure out which device / mountpoint is getting full

user@builder:~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           1,6G  1,2M  1,6G   1% /run
/dev/sda3        79G   31G   44G  41% /
tmpfs           7,9G     0  7,9G   0% /dev/shm
tmpfs           5,0M     0  5,0M   0% /run/lock
tmpfs           4,0M     0  4,0M   0% /sys/fs/cgroup
/dev/sda2        94M  5,2M   89M   6% /boot/efi
tmpfs           1,6G  136K  1,6G   1% /run/user/1000

We now know, /dev/sda3 is fullest, and is mounted on /

 

- next use "sudo ncdu -x /" to check the contents / where the space goes:

ncdu 1.15.1 ~ Use the arrow keys to navigate, press ? for help                  
--- / --------------------------------------------------------------------------
   17,2 GiB [##########] /home                                                  
    8,5 GiB [####      ] /usr
    2,0 GiB [#         ]  swapfile
    1,0 GiB [          ] /var
  760,0 MiB [          ] /root
  508,4 MiB [          ] /opt
  251,1 MiB [          ] /boot
   11,6 MiB [          ] /etc
   68,0 KiB [          ] /tmp
   36,0 KiB [          ] /snap
e  16,0 KiB [          ] /lost+found
    8,0 KiB [          ] /media
e   4,0 KiB [          ] /srv
e   4,0 KiB [          ] /mnt
e   4,0 KiB [          ] /cdrom
@   0,0   B [          ]  libx32
@   0,0   B [          ]  lib64
@   0,0   B [          ]  lib32
@   0,0   B [          ]  sbin
@   0,0   B [          ]  lib
@   0,0   B [          ]  bin
 Total disk usage:  30,1 GiB  Apparent size:  28,8 GiB  Items: 789697           

We see that most storage goes to /home, /usr and the swapfile.

 

As the interface is interactive, you can just navigate down the folder structure to find the culprit.

[Cpz]

Thank you Heisath and Werner for your help. It was something in my tmp folder that was growing. Nuked it and now everything is back to normal.