Dear all,
I have a strage problem here with handling large files and I ran out of ideas.
System:
Odroid C4 (running from SD-card)
OS: 5.7.15-meson64 #20.08 SMP PREEMPT
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
Attached to the system are a few USB disks:
3x 8TB, 1x 4TB, 1x 3TB, 1x 2TB
All filesystems: ext4 (except the remote NAS filesystem, which is ZFS - see below)
3 of these disks are encrypted via truecrypt 7.1
On the 3 TB disk (ext4, unencrypted) there is a zipped dd-image of a harddisk.
That file has about 760 GB size
Now I tried to copy this file to a different location.
1st try: rsync to a NAS filer (ZFS, OmniOS, NFSv4) -> The rsync process goes to "D" state after a couple of GB
2nd try: scp to the same NAS -> same result. Copies a couple of gigabytes and then freezes
3rd to 5th try: mounted the NAS via NFSv4 and used rsync, scp and cp to the mountpoint -> same result
6th and 7th try: Tried to copy to a local (truecrypted) 8TB disk using rsync and cp -> same result
All these attemts stopped after transferring a couple of gigabytes and then the involved processes showed "defunct" state.
The transferred size is different every time and is between 40 an 200 GBytes.
What I tried to pinpoint the problem:
Tried to read the big file:
- md5sum <big_imagefile> -> works!
- dd if=<file> of=/dev/null bs=100M -> works!
Tried to write a big file:
- dd if=/dev/zero of=<file_on_8TB_truecrypted_disk> bs=100M count=10000 -> works!
Tried to copy using dd:
- dd if=<the_image_file> of=<file_on_truecrypted_disk> bs=100M -> works! Checksum is ok.
So I finally managed to copy the file to a different location using dd.
But why is cp, scp and rsync failing?
Additional information:
During the process I did several experiments with the swapspace (disabled, used swapfile on SD-card, etc) nothing helped.
In the logfiles there is not the smallest hint to a problem.
Also observing the memory during copy using "free" command did not show any unusual state.
Systemlimits:
root@odroidc4:/var# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 11363
max locked memory (kbytes, -l) 65536
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 11363
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
No overclocking or something. Just a plain Armbian installation. No funny "tuning".
I also have the same problem with some other applications.
For example rmlint, for eliminating duplicates, runs for a while (30 minutes?) and then freezes.
After that I connected the USB disk to a different Linux system and rmlint worked fine.
Fun fact: I'm pretty sure I originally copied the long file from a "full size" Linux system via rsync
to the Odroid a couple of months ago.
Any ideas what the problem could be?
Thank you!
Lothar
