Luna the Deer Posted Sunday at 03:38 AM Posted Sunday at 03:38 AM This started happening fairly recently. If my Docker containers are left running for a period of time, this message will start appearing over and over in journalctl: dockerd[1491]: time="2025-12-27T20:55:44.808894213-05:00" level=error msg="Error writing log message" driver=json-file error="error writing log entry: write /var/lib/docker/containers/97e823567c56600b69888e6abf63189c6f586afbf6bcf06fca820eaf4ef46f7d/97e823567c56600b69888e6abf63189c6f586afbf6bcf06fca820eaf4ef46f7d-json.log: no space left on device" message= After a while, these two messages will start repeating (these were taken from a different instance of this happening, so the PID is different): rsyslogd[599]: action 'action-0-builtin:omfile' (module 'builtin:omfile') message lost, could not be processed. Check for additional error messages before this one. [v8.2504.0 try https://www.rsyslog.com/e/2027 ] rsyslogd[599]: file '/var/log/syslog'[7] write error - see https://www.rsyslog.com/solving-rsyslog-write-errors/ for help OS error: No space left on device [v8.2504.0 try https://www.rsyslog.com/e/2027 ] Looking at btop when this is happening shows that the rsyslog and dockerd processes are using up most of the CPU, along with fail2ban for some reason. The full command for dockerd is: /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock Here are all my docker packages: docker-buildx-plugin/trixie,now 0.30.1-1~debian.13~trixie arm64 [installed] docker-ce-cli/trixie,now 5:29.1.3-1~debian.13~trixie arm64 [installed] docker-ce/trixie,now 5:29.1.3-1~debian.13~trixie arm64 [installed] docker-compose-plugin/trixie,now 5.0.0-1~debian.13~trixie arm64 [installed] 0 Quote
Luna the Deer Posted 1 hour ago Author Posted 1 hour ago After having this happen again and investigating further, it looks like my container for transmission is writing to a log file that is just ballooning bigger and bigger until the system crashes. root@modron:/var/lib/docker/containers/899f42253bb7772351219b7a65ff21347455f11950f983fb7a8883c6b93cdb17# ls -al total 101656680 drwx--x--- 4 root root 4096 Dec 27 22:28 . drwx--x--- 22 root root 4096 Dec 27 22:28 .. -rw-r----- 1 root root 104095768576 Dec 31 15:08 899f42253bb7772351219b7a65ff21347455f11950f983fb7a8883c6b93cdb17-json.log drwx------ 2 root root 4096 Dec 27 22:28 checkpoints -rw------- 1 root root 5949 Dec 27 22:28 config.v2.json -rw------- 1 root root 1776 Dec 27 22:28 hostconfig.json -rw-r--r-- 1 root root 13 Dec 27 22:28 hostname -rw-r--r-- 1 root root 172 Dec 27 22:28 hosts drwx--x--- 2 root root 4096 Dec 27 22:28 mounts -rw-r--r-- 1 root root 336 Dec 27 22:28 resolv.conf -rw-r--r-- 1 root root 71 Dec 27 22:28 resolv.conf.hash I'm not sure what this file is. Tailing it prints this out, and I'm worried trying to load it into memory with less would cause a crash: {"log":"[warn] Error from accept() call: No file descriptors available\n","stream":"stderr","time":"2025-12-31T20:06:58.854663511Z"} {"log":"[warn] Error from accept() call: No file descriptors available\n","stream":"stderr","time":"2025-12-31T20:06:58.854793008Z"} {"log":"[warn] Error from accept() call: No file descriptors available\n","stream":"stderr","time":"2025-12-31T20:06:58.855042376Z"} {"log":"[warn] Error from accept() call: No file descriptors available\n","stream":"stderr","time":"2025-12-31T20:06:58.855144456Z"} {"log":"[warn] Error from accept() call: No file descriptors available\n","stream":"stderr","time":"2025-12-31T20:06:58.855165456Z"} {"log":"[warn] Error from accept() call: No file descriptors available\n","stream":"stderr","time":"2025-12-31T20:06:58.855213288Z"} {"log":"[warn] Error from accept() call: No file descriptors available\n","stream":"stderr","time":"2025-12-31T20:06:58.855254412Z"} {"log":"[warn] Error from accept() call: No file descriptors available\n","stream":"stderr","time":"2025-12-31T20:06:58.855304285Z"} {"log":"[warn] Error from accept() call: No file descriptors available\n","stream":"stderr","time":"2025-12-31T20:06:58.855320618Z"} {"log":"[warn] Error from accept() call: No file descriptors available\n","stream":"stderr","time":"2025-12- I've been running the 4.0.5 branch of the linuxserver.io container for transmission, I'm updating to their "latest" branch and seeing if that improves anything. 0 Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.