MattCostamagna

Hi @TRS-80, I'm the other guy of the "we" pronoun. The fact that my colleague is asking for help in a forum that is supposed to help people having trouble making stuff working seems pretty normal to me, even if the target is to make a commercial product. Open source projects (like Armbian) rely on a community of people helping each other out. I've never seen anybody on StackOverflow reply with "If you don't know how to do it, just hire somebody who does". Armbian is a very nice product with a well made documentation and we (at work) are using it on different projects because we think it's stable and reliable, and we always found this fourm and its topics helpful to us to solve our problems. Nobody is born knowing everything, so when I cannot figure how to solve a problem, I try finding a solution and if needed, I ask for help. I hope you understand. If nobody can't help us, we'll keep trying until we find a solution.

Yes you are right, making a new image won't take me that long and it's something I've been considering since I saw the sort of problem I was facing. I'll just leave this thread as unresolved so if someone in the future will manage to find a solution they can post the answer here. Thanks a lot for your support.

Thanks @TRS-80 That might be the reason (also because I've excluded all the other possible causes), but I don't understand what could be the cause... I didn't quite understand what you meant here: And yes, I can make a new version of the image of our project, but before doing that I wanted to be sure that I can't do anything else.

@Werner I meant that I tried flashing a vanilla version of Armbian into one of the SD cards that are not working with the production image I've been using, and with a vanilla image I don't see the errors that I see with the other image (which however works with the old SD cards). Hopefully that's more clear.

@TRS-80 Thanks for your reply. The SD cards are brand new and they've never been used. We have verified that all sectors are writable using h2testw and we've got zero errors. We also tried different SD cards coming from the same batch but the outcome was the same.

Hi @Werner I tried making an exact copy of the source image using etcher without modifying anything else. The validation of the written data was successful and I didn't get any errors, but unfortunately the result was the same: card stuck in programming state and sometimes I can't even reach the login.

Actually I didn't. I supposed that if dd doesn't return any error it means that the operation went succesfully. What can I do to verify it?

Hi, I've been using Armbian for three years on some work's project and this is the first time I see this issue. Basically I have an image of the operative system and all the programs and scripts already saved in it, so when I have to prepare a new SD card I just use the command "dd" and then I modify some registry stuff specific for the object I'm creating. It has always worked without problem, but now we bought a new batch of SD cards and I'm facing this issue: the dd command goes succesfully and if I mount the partitions after it has finished I can see all my data, but when I boot the system sometimes I get some errors. Here is the last part of the boot log: [ 12.866043] systemd[1]: Mounted Kernel Configuration File System. [ 13.948285] thermal thermal_zone0: binding zone cpu_thermal with cdev thermal-cpufreq-0 failed:-22 [ 93.570475] mmc0: Card stuck in programming state! mmc_do_erase [ 94.342442] sunxi-mmc 1c0f000.mmc: fatal err update clk timeout [ 95.110441] sunxi-mmc 1c0f000.mmc: fatal err update clk timeout [ 95.870438] sunxi-mmc 1c0f000.mmc: fatal err update clk timeout [ 95.877698] blk_update_request: I/O error, dev mmcblk0, sector 1863688 op 0x3:(DISCARD) flags 0x800 phys_seg 1 prio class 0 [ 95.889014] blk_update_request: I/O error, dev mmcblk0, sector 1281568 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0 [ 95.889652] EXT4-fs error (device mmcblk0p1): __ext4_find_entry:1532: inode #13767: comm fstrim: reading directory lblock 0 [ 95.900257] EXT4-fs error (device mmcblk0p2): __ext4_get_inode_loc:4699: inode #135743: block 524611: comm rc.local: unable to read itable block [ 95.911460] EXT4-fs (mmcblk0p1): I/O error while writing superblock [ 95.924553] EXT4-fs (mmcblk0p2): I/O error while writing superblock [ 95.937750] EXT4-fs error (device mmcblk0p1): __ext4_find_entry:1532: inode #13767: comm fstrim: reading directory lblock 0 [ 95.938429] EXT4-fs error (device mmcblk0p1): __ext4_find_entry:1532: inode #13767: comm find: reading directory lblock 0 [ 95.949147] EXT4-fs (mmcblk0p1): I/O error while writing superblock [ 95.950615] EXT4-fs error (device mmcblk0p2): __ext4_get_inode_loc:4699: inode #135743: block 524611: comm rc.local: unable to read itable block [ 95.950697] EXT4-fs (mmcblk0p2): I/O error while writing superblock [ 95.960128] EXT4-fs (mmcblk0p1): previous I/O error to superblock detected [ 95.967624] EXT4-fs error (device mmcblk0p1): __ext4_find_entry:1532: inode #13767: comm fstrim: reading directory lblock 0 [ 95.979752] EXT4-fs (mmcblk0p1): I/O error while writing superblock [ 95.985982] EXT4-fs (mmcblk0p1): previous I/O error to superblock detected [ 95.993275] EXT4-fs error (device mmcblk0p1): __ext4_find_entry:1532: inode #13767: comm find: reading directory lblock 0 [ 96.004308] EXT4-fs (mmcblk0p1): I/O error while writing superblock [ 96.010602] EXT4-fs (mmcblk0p1): previous I/O error to superblock detected [ 96.010702] EXT4-fs (mmcblk0p1): I/O error while writing superblock [ 96.018166] EXT4-fs error (device mmcblk0p1): __ext4_find_entry:1532: inode #13767: comm fstrim: reading directory lblock 0 [ 96.029509] EXT4-fs error (device mmcblk0p1): __ext4_find_entry:1532: inode #13767: comm find: reading directory lblock 0 [ 96.032375] EXT4-fs error (device mmcblk0p1): __ext4_find_entry:1532: inode #99: comm systemd-udevd: reading directory lblock 0 [ 96.032420] EXT4-fs (mmcblk0p1): I/O error while writing superblock [ 96.035115] EXT4-fs (mmcblk0p1): I/O error while writing superblock [ 96.094862] EXT4-fs error (device mmcblk0p1): __ext4_find_entry:1532: inode #13767: comm find: reading directory lblock 0 [ 96.094899] EXT4-fs error (device mmcblk0p1): __ext4_find_entry:1532: inode #13767: comm fstrim: reading directory lblock 0 [ 96.918443] sunxi-mmc 1c0f000.mmc: fatal err update clk timeout [ 97.674443] sunxi-mmc 1c0f000.mmc: fatal err update clk timeout [ 98.442444] sunxi-mmc 1c0f000.mmc: fatal err update clk timeout [ 99.210440] sunxi-mmc 1c0f000.mmc: fatal err update clk timeout [ 99.974443] sunxi-mmc 1c0f000.mmc: fatal err update clk timeout [ 100.730444] sunxi-mmc 1c0f000.mmc: fatal err update clk timeout [ 100.810651] debugfs: Directory 'mmcblk0' with parent 'block' already present! [ 125.415973] EXT4-fs error (device mmcblk0p1): __ext4_find_entry:1532: inode #7779: comm cron: reading directory lblock 0 [ 125.427247] EXT4-fs error (device mmcblk0p1): __ext4_find_entry:1532: inode #7779: comm cron: reading directory lblock 0 [ 125.438653] EXT4-fs error (device mmcblk0p1): __ext4_find_entry:1532: inode #46845: comm cron: reading directory lblock 0 [ 125.450113] EXT4-fs error (device mmcblk0p1): __ext4_find_entry:1532: inode #7779: comm cron: reading directory lblock 0 [ 125.461370] EXT4-fs error (device mmcblk0p1): __ext4_find_entry:1532: inode #7779: comm cron: reading directory lblock 0 [ 125.472605] EXT4-fs error (device mmcblk0p1): __ext4_find_entry:1532: inode #7779: comm cron: reading directory lblock 0 [ 125.483833] EXT4-fs error (device mmcblk0p1): __ext4_find_entry:1532: inode #46845: comm cron: reading directory lblock 0 Sometimes it manages to exit from the error state, other times it keeps logging the same error message. We have tested the SD cards using some broken sectors detector software, and none of them found any problems. The source image is working because if I use an old SD card, I don't see any errors. I tried dd-ing a vanilla version of Armbian into one of the SD cards that is giving me problems and it is working nicely. I tried the same "broken-OS SD cards" into differents OrangePI and none of them worked. The problem happens sometimes at boot, sometimes after a couple of minutes (I saw some kernel panic). This is the output of "uname -a": Linux smartbit 5.4.45-sunxi #20.05.3 SMP Wed Jun 10 12:09:20 CEST 2020 armv7l armv7l armv7l GNU/Linux The "dd" command (which is working on old SD cards) is the following: dd if=./latest.img of=/dev/sdd bs=1M conv=fsync So if the problem is in the source image, why is it working on the old SD cards batch? Otherwise, if the problem is in the SD card, why an Armbian vanilla image isn't giving any problems? Thanks a lot in advance for any help. Mattia

I found out that the networking.service has a param called TimeoutStartSec that defaults to 5min in /lib/systemd/system/networking.service. I lowered that value to 15 seconds and the problem is gone. Here's a useful command: sed -i -r 's/TimeoutStartSec=.*/TimeoutStartSec=15/' /lib/systemd/system/networking.service

Hi, I'm facing a problem with my Orange Pi Zero with the latest Armbian version. I noticed that when I provide an internet connection type that is not valid (wrong psk, ethernet but no cable plugged in, etc.) and I reboot the system, it hangs just before asking the username and password. Sometimes it takes up to 10 minutes before keeping going. These are the last boot logs that I see: [ 11.209226] systemd[1]: Starting Create list of static device nodes for the current kernel... [ 11.219428] systemd[1]: Condition check resulted in Load Kernel Module drm being skipped. [ 11.244988] systemd[1]: Started Nameserver information manager. [ 11.254315] systemd[1]: Reached target Network (Pre). [ 11.264815] systemd[1]: Condition check resulted in Set Up Additional Binary Formats being skipped. [ 11.274833] systemd[1]: Condition check resulted in File System Check on Root Device being skipped. [ 11.293312] systemd[1]: Starting Load Kernel Modules... [ 11.317312] systemd[1]: Starting Remount Root and Kernel File Systems... [ 11.343632] systemd[1]: Starting udev Coldplug all Devices... [ 11.376556] systemd[1]: Mounted POSIX Message Queue File System. [ 11.386689] systemd[1]: Mounted Kernel Debug File System. [ 11.396501] systemd[1]: Mounted Kernel Trace File System. [ 11.413706] systemd[1]: Finished Create list of static device nodes for the current kernel. [ 11.416094] EXT4-fs (mmcblk0p1): re-mounted. Opts: commit=600,errors=remount-ro [ 11.458472] systemd[1]: Finished Remount Root and Kernel File Systems. [ 11.472062] systemd[1]: Condition check resulted in Rebuild Hardware Database being skipped. [ 11.481213] systemd[1]: Condition check resulted in Platform Persistent Storage Archival being skipped. [ 11.505488] systemd[1]: Starting Load/Save Random Seed... [ 11.514898] systemd[1]: Condition check resulted in Create System Users being skipped. [ 11.553531] systemd[1]: Starting Create Static Device Nodes in /dev... [ 11.583442] g_serial gadget: Gadget Serial v2.4 [ 11.588174] g_serial gadget: g_serial ready [ 11.610410] systemd[1]: Finished Load Kernel Modules. [ 11.626029] systemd[1]: Finished Load/Save Random Seed. [ 11.650175] systemd[1]: Mounting FUSE Control File System... [ 11.671958] systemd[1]: Mounting Kernel Configuration File System... [ 11.694089] systemd[1]: Starting Apply Kernel Variables... [ 11.724736] systemd[1]: Mounted FUSE Control File System. [ 11.741343] systemd[1]: Finished Create Static Device Nodes in /dev. [ 11.751604] systemd[1]: Mounted Kernel Configuration File System. [ 11.776484] systemd[1]: Starting udev Kernel Device Manager... [ 12.843290] thermal thermal_zone0: binding zone cpu_thermal with cdev thermal-cpufreq-0 failed:-22 **Usually here after a couple of seconds I'm asked to provide the username and password** After that the system hangs for a very long time. I stopped and disabled NetworkManager-wait-online.service but nothing changed. Is there something else that prevents the boot phase to finish before having established the connection? Thanks a lot Mattia

@xwiggen Yes, this is something I didn't know before, so now I removed the tmpfs entry in fstab. When I did this yesterday, the copyctruncate in logrotate magically started working. I have no idea why the tmpfs conflicted with logrotate. I wanted to post this yesterday but the Armbian forum didn't let me because I reached the maximum number of posts I can make per day (1 I guess)... Yes I know but for the type of application I have to develop I don't care that much o logging because it's all saved on a cloud environment; local logs are useful to me only when everything else fails. Anyway now my logs are working as I expected. The only other thing that doesn't make sense to me is /var/log.hdd that is not updating. This is its current content: $ ls -halt /var/log.hdd/ -rw-r--r-- 1 root root 1.2K Jun 4 12:56 armbian-ramlog.log -rw-rw-r-- 1 root utmp 5.3K Jun 4 12:56 wtmp -rw-r----- 1 syslog adm 156K Jun 4 12:56 syslog -rw-r----- 1 syslog adm 6.0K Jun 4 12:55 auth.log -rw-r--r-- 1 root root 264K Jun 4 12:54 dpkg.log -rw-r----- 1 syslog adm 68K Jun 4 12:54 kern.log -rw-r--r-- 1 root root 24K Jun 4 12:54 faillog -rw-rw-r-- 1 root utmp 286K Jun 4 12:54 lastlog drwxrwxr-x 10 root syslog 4.0K Jun 4 12:54 . drwxr-xr-x 2 root root 4.0K Jun 4 12:54 apt drwxr-xr-x 2 root adm 4.0K Jun 4 12:49 nginx drwxr-xr-x 13 root root 4.0K Jun 4 12:49 .. -rw-r--r-- 1 root root 42K Jun 4 12:35 armbian-hardware-monitor.log -rw-r--r-- 1 root root 18K Jun 4 12:35 alternatives.log -rw-r--r-- 1 root root 118K Jun 4 12:35 bootstrap.log -rw-rw---- 1 root utmp 0 Jun 4 12:35 btmp -rw-r--r-- 1 root adm 38K Jun 4 12:35 dmesg drwxr-x--- 2 root adm 4.0K Jun 3 13:45 unattended-upgrades drwx------ 2 root root 4.0K May 28 18:53 private drwxr-sr-x 2 root systemd-journal 4.0K May 28 18:53 journal drwxr-xr-x 2 _chrony _chrony 4.0K Apr 20 13:58 chrony drwxr-xr-x 2 ntp ntp 4.0K Apr 2 17:37 ntpstats drwxr-xr-x 2 root root 4.0K Dec 23 2019 sysstat As you can see files are very old. I'm not an expert but I thing it should be periodically updated with the content of the /var/log directory, isn't it? I thought this was caused by the fact that I made the "/" partition read-only for security purpose, but I can write into that directory without any problem. So /var/log.hdd is not updating (which is kind of what I want, even though I don't like the fact that is working for reasons unknown to me), and if I mount the sd card into my computer to see the content of the partitions I made, the /var/log.hdd is empty. So is the content really stored on the physical memory? Too many mysteries... I haven't found any documentation on the internet about all of this, so if anybody knows where I can read something about how logs are managed by Armbian, please send it to me. Thanks a lot

@xwiggen Thank you for your reply. I tried doing what you suggested and it worked. I still don't understand why /var/log does not work while /var/log.hdd works. Now I have two problems: I configured /var/log as a tmpfs in /etc/fstab (tmpfs /var/log tmpfs nodev,nosuid 0 0) so that log files do not count as writes to the SD card. I previously didn't know about /var/log.hdd, so I think that now this doesn't make much sense. I'd still like to write logs in a way that do not spoil the SD card on the long term, how can I do it? How can I disable the writes to /var/log.hdd? Another strange thing is that in the Armbian version I was using before (where logrotate is triggered by a daily cron and not by systemd) I didn't have the copytruncate problem, now it seems that the problem is systemd because if I run /usr/sbin/logrotate /etc/logrotate.conf everything works as expected.

Hi everyone, I'm facing a problem with logrotate on my OrangePi Zero with Armbian Focal mainline. Basically I have a process that is always running with a `tee` command that outputs the stdout to a log file in the /var/log/ directory. My logrotate configuration file looks like the following: /var/log/smartbit/gocpp.log { rotate 10 daily compress missingok notifempty copytruncate dateext } It's all working correctly except the copytruncate part: logs are daily rotated and gzipped but the gocpp.log is not truncated to 0 bytes, so it gets bigger and bigger. I noticed that in this new Armbian version logrotate is not triggered by a cron but by systemd logrotate.service. If I run /usr/sbin/logrotate /etc/logrotate.conf, everything works included the copytruncate line, but with `systemctl start logrotate.service` the copytruncate line is not considered. Does anyone know why? Thanks in advance Mattia