1 1
ygoe

Is the ext4 journal really active?

Recommended Posts

Hello,

 

I've copied the Armbian Bionic image onto an SD card (and verified it) and put it into my new Orange Pi Zero LTS 256 MB. I already have experience with Raspberry Pis and Raspbian for a few years, so this isn't entirely new to me. While I was trying to establish a stable network connection (wired and wireless), I misinterpreted some "host not reachable" as "network failed" when instead other issues existed. In the process, I unplugged the power a few times after the device has booted up but wasn't reachable via the network.

 

Only then I connected the serial console and investigated further. There I found out that it wasn't actually booting anymore because it needed a manual filesystem check. When I ran that, a large number of errors was corrected. This was surprising because I thought ext4 fs journalling was active and should prevent such situations completely. It might still ask (not sure, but I haven't configured anything else yet), but it really shouldn't see dozens of errors.

 

To be sure, I reflashed (and verified) the SD card to have a clean filesystem again. Now I'm trying to figure out why this could happen and how I can prevent it in the future.

 

There are discussions on the web (including this forum) that explain the use of 'dumpe2fs' and 'tune2fs' and the options for the '/etc/fstab' file. But it doesn't make much sense in my case. According to 'dumpe2fs', the 'has_journal' flag is set and the journal should be active. 'tune2fs' cannot disable the journal because I cannot remount the root filesystem as read-only ("mount point is busy"). So I couldn't see what it looks like when the journal should be inactive. Then, I should be able to use the "data=journal" option in fstab to actually use the journal. But last time I did this, it couldn't mount the filesystem anymore and failed to boot. Correcting this from the outside is complicated (I have no other Linux box at hand) so I'd have to reflash the SD card again in this case.

 

Can somebody please explain to me what the real situation with the ext4 journal is? Should it be used by default or do I have to opt into it? If it is active, then why could the filesystem become so corrupted by plugging out the power? If it's not active, and should prevent this corruption at all, then how should I proceed to enable it?

 

Here's the output from 'armbianmonitor -u': http://ix.io/2o7w

 

BTW, why is the same device mounted twice to '/' and '/var/log.hdd'? How can this be possible? And should I ignore the second mount point?

Share this post


Link to post
Share on other sites
Armbian is a community driven open source project. Do you like to contribute your code?
On 6/2/2020 at 7:36 PM, ygoe said:

I was trying to establish a stable network connection (wired and wireless


This later, wireless, is not very good one on this board.

 

On 6/2/2020 at 7:36 PM, ygoe said:

Can somebody please explain to me what the real situation with the ext4 journal is? Should it be used by default or do I have to opt into it? If it is active, then why could the filesystem become so corrupted by plugging out the power?


Corruption is not a journal problem but garbage collection, timeout for writing things to the media. This performance tweak you can alter (commit) in /etc/fstab 

https://unix.stackexchange.com/questions/155784/advantages-disadvantages-of-increasing-commit-in-fstab Change it to 0 but to be fully safe from power cut corruption, you need read write file system.

Armbian defaults regarding fs:
https://github.com/armbian/build/blob/master/lib/debootstrap.sh#L467

 

On 6/2/2020 at 7:36 PM, ygoe said:

why is the same device mounted twice to '/' and '/var/log.hdd'? How can this be possible? And should I ignore the second mount point?

 

### df:

udev             50M     0   50M   0% /dev
tmpfs            24M  876K   24M   4% /run
/dev/mmcblk0p1   30G  859M   28G   3% /
tmpfs           120M     0  120M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           120M     0  120M   0% /sys/fs/cgroup
tmpfs           120M  4.0K  120M   1% /tmp
/dev/zram0       49M  1.2M   44M   3% /var/log

### lsblk:

NAME        FSTYPE   SIZE MOUNTPOINT UUID
mmcblk0             29.8G            
└─mmcblk0p1 ext4    29.5G /          4618846b-19e6-421b-8872-767a46ed5e1e
zram0                 50M /var/log   

### zramctl:

NAME       ALGORITHM DISKSIZE  DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram1 lzo         119.7M    4K   78B   12K       4 [SWAP]
/dev/zram0                50M  1.2M  203K  580K       4 /var/log

I see no problems from your logs.

Share this post


Link to post
Share on other sites
40 minutes ago, Igor said:

to be fully safe from power cut corruption, you need read write file system.

 

What does that mean? The file system is read/write.

 

Meanwhile I have another suspect. After recreating the system, I never unplugged the power and let it run for a few days. Still, after a reboot, the filesystem was corrupt and remounted read-only. When I ran 'fsck' on it, it found and fixed hundreds of errors. I don't know whether it would ever complete. I unplugged the power while it was still running. Maybe the SD card isn't reliable anymore. I'm going to order new ones and try again. If they fail, too, the RAM or the entire device would be defective. I have a second device of that type I can then try.

 

Regarding the same device mounted twice:

44 minutes ago, Igor said:

I see no problems from your logs.

 

Does that mean you can't tell why that's the case? I found it in the output of 'mount'. It just appears twice. No idea why. There are a dozen other mounts I also mostly don't know. Can't look up anything right now without a working SD card. But what is that /var/log.hdd anyway?

 

47 minutes ago, Igor said:

This later, wireless, is not very good one on this board.

 

I also had that impression, but found out my test was incomplete. I tried to connect via SSH but there's a bug in SSH/Debian for over 10 years: if there's no IP address at boot time, sshd will fail and never start again. The WLAN connection seems to take a minute or two to come up, though. My file system broke again before I could test a workaround for that.

Share this post


Link to post
Share on other sites
1 hour ago, ygoe said:

What does that mean? The file system is read/write.

 

I meant read / only but write read / write. :unsure::P

 

1 hour ago, ygoe said:

Maybe the SD card isn't reliable anymore.

 

This is always first thing to check.

When SD card goes permanently to read only state its anyway dead. SD card are not a very good media per se. Problems are so common that we dedicate a sub-forum for collecting issues. Check them: https://forum.armbian.com/forum/31-sd-card-and-psu-issues/

 

1 hour ago, ygoe said:

But what is that /var/log.hdd anyway?

 

We use compressed memory to store logs since early days of the project. Do some search on the forum or check our scripts to understand how things works https://github.com/armbian/build/tree/master/packages/bsp/common/usr/lib/armbian If you want to debug crashes, you surely wants to disable this, peek into /etc/defaults , but for normal operations its a nice performance boost. Constant writing of small files is killing SD card. If you used it for some time on some Raspbian before - which does't care about SD card, its dead or almost there.

 

1 hour ago, ygoe said:

if there's no IP address at boot time, sshd will fail and never start again.

 

It probably depends on the user land ... we are using Debian Stretch/Buster/Bullseye and Bionic/Focal ... all have some, usually minor, problems. Can't check ATM.

Share this post


Link to post
Share on other sites
On 6/6/2020 at 5:54 PM, Igor said:

If you used it for some time on some Raspbian before - which does't care about SD card, its dead or almost there.

 

No, that card was in use in an Android phone for a few years. It was a light usage, not too much activity. But do you have more information about how Armbian treats SD cards better than Raspbian?

 

The SSH bug is already known here: https://bugs.launchpad.net/ubuntu/+source/openssh/+bug/216847

 

PS: Why can I only post once a day? Am I considered a spam bot or so?

Share this post


Link to post
Share on other sites
3 minutes ago, ygoe said:

But do you have more information about how Armbian treats SD cards better than Raspbian?


That is best summed here:
https://docs.armbian.com/

 

Raspbian has no treats whatsoever. It is a plan Debian with all bugs and special boot proces / kernel for RPi.

 

8 minutes ago, ygoe said:

Why can I only post once a day? Am I considered a spam bot or so?

 

You can unlock your limits with a https://forum.armbian.com/subscriptions/ (currently by any), by contributing a valuable content (to get a post like) or by waiting one week. This limit can be enforced once again if a users mistakes forum conversation for granted personal technical support, if he is too lazy to search for answers before asking new ones or he is trying to get attention to his/hers issue. That's not on the menu.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
1 1