The partition is not resized to full SD card size.

[nemonein]

I'm using Armbian 5.25(mainline, Ubuntu server) on Cubietruck.

 

The partion is not extended to the full size of the SD card.

Here's my results of fdisk -l and df.

 

$ sudo fdisk -l
Disk /dev/ram0: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk /dev/ram1: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk /dev/ram2: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk /dev/ram3: 4 MiB, 4194304 bytes, 8192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk /dev/mmcblk0: 59.6 GiB, 64021856256 bytes, 125042688 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xf3693e57
Device         Boot Start       End   Sectors  Size Id Type
/dev/mmcblk0p1       2048 122541823 122539776 58.4G 83 Linux

$ df -h /dev/mmcblk0
Filesystem      Size  Used Avail Use% Mounted on
udev            996M     0  996M   0% /dev

$ df -h /dev/mmcblk0p1
Filesystem      Size  Used Avail Use% Mounted on
/dev/mmcblk0p1   58G   29G   29G  51% /

About 1.6GB is remained in unused space.

 

I know that Armbian needs 128MB emergency swap, but this 1.6GB is for swap?

 

Quote

Insert SD card into a slot and power the board. First boot takes around 3 minutes than 
it reboots and you will need to wait another one minute to login. This delay is 
because system updates package list and creates 128Mb emergency SWAP on the SD card. 

 

Last time I installed Armbian, I manually steretched the partition to the full size.

I'm wondering that might cause this.

[hrip6]

I have pretty the same:

 

Disk /dev/mmcblk0: 14.9 GiB, 16018046976 bytes, 31285248 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xe2f8e3ae

Device         Boot Start      End  Sectors  Size Id Type
/dev/mmcblk0p1       2048 30659519 30657472 14.6G 83 Linux

root@bananapi:~# df -h /dev/mmcblk0p1
Filesystem      Size  Used Avail Use% Mounted on
/dev/mmcblk0p1   15G  1.4G   13G  10% /

Disk has 14.9 GiB. Have only 1 partition with the size of 14.6 GiB. The gap of 0.3 GiB has gone in reserved space maybe, but AFAIK this should be 5%. I'm not sure about this. Try this to set 5% to 0% (be careful if you are going to do this on system partition, it might have negative affect):

sudo tune2fs -m 0 /dev/mmcblk0p1

This sets the reserved blocks to 0%. You can verify that this actually worked with:

sudo tune2fs -l /dev/mmcblk0p1 | grep 'Reserved block count'

  Reserved block count:     41413
    ### This is in my case (I didn't run the command to set 5% to 0% before as it's my system partition)

Disclaimer: I would not say this will work, but you can get your own experience at least xD

[tkaiser]

7 hours ago, nemonein said:

The partion is not extended to the full size of the SD card.

 

This is by intention (a feature not a bug): https://github.com/igorpecovnik/lib/blob/c075706ccf4e8104bcd4e3e265e81b4d6646885b/scripts/resize2fs#L53-L75

 

Armbian reserves 5% on 4GB cards, 2% on 8GB cards and 1% above. This is not related to swap or 'Reserved block count' but due to partitioning. And the reason is simple: All those 'cards of the same size' differ in reality. If you use a 64GB card that is 125042688 sectors large (roughly 60GiB) and want to clone this to another '64GB card' that is 125042687 sectors or less then you run into a problem. With Armbian's attempt to leave a small amount unpartitioned you then will run in an error message (if you clone the whole device) but not a problem any longer since all partitions fit even on smaller cards 'of the same size'.

[nemonein]

@hrip6 Thanks, But that's not the point that I want to talk about.

@tkaiser Thanks again, you've been very helpful.
I have one more question.

 

If I extend the partition to the end(like what I did before), is there any possibility that causes any trouble?

I don't need to clone this card to another.

[martinayotte]

No, there is no problem about using the whole space.

The issue will be present when cloning, except if the image is first backed-up on external disk bigger than the sdcard and then shrinked.

 

[tkaiser]

7 hours ago, nemonein said:

is there any possibility that causes any trouble?

What's the use case for 'use all available space partitioned on SD card'? That's the important part. Now you've

/dev/mmcblk0p1   58G   29G   29G  51% /

So if you resize the partition it will look like this later:

/dev/mmcblk0p1   59G   29G   30G  50% /

Nothing relevant changed, it's just that you potentially have a little more space to be filled. What happens if you really fill an SD card to the max? Depends on the controller used inside, the controller's strategies how to deal with 'running out of space' situations and the amount of overprovisioning: how much reserve sectors are on a card that announces 125042688? Is it just 8192 more (4MB), is it 1048576 (512MB), is it any other value? The card's controller has to take care about wear leveling and garbage collection in the background. Poor controller strategies paired with not that much overprovisioning (read as: yeah, that's the average SD card we're talking about) will lead to dramatically reduced performance in this situation.

 

Just try it out yourself with 1% reserve and without:

cd $HOME
iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
dd if=/dev/zero of=110mb.file bs=1M count=110
sync
dd if=/dev/zero of=fill-it bs=10M
# now you get an error 'No space left on device', so let's delete the 110MB file and test again
rm 110mb.file
sync
iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
rm fill-it

Please post numbers together with 'armbianmonitor -u' output (will take ages if you've a slow SD card, nice read in another shell is 'iostat 5')

[hrip6]

10 hours ago, nemonein said:

@hrip6 Thanks, But that's not the point that I want to talk about.

Yeah, just try to help

10 hours ago, tkaiser said:

Armbian reserves 5% on 4GB cards, 2% on 8GB cards and 1% above. This is not related to swap or 'Reserved block count' but due to partitioning.

Thanks for again let me learning from you. I thought this about reserve block which is have on almost Linux kernel, but is this an Armbian function only?

[Igor]

5 hours ago, hrip6 said:

but is this an Armbian function only?

Yes. It's a part of optimisations that we did on the top of stock Debian / Ubuntu. Valuable on SD card media installations; pointless to have it elsewhere, on Intel desktop PC for example.

[nemonein]

First iozone..

iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
    Iozone: Performance Test of File I/O
            Version $Revision: 3.429 $
        Compiled for 32 bit mode.
        Build: linux 
    Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
                 Al Slater, Scott Rhine, Mike Wisner, Ken Goss
                 Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
                 Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
                 Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
                 Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
                 Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
                 Vangel Bojaxhi, Ben England, Vikentsi Lapa.
    Run began: Mon Mar 13 14:20:13 2017
    Include fsync in write timing
    O_DIRECT feature enabled
    Auto Mode
    File size set to 102400 kB
    Record Size 4 kB
    Record Size 16 kB
    Record Size 512 kB
    Record Size 1024 kB
    Record Size 16384 kB
    Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
    Output is in kBytes/sec
    Time Resolution = 0.000001 seconds.
    Processor cache size set to 1024 kBytes.
    Processor cache line size set to 32 bytes.
    File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride                                    
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
          102400       4     1445     3069     7205     7220     7063     3036                                                          
          102400      16     8590    10141    14269    14343    14297    11498                                                          
          102400     512    13829    17603    22168    22161    22188    21008                                                          
          102400    1024    16877    19525    22432    22405    22403    20929                                                          
          102400   16384    17740    19398    22560    22534    22538    20072                                                          
iozone test complete.

and results of 'dd's.

$ dd if=/dev/zero of=110mb.file bs=1M count=110
110+0 records in
110+0 records out
115343360 bytes (115 MB, 110 MiB) copied, 1.02726 s, 112 MB/s
$ dd if=/dev/zero of=fill-it bs=10M
dd: error writing 'fill-it': No space left on device
2630+0 records in
2629+0 records out
27569053696 bytes (28 GB, 26 GiB) copied, 3021.98 s, 9.1 

 

and the last iozone.

$ iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
    Iozone: Performance Test of File I/O
            Version $Revision: 3.429 $
        Compiled for 32 bit mode.
        Build: linux 
    Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
                 Al Slater, Scott Rhine, Mike Wisner, Ken Goss
                 Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
                 Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
                 Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
                 Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
                 Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
                 Vangel Bojaxhi, Ben England, Vikentsi Lapa.
    Run began: Mon Mar 13 15:44:28 2017
    Include fsync in write timing
    O_DIRECT feature enabled
    Auto Mode
    File size set to 102400 kB
    Record Size 4 kB
    Record Size 16 kB
    Record Size 512 kB
    Record Size 1024 kB
    Record Size 16384 kB
    Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
    Output is in kBytes/sec
    Time Resolution = 0.000001 seconds.
    Processor cache size set to 1024 kBytes.
    Processor cache line size set to 32 bytes.
    File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride                                    
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
          102400       4     1144     2648     6812     6891     6712     2742                                                          
          102400      16     8012     9677    13899    13896    13818    10989                                                          
          102400     512    14474    17739    21164    21250    21239    19714                                                          
          102400    1024    17004    18842    21424    21337    21481    20043                                                          
          102400   16384    16069    18256    21861    21860    21842    19104                                                          
iozone test complete.

 

Armbian monitor is here.

http://sprunge.us/aEGY
 

 

I don't know what you meant by this.

Quote

how much reserve sectors are on a card that announces 125042688? Is it just 8192 more (4MB), is it 1048576 (512MB), is it any other value?

 

 

[Igor]

39 minutes ago, nemonein said:

Armbian monitor is here.

http://sprunge.us/aEGY
 

I don't see any problems with your SD / hard drive.

[tkaiser]

3 hours ago, nemonein said:

http://sprunge.us/aEGY

 

So this is a 64GB Samsung. EVO, EVO+, Pro or Pro+? Anyway there's still the 1% reserve present and this is not 'the average' SD card but Samsungs all contain rather good controllers and NAND dies.

 

Regarding 'reserve sectors' and overprovisioning: If a card claims it's n bytes in capacity it has internally a larger capacity. This is used as reserve (if the controller detects bad sectors, then reserve sectors are mapped in) and to allow somewhat ok-ish write performance when the card gets full. On flash media you can't overwrite directly, it's always a very time consuming read/delete/write cycle, the number a flash cell can be written to is determined by the count of program/erase (P/E) cycles it is designed for, the controller has to take care of this so that all flash cells wear out equally (wear leveling).

 

Since the controller has no idea which sectors contain real data and which not (there's no TRIM support for SD cards) as soon as you completely fill the card once (all space partitioned) from now on the controller considers every sector containing useful data (even if you deleted the data in the meantime -- since there's no TRIM support the controller doesn't know what's empty or not, from now on the whole capacity is considered in use). Now only the 'reserve sectors' are available to perform read/delete/write cycles and if this amount of sectors is small things slow down a lot on average SD cards (not those more recent Samsung).

 

Just check articles explaining SSDs and keep in mind that SD cards behave like crappy SSDs from a decade ago containing slow/primitive controllers and do not support TRIM.

 

BTW: This is the only great use case for SD Association's 'SD Formatter'. This tool is used to format SD cards appropriately (partitions it while choosing the 'correct' file system which is either FAT or exFAT) which obviously is pretty useless from Armbian's perspective since burning an OS image as next step both overwrites the partition table and the filesystems present before. So why using SD Formatter in the first place? Since this tool implements ERASE CMD38. It tells the card's controller that every sector/block of the card does not contain any real data any more and can be considered empty. On 'the average' SD card this also might restore horribly low performance back to 'factory default' performance. But more recent SD card controllers especially when paired with many reserve sectors aren't that much affected.

 

With Armbian's auto resize behaviour leaving at least 1% unpartitioned all that's happening in the background is that we ensure there are more sectors available for the controller's housekeeping which prevents older cards from becoming slow like hell as soon as they've been 'filled completely'.

 

Edit: A small note regarding 'SD cards don't support TRIM'. The SD protocol defines a block erase command and tools like fstrim are supposed to do the job. Whether your kernel + SD card combination supports that or not a simple 'sudo fstrim -v /' might tell. Whether this has the desired effect or not is a different question though (see this attempt to test this -- I'm not sure whether the method is sufficient since the point of marking data segments as already erased should not involve overwriting them, it's just that the SD card's controller knows that specific sectors/pages can be added to the wear level pool since marked 'emtpy' now)

[nemonein]

@tkaiser

Your advice is always very helpful to me.(to everyone of course.) Thank you again.

 

For the record, my SD card is Samsung Evo plus. I have not known that this SD card is better than average one, because I live in the country where Samsung's HQ is. ;-)

 

And for the conclusion, it would be good not to resize the partiotion.

 

Thank you all.