tkaiser Posted November 19, 2016 Author Posted November 19, 2016 https://www.sandisk.com/home/memory-cards/microsd-cards/high-endurance-microsd I'm guessing this must be using a better binned MLC flash. Why? Ever listened to marketing guys playing bullshit bingo? 'Hey, let's make a white variant and write something about endurance on the packaging! Might increase market share by 1%' Better look on their numbers (32GB: 5,000 hours 'Full HD (1920x1080) video content recorded at 26 Mbps to one device' --> 5000 * 3600 * 26 / 8 / 1024 = 57,129 GB). So they claim a 1,785 ratio here. The card could be filled with roughly 2,000 times the amount of data compared to its capacity. So given that's the usual flash capable of 1000 write cycles they could simply use 'a bit more' overprovisioning (not a few percent as on normal quality SD cards but 100 percent: the 32GB variant has 512Gb capacity and the larger one is 1024Gb and both expose just half of it through their controller/FTL). Since the only use case mentioned is video recording this means just 'sequential write' of large block sizes so FTL (flash translation layer) might be highly optimised for keeping write amplificatiojn as low as possible which may result in painfully low random IO performance (since FTL/controller priorize minimizing write cycles over performance, something that works pretty well if it's only about sequential writes but is challenging as soon as random IO is also concerned). In other words: As soon as we think about an SD card being used in a computer things might be totally different (not that surprising but one of the issues so many people run into bad experiences with SBC, since they rely on irrelevant specs like 'speed class' and choose horribly bad performing SD cards). With random IO write amplification might be magnitues higher especially if/when this card type chose a rather large erase block size. If the latter is eg. 8Mb then writing just a single byte followed by a sync will result in a 1:1,000,000 ratio and not 1:1 as with the 'recording video' use case. So still based on the assumption that we're talking here about 8Mb erase block size (no idea whether that's really the case, this is something a short flashbench run could show), using this card in an SBC used as desktop replacement and browsing the web Firefox alone (permanently saving a lot of stuff to a few SQLite databases with 32KB chunk size configured) could result in a write amplification ratio of 1:32 and the card will show failures already after just 1,785 GB written and not 57,129 GB as with the video use case. TL;DR: It's all about the use case. SD card numbers are based on an entirely different use case (sequential writes as it happens in digital cameras and video recorders) so both performance and endurance numbers you get are ALWAYS WRONG when talking about SBC use case. You always have to check yourself.
tedaz Posted November 19, 2016 Posted November 19, 2016 Looking forward for review and comparison of sandisk 200GB, 256GB and Samsung plus and pro 255GB. Sent from my SM-G900F using Tapatalk
tkaiser Posted November 19, 2016 Author Posted November 19, 2016 Looking forward for review and comparison of sandisk 200GB, 256GB and Samsung plus and pro 255GB. There you go: http://www.armbian.com/donate/
hojnikb Posted November 19, 2016 Posted November 19, 2016 Better look on their numbers (32GB: 5,000 hours 'Full HD (1920x1080) video content recorded at 26 Mbps to one device' --> 5000 * 3600 * 26 / 8 / 1024 = 57,129 GB). So they claim a 1,785 ratio here. The card could be filled with roughly 2,000 times the amount of data compared to its capacity. So given that's the usual flash capable of 1000 write cycles they could simply use 'a bit more' overprovisioning (not a few percent as on normal quality SD cards but 100 percent: the 32GB variant has 512Gb capacity and the larger one is 1024Gb and both expose just half of it through their controller/FTL). 2000 p/e is actually quite a lot for an sd card. First note, that this is calculated, so it means that if you take into account write amplification you're getting closer to 3000 erase cycles. This is quite a lot for 2d mlc flash these days and especially a lot of an sd card. You must know, that most sd cards use lower bin tlc flash, which is only good for around a few 100 p/e. So this is definitely an improvement here. In other words: As soon as we think about an SD card being used in a computer things might be totally different (not that surprising but one of the issues so many people run into bad experiences with SBC, since they rely on irrelevant specs like 'speed class' and choose horribly bad performing SD cards). With random IO write amplification might be magnitues higher especially if/when this card type chose a rather large erase block size. If the latter is eg. 8Mb then writing just a single byte followed by a sync will result in a 1:1,000,000 ratio and not 1:1 as with the 'recording video' use case. So still based on the assumption that we're talking here about 8Mb erase block size (no idea whether that's really the case, this is something a short flashbench run could show), using this card in an SBC used as desktop replacement and browsing the web Firefox alone (permanently saving a lot of stuff to a few SQLite databases with 32KB chunk size configured) could result in a write amplification ratio of 1:32 and the card will show failures already after just 1,785 GB written and not 57,129 GB as with the video use case. TL;DR: It's all about the use case. SD card numbers are based on an entirely different use case (sequential writes as it happens in digital cameras and video recorders) so both performance and endurance numbers you get are ALWAYS WRONG when talking about SBC use case. You always have to check yourself. Unless we have exact specification, we don't know how controller handles random data, so we really can't predict how high write amplification will be when dealing with random data. But given the low sequential speed of the cards, i'm quite confident that this is not strictly optimized for sequential data. If it was, sequential speeds would be much better than 20MB/s.
tkaiser Posted November 19, 2016 Author Posted November 19, 2016 Unless we have exact specification, we don't know how controller handles random data, so we really can't predict how high write amplification will be when dealing with random data. But given the low sequential speed of the cards, i'm quite confident that this is not strictly optimized for sequential data. If it was, sequential speeds would be much better than 20MB/s. No one knows until controller strategy has been revealed (I would assume the controller's 1st priority is always minimizing write amplification so maybe that's the reason). But as already written: As soon as the use case differs from that what SD cards are made for one always has to test. And with new cards it's getting characteristics first (flashbench), then testing performance and maybe doing an endurance test (would be both interesting and somewhat braindead when done 'just in case' since card is destroyed afterwards). My attempt to deal with limited endurance of SD cards is simply limiting writes to it, using filesystems that can detect corruption (btrfs together with mainline kernel or with Xenial now also zfs in a very convenient way) and check warranty parameters instead of pseudo specs like the above. Samsung EVOs have 60 months compared to 24 with the SanDisks so I already know which to buy. But my use cases are different. And by using superiour filesystems write amplification remains low. We started testing one rsyslogd central log host in May: H3 board with kernel 4.4, 64 GB EVO and btrfs parameters as follows: 'commit=300,compress-force=zlib,discard'. Holds now approx. 400 GB log data, might store another 400 GB until the first erase cycle has to happen
hojnikb Posted November 25, 2016 Posted November 25, 2016 http://www.phonearena.com/news/microSD-cards-fast-enough-for-running-Android-apps-will-be-marked-with-the-new-A1-symbol_id88144 this looks good for our little SBCs
hojnikb Posted December 9, 2016 Posted December 9, 2016 Some benchmarks on PC2 using ubuntu_xfce 1.0.0 image from XunLong Card used is Samsung 32GB EVO+ root@Orangepi:/home/orangepi# iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2 Iozone: Performance Test of File I/O Version $Revision: 3.429 $ Compiled for 64 bit mode. Build: linux Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins Al Slater, Scott Rhine, Mike Wisner, Ken Goss Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, Vangel Bojaxhi, Ben England, Vikentsi Lapa. Run began: Fri Dec 9 17:36:07 2016 Include fsync in write timing O_DIRECT feature enabled Auto Mode File size set to 102400 kB Record Size 4 kB Record Size 16 kB Record Size 512 kB Record Size 1024 kB Record Size 16384 kB Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2 Output is in kBytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 kBytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. random random bkwd record stride kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 102400 4 1489 2265 7510 7502 7476 2884 102400 16 9689 11278 14578 14538 14423 11931 102400 512 17672 18220 23048 23049 23049 18416 102400 1024 17325 19293 23017 23031 23016 21126 102400 16384 15370 17377 23040 23042 23042 18059 iozone test complete. As suspected, H5 sd controller is still shit, just like it was in h3
tkaiser Posted December 9, 2016 Author Posted December 9, 2016 As suspected, H5 sd controller is still shit, just like it was in h3 Wrong: http://linux-sunxi.org/H5#Overview It's a board design 'problem' or let's better say design decision. Without providing the abillity to switch voltage (3.3V and also 1.8V) no faster modes are possible. And once such a board exists (maybe Olimex A64 board allows that) drivers have to be written. So the sequential read/write limitation of approx 23 MB/s is due to board design decisions and not the SoC's controller. BTW: I always have a good laugh when people complain about literally $anything since we're talking about devices for $20 or even less here. Did you check cpufreq while running the test? Governor set to performance before to ensure running on at least 1008 MHz all the time?
hojnikb Posted December 10, 2016 Posted December 10, 2016 Wrong: http://linux-sunxi.org/H5#Overview It's a board design 'problem' or let's better say design decision. Without providing the abillity to switch voltage (3.3V and also 1.8V) no faster modes are possible. And once such a board exists (maybe Olimex A64 board allows that) drivers have to be written. So the sequential read/write limitation of approx 23 MB/s is due to board design decisions and not the SoC's controller. BTW: I always have a good laugh when people complain about literally $anything since we're talking about devices for $20 or even less here. Did you check cpufreq while running the test? Governor set to performance before to ensure running on at least 1008 MHz all the time? I used nmon and cpufreq panel and frequency was always at 1008Mhz. As for the whole sd card controller; isn't the actual controller in the soc still limited to 25MB/s for SD cards ? Or can you use mmc controller for sd cards as well (obviously that would boost sequential performance given how that is faster)
hojnikb Posted December 10, 2016 Posted December 10, 2016 I always have a good laugh when people complain about literally $anything since we're talking about devices for $20 or even less here. Well, complaining makes progress, otherwise we would still be stuck with CRAPPY raspberries for 35$. Single core arm6, shit IO, shit everything else. It really doesn't hurt to be critical, regardless of price. Things like better sd controller don't exactly add that much more cost.
mss Posted December 13, 2016 Posted December 13, 2016 (edited) And dmesg will tell you the reason: Filesystem corrupted due to using faulty or counterfeit SD card and kernel trying to save your FS by setting it to read-only Call dmesg and watch for ext4/corruption, then follow the checking procedure on another system as outlined in our documentation: http://docs.armbian.com/User-Guide_Getting-Started/#how-to-prepare-a-sd-card or throw the card away. Thanks but I tested SD cart with H2testw and no issue detected. output log: Warning: Only 14786 of 14787 MByte tested. Test finished without errors. You can now delete the test files *.h2w or verify them again. Writing speed: 8.07 MByte/s Reading speed: 20.6 MByte/s H2testw v1.4 and demsg log after filesystem crash : [16631.756436] systemd-logind[25616]: New seat seat0. [16631.809140] systemd-logind[25616]: Failed to start user service: Unknown unit: user@0.service [16631.852567] systemd-logind[25616]: New session 1450 of user root. [16980.635700] systemd-logind[25616]: New session 1482 of user root. [22881.545071] systemd-logind[25616]: New session 2003 of user root. [23419.144228] mmcblk0: mmc_blk_err_check: general error sending status command, card status 0x80e00 [23419.146272] mmcblk0: retrying write for general error [23419.148499] mmcblk0: mmc_blk_cmd_recovery: general error sending stop or status command, stop cmd response 0x0, card status 0x80d00 [23419.151556] mmcblk0: timed out sending r/w cmd command, card status 0x80d00 [23419.154673] mmcblk0: not retrying timeout [23419.157856] end_request: I/O error, dev mmcblk0, sector 23929992 [23423.502223] mmcblk0: mmc_blk_cmd_recovery: general error sending stop or status command, stop cmd response 0x0, card status 0x80d00 [23423.507197] mmcblk0: timed out sending r/w cmd command, card status 0x80d00 [23423.512222] mmcblk0: not retrying timeout [23423.517266] end_request: I/O error, dev mmcblk0, sector 3936304 [23423.522180] end_request: I/O error, dev mmcblk0, sector 3936312 [23423.522180] end_request: I/O error, dev mmcblk0, sector 3936320 [23423.522180] end_request: I/O error, dev mmcblk0, sector 3936328 [23423.522180] end_request: I/O error, dev mmcblk0, sector 3936336 [23423.522180] end_request: I/O error, dev mmcblk0, sector 3936344 [23423.522180] end_request: I/O error, dev mmcblk0, sector 3936352 [23423.522180] end_request: I/O error, dev mmcblk0, sector 3936360 [23423.522180] end_request: I/O error, dev mmcblk0, sector 3936368 [23423.522180] end_request: I/O error, dev mmcblk0, sector 3936376 [23423.522180] end_request: I/O error, dev mmcblk0, sector 3936384 [23423.522180] end_request: I/O error, dev mmcblk0, sector 3936392 [23423.522180] end_request: I/O error, dev mmcblk0, sector 3936400 [23423.522180] end_request: I/O error, dev mmcblk0, sector 3936408 [23423.522180] end_request: I/O error, dev mmcblk0, sector 3936416 [23423.598498] Aborting journal on device mmcblk0p1-8. [23423.598852] mmcblk0: mmc_blk_cmd_recovery: general error sending stop or status command, stop cmd response 0x0, card status 0x80d00 [23423.598945] mmcblk0: timed out sending r/w cmd command, card status 0x80d00 [23423.598964] mmcblk0: not retrying timeout [23423.598985] end_request: I/O error, dev mmcblk0, sector 5197824 [23423.599008] end_request: I/O error, dev mmcblk0, sector 5197832 [23423.599030] end_request: I/O error, dev mmcblk0, sector 5197840 [23423.599050] end_request: I/O error, dev mmcblk0, sector 5197848 [23423.599070] end_request: I/O error, dev mmcblk0, sector 5197856 [23423.599090] end_request: I/O error, dev mmcblk0, sector 5197864 [23423.599110] end_request: I/O error, dev mmcblk0, sector 5197872 [23423.599130] end_request: I/O error, dev mmcblk0, sector 5197880 [23423.599149] end_request: I/O error, dev mmcblk0, sector 5197888 [23423.599169] end_request: I/O error, dev mmcblk0, sector 5197896 [23423.599189] end_request: I/O error, dev mmcblk0, sector 5197904 [23423.599208] end_request: I/O error, dev mmcblk0, sector 5197912 [23423.599228] end_request: I/O error, dev mmcblk0, sector 5197920 [23423.599248] end_request: I/O error, dev mmcblk0, sector 5197928 [23423.599267] end_request: I/O error, dev mmcblk0, sector 5197936 [23423.599286] end_request: I/O error, dev mmcblk0, sector 5197944 [23423.599382] Buffer I/O error on device mmcblk0p1, logical block 649472 [23423.599446] Buffer I/O error on device mmcblk0p1, logical block 649473 [23423.599507] Buffer I/O error on device mmcblk0p1, logical block 649474 [23423.599554] Buffer I/O error on device mmcblk0p1, logical block 649475 [23423.599600] Buffer I/O error on device mmcblk0p1, logical block 649476 [23423.599643] Buffer I/O error on device mmcblk0p1, logical block 649477 [23423.599689] Buffer I/O error on device mmcblk0p1, logical block 649478 [23423.599728] Buffer I/O error on device mmcblk0p1, logical block 649479 [23423.599767] Buffer I/O error on device mmcblk0p1, logical block 649480 [23423.599806] Buffer I/O error on device mmcblk0p1, logical block 649481 [23423.599845] Buffer I/O error on device mmcblk0p1, logical block 649482 [23423.599884] Buffer I/O error on device mmcblk0p1, logical block 649483 [23423.599923] Buffer I/O error on device mmcblk0p1, logical block 649484 [23423.599962] Buffer I/O error on device mmcblk0p1, logical block 649485 [23423.600002] Buffer I/O error on device mmcblk0p1, logical block 649486 [23423.600028] Buffer I/O error on device mmcblk0p1, logical block 649487 [23423.600062] EXT4-fs warning (device mmcblk0p1): ext4_end_bio:249: I/O error writing to inode 386396 (offset 1048576 size 65536 starting block 649728) [23423.696785] mmcblk0: mmc_blk_cmd_recovery: general error sending stop or status command, stop cmd response 0x0, card status 0x80d00 [23423.826926] mmcblk0: timed out sending r/w cmd command, card status 0x80d00 [23423.832883] mmcblk0: not retrying timeout [23423.838534] end_request: I/O error, dev mmcblk0, sector 3825800 [23423.842833] Buffer I/O error on device mmcblk0p1, logical block 477969 [23423.842833] lost page write due to I/O error on mmcblk0p1 [23423.857179] JBD2: Error -5 detected when updating journal superblock for mmcblk0p1-8. [23423.857591] mmcblk0: mmc_blk_cmd_recovery: general error sending stop or status command, stop cmd response 0x0, card status 0x80d00 [23423.857678] mmcblk0: timed out sending r/w cmd command, card status 0x80d00 [23423.857696] mmcblk0: not retrying timeout [23423.857716] end_request: I/O error, dev mmcblk0, sector 2048 [23423.857745] Buffer I/O error on device mmcblk0p1, logical block 0 [23423.857763] lost page write due to I/O error on mmcblk0p1 [23423.857822] EXT4-fs error (device mmcblk0p1): ext4_journal_start_sb:328: Detected aborted journal [23423.857853] EXT4-fs (mmcblk0p1): Remounting filesystem read-only [23423.857878] ------------[ cut here ]------------ [23423.857917] WARNING: at fs/buffer.c:1112 mark_buffer_dirty+0x40/0xe4() [23423.857933] Modules linked in: 8188eu bnep bluetooth 8189es [last unloaded: 8188eu] [23423.858016] [<c0016a20>] (unwind_backtrace+0x0/0xe8) from [<c0695870>] (dump_stack+0x20/0x24) [23423.858060] [<c0695870>] (dump_stack+0x20/0x24) from [<c0029720>] (warn_slowpath_common+0x5c/0x74) [23423.858090] mmcblk0: mmc_blk_cmd_recovery: general error sending stop or status command, stop cmd response 0x0, card status 0x80d00 [23423.858130] [<c0029720>] (warn_slowpath_common+0x5c/0x74) from [<c00297f4>] (warn_slowpath_null+0x2c/0x34) [23423.858172] [<c00297f4>] (warn_slowpath_null+0x2c/0x34) from [<c0149b6c>] (mark_buffer_dirty+0x40/0xe4) [23423.858198] mmcblk0: timed out sending r/w cmd command, card status 0x80d00 [23423.858238] [<c0149b6c>] (mark_buffer_dirty+0x40/0xe4) from [<c01a21e8>] (ext4_commit_super+0x1a4/0x218) [23423.858258] mmcblk0: not retrying timeout [23423.858287] [<c01a21e8>] (ext4_commit_super+0x1a4/0x218) from [<c01a2284>] (save_error_info+0x28/0x2c) [23423.858310] end_request: I/O error, dev mmcblk0, sector 2048 [23423.858341] [<c01a2284>] (save_error_info+0x28/0x2c) from [<c01a2974>] (__ext4_abort+0xbc/0xf0) [23423.858361] Buffer I/O error on device mmcblk0p1, logical block 0 [23423.858394] [<c01a2974>] (__ext4_abort+0xbc/0xf0) from [<c01a2acc>] (ext4_journal_start_sb+0x124/0x1a4) [23423.858414] lost page write due to I/O error on mmcblk0p1 [23423.858446] [<c01a2acc>] (ext4_journal_start_sb+0x124/0x1a4) from [<c018ce68>] (ext4_dirty_inode+0x24/0x50) [23423.858474] EXT4-fs error (device mmcblk0p1): ext4_journal_start_sb:328: [<c018ce68>] (ext4_dirty_inode+0x24/0x50) from [<c01417d0>] (__mark_inode_dirty+0x3c/0x1c8) [23423.858521] Detected aborted journal[<c01417d0>] (__mark_inode_dirty+0x3c/0x1c8) from [<c0134d88>] (file_update_time+0xf8/0x114) [23423.858600] [<c0134d88>] (file_update_time+0xf8/0x114) from [<c00dccc0>] (__generic_file_aio_write+0x2f0/0x434) [23423.858666] [<c00dccc0>] (__generic_file_aio_write+0x2f0/0x434) from [<c00dce74>] (generic_file_aio_write+0x70/0xd8) [23423.858722] [<c00dce74>] (generic_file_aio_write+0x70/0xd8) from [<c01832ec>] (ext4_file_write+0x188/0x2c0) [23423.858784] [<c01832ec>] (ext4_file_write+0x188/0x2c0) from [<c011d8cc>] (do_sync_write+0xac/0xe8) [23423.858826] [<c011d8cc>] (do_sync_write+0xac/0xe8) from [<c011dfc4>] (vfs_write+0xc0/0x16c) [23423.858863] [<c011dfc4>] (vfs_write+0xc0/0x16c) from [<c011e360>] (sys_write+0x4c/0x78) [23423.858902] [<c011e360>] (sys_write+0x4c/0x78) from [<c000df60>] (ret_fast_syscall+0x0/0x30) [23423.858928] ---[ end trace 7415e919e2cb81fd ]--- [23423.859219] mmcblk0: mmc_blk_cmd_recovery: general error sending stop or status command, stop cmd response 0x0, card status 0x80d00 [23423.859293] mmcblk0: timed out sending r/w cmd command, card status 0x80d00 [23423.859312] mmcblk0: not retrying timeout [23423.859332] end_request: I/O error, dev mmcblk0, sector 2048 [23423.859361] Buffer I/O error on device mmcblk0p1, logical block 0 [23423.859379] lost page write due to I/O error on mmcblk0p1 [23448.500734] mmcblk0: mmc_blk_cmd_recovery: general error sending stop or status command, stop cmd response 0x0, card status 0x80d00 [23448.515357] mmcblk0: timed out sending r/w cmd command, card status 0x80d00 [23448.530133] mmcblk0: not retrying timeout [23448.544804] end_request: I/O error, dev mmcblk0, sector 4199624 [23448.554755] Buffer I/O error on device mmcblk0p1, logical block 524697 [23448.554755] lost page write due to I/O error on mmcblk0p1 [23448.590706] mmcblk0: mmc_blk_cmd_recovery: general error sending stop or status command, stop cmd response 0x0, card status 0x80d00 [23448.607128] mmcblk0: timed out sending r/w cmd command, card status 0x80d00 [23448.623677] mmcblk0: not retrying timeout [23448.640295] end_request: I/O error, dev mmcblk0, sector 4201016 [23448.650254] Buffer I/O error on device mmcblk0p1, logical block 524871 [23448.650254] lost page write due to I/O error on mmcblk0p1 [23474.133018] mmcblk0: timed out sending r/w cmd command, card status 0x80b00 [23474.148595] mmcblk0: not retrying timeout [23474.163811] end_request: I/O error, dev mmcblk0, sector 12875400 [23474.514701] mmcblk0: timed out sending r/w cmd command, card status 0x80b00 [23474.529646] mmcblk0: not retrying timeout [23474.544383] end_request: I/O error, dev mmcblk0, sector 12875400 [23534.270797] mmcblk0: timed out sending r/w cmd command, card status 0x80b00 [23534.284202] mmcblk0: not retrying timeout [23534.297107] end_request: I/O error, dev mmcblk0, sector 12875400 [23553.805118] mmcblk0: timed out sending r/w cmd command, card status 0x80b00 [23553.818195] mmcblk0: not retrying timeout [23553.830636] end_request: I/O error, dev mmcblk0, sector 12875400 [23594.555994] mmcblk0: timed out sending r/w cmd command, card status 0x80b00 [23594.567161] mmcblk0: not retrying timeout [23594.578194] end_request: I/O error, dev mmcblk0, sector 12875400 [23652.753856] mmcblk0: timed out sending r/w cmd command, card status 0x80b00 [23652.763856] mmcblk0: not retrying timeout [23652.773565] end_request: I/O error, dev mmcblk0, sector 12875400 [23653.505575] mmcblk0: timed out sending r/w cmd command, card status 0x80b00 [23653.515097] mmcblk0: not retrying timeout [23653.524382] end_request: I/O error, dev mmcblk0, sector 12875400 [23660.556280] mmcblk0: timed out sending r/w cmd command, card status 0x80b00 [23660.565967] mmcblk0: not retrying timeout [23660.575746] end_request: I/O error, dev mmcblk0, sector 12875400 [23691.298438] mmcblk0: timed out sending r/w cmd command, card status 0x80b00 [23691.308043] mmcblk0: not retrying timeout [23691.317425] end_request: I/O error, dev mmcblk0, sector 12875400 [23709.104535] mmcblk0: timed out sending r/w cmd command, card status 0x80b00 [23709.113909] mmcblk0: not retrying timeout [23709.123413] end_request: I/O error, dev mmcblk0, sector 12875400 [23713.866279] mmcblk0: timed out sending r/w cmd command, card status 0x80b00 [23713.876803] mmcblk0: not retrying timeout [23713.887318] end_request: I/O error, dev mmcblk0, sector 12875400 [23714.233959] mmcblk0: timed out sending r/w cmd command, card status 0x80b00 [23714.244696] mmcblk0: not retrying timeout [23714.255197] end_request: I/O error, dev mmcblk0, sector 12875400 Please Help Edited December 13, 2016 by zador.blood.stained Added spoiler tags
zador.blood.stained Posted December 13, 2016 Posted December 13, 2016 Please Help What do you expect to hear? Some magic spells that will turn a crappy SD card into a Samsung Evo Plus? Sorry, this probably won't happen. If sequential write/read tests don't show you any errors it doesn't mean that this card will behave perfectly in other scenarios - i.e. combined reads and writes of blocks of different sizes. There is only one reliable solution: 1
atharmian Posted January 2, 2017 Posted January 2, 2017 I just read this article...Any relevance for OPi? It talks about tweaking settings to get much better memory performance. To me good SD cards cost a lot compared with OPi prices themselves, and have usability and reliability issues. Can "slower" SD cards be used with some tricks? Perhaps with onboard SPI NOR bootloader, with or without netbooting? http://hackaday.com/2016/12/29/improving-raspberry-pi-disk-performance/
zador.blood.stained Posted January 2, 2017 Posted January 2, 2017 Any optimizations are a tradeoff between read/write speeds (sequential and random, observed by application, not actual SD speeds since they don't change), CPU time utilization and RAM utilization (and reliability if you don't have an UPS or are using unstable kernel drivers), so different things can be tweaked to achieve different performance improvements in different tasks. Regarding reliability - it is more a statistics problem, while minimizing writes to the card may help in theory, you can't predict when or how the SD card will malfunction (this applies both to "good" and "bad" ones). You can get anything depending on your luck - completely read-only card, partially read-only card, silent data corruption that won't be noticed until it's too late, or a completely "bricked" (unreadable) device. If we are not talking about "usual" use cases, there are different possibilities depending on your usage scenarios: Read-only rootfs on SD Rootfs on local device (HDD, SSD, USB flash) Rootfs on NFS SPI flash can help with eliminating the SD as a boot device in last 2 scenarios. Edit: Well, measuring SD card "performance" with "dd"... you may want to read the first post of this thread (and links at the end of it) before @tkaiser says that he thinks about this article
Magnets Posted January 16, 2017 Posted January 16, 2017 Samsung Evo 64GB microsd. Brand new, just run through crystaldiskmark and h2testw Orange pi PC Linux orangepipc 3.4.112-sun8i #10 SMP PREEMPT Sun Oct 23 16:06:55 CEST 2016 armv7l GNU/Linux root@orangepipc:~# iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2 Iozone: Performance Test of File I/O Version $Revision: 3.429 $ Compiled for 32 bit mode. Build: linux Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins Al Slater, Scott Rhine, Mike Wisner, Ken Goss Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, Vangel Bojaxhi, Ben England, Vikentsi Lapa. Run began: Mon Jan 16 22:16:47 2017 Include fsync in write timing O_DIRECT feature enabled Auto Mode File size set to 102400 kB Record Size 4 kB Record Size 16 kB Record Size 512 kB Record Size 1024 kB Record Size 16384 kB Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2 Output is in kBytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 kBytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. random random bkwd record stride kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread frer ead 102400 4 1004 689 4453 4434 4381 839 102400 16 2827 3298 11694 11903 11507 2441 102400 512 11333 13769 21382 21393 21106 12297 102400 1024 13565 13151 22074 22067 21998 11406 102400 16384 13517 14777 22850 22838 22844 11913 iozone test complete. It seems samsung have crippled the later evos! it's showing: date: 10/2016 second run with performance governor root@orangepipc:~# iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2 Iozone: Performance Test of File I/O Version $Revision: 3.429 $ Compiled for 32 bit mode. Build: linux Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins Al Slater, Scott Rhine, Mike Wisner, Ken Goss Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, Vangel Bojaxhi, Ben England, Vikentsi Lapa. Run began: Mon Jan 16 22:43:53 2017 Include fsync in write timing O_DIRECT feature enabled Auto Mode File size set to 102400 kB Record Size 4 kB Record Size 16 kB Record Size 512 kB Record Size 1024 kB Record Size 16384 kB Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2 Output is in kBytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 kBytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. random random bkwd record stride kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 102400 4 590 732 4510 4394 4251 805 102400 16 2927 3948 11782 11798 11642 2904 102400 512 12633 14333 21418 21406 21334 12595 102400 1024 14782 14092 21998 21981 21950 12414 102400 16384 15195 12667 22833 22845 22839 13129 iozone test complete.
Magnets Posted January 16, 2017 Posted January 16, 2017 32GB sandisk ultra root@orangepipcplus:~# iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2 Iozone: Performance Test of File I/O Version $Revision: 3.429 $ Compiled for 32 bit mode. Build: linux Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2 Output is in kBytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 kBytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. random random bkwd record stride kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 102400 4 2087 3278 7660 7666 6585 2525 102400 16 2193 2680 15009 15004 14165 4382 102400 512 7311 10130 21708 21690 21675 9219 102400 1024 11290 16439 22156 22163 22137 13792 102400 16384 17462 18827 22777 22779 22776 18479 1
hojnikb Posted January 18, 2017 Posted January 18, 2017 (edited) Samsung Evo 64GB microsd. Brand new, just run through crystaldiskmark and h2testw Orange pi PC Linux orangepipc 3.4.112-sun8i #10 SMP PREEMPT Sun Oct 23 16:06:55 CEST 2016 armv7l GNU/Linux root@orangepipc:~# iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2 Iozone: Performance Test of File I/O Version $Revision: 3.429 $ Compiled for 32 bit mode. Build: linux Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins Al Slater, Scott Rhine, Mike Wisner, Ken Goss Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, Vangel Bojaxhi, Ben England, Vikentsi Lapa. Run began: Mon Jan 16 22:16:47 2017 Include fsync in write timing O_DIRECT feature enabled Auto Mode File size set to 102400 kB Record Size 4 kB Record Size 16 kB Record Size 512 kB Record Size 1024 kB Record Size 16384 kB Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2 Output is in kBytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 kBytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. random random bkwd record stride kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread frer ead 102400 4 1004 689 4453 4434 4381 839 102400 16 2827 3298 11694 11903 11507 2441 102400 512 11333 13769 21382 21393 21106 12297 102400 1024 13565 13151 22074 22067 21998 11406 102400 16384 13517 14777 22850 22838 22844 11913 iozone test complete. It seems samsung have crippled the later evos! it's showing: date: 10/2016 second run with performance governor root@orangepipc:~# iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2 Iozone: Performance Test of File I/O Version $Revision: 3.429 $ Compiled for 32 bit mode. Build: linux Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins Al Slater, Scott Rhine, Mike Wisner, Ken Goss Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, Vangel Bojaxhi, Ben England, Vikentsi Lapa. Run began: Mon Jan 16 22:43:53 2017 Include fsync in write timing O_DIRECT feature enabled Auto Mode File size set to 102400 kB Record Size 4 kB Record Size 16 kB Record Size 512 kB Record Size 1024 kB Record Size 16384 kB Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2 Output is in kBytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 kBytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. random random bkwd record stride kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 102400 4 590 732 4510 4394 4251 805 102400 16 2927 3948 11782 11798 11642 2904 102400 512 12633 14333 21418 21406 21334 12595 102400 1024 14782 14092 21998 21981 21950 12414 102400 16384 15195 12667 22833 22845 22839 13129 iozone test complete. Maybe you got a fake one. Edited February 10, 2017 by tkaiser Hide useless fullquote
Magnets Posted January 18, 2017 Posted January 18, 2017 Maybe you got a fake one. I thought that too, but it looks real to me. It has the EU model number, made in Philippines, the print quality on the card and packet are top notch And it passes h2testw
hojnikb Posted January 18, 2017 Posted January 18, 2017 I thought that too, but it looks real to me. It has the EU model number, made in Philippines, the print quality on the card and packet are top notch and it was from Amazon (not marketplace) And it passes h2testw Sadly all of this can be faked just as well. And amazon is no way a guarantee to get a real one sadly.
Magnets Posted January 20, 2017 Posted January 20, 2017 PNY SDXC 64GB UHS-I - rated 50MB/s Include fsync in write timing O_DIRECT feature enabled Auto Mode File size set to 102400 kB Record Size 4 kB Record Size 16 kB Record Size 512 kB Record Size 1024 kB Record Size 16384 kB Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2 Output is in kBytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 kBytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. random random bkwd record stride kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread frer ead 102400 4 1182 1182 7421 7419 3163 1029 102400 16 2577 3512 14447 14524 8940 3230 102400 512 11793 16506 21625 21623 20965 17871 102400 1024 15319 16764 22187 22184 21804 18210 102400 16384 15889 17232 22911 22908 22886 18651 1
tkaiser Posted February 10, 2017 Author Posted February 10, 2017 Good news! The SDA (SD Association) responsible for specifications around SD/TF cards released SD 5.1 specifications just recently which define the first time performance numbers for random read/write performance (just check the link and the press release from Nov 2016)! SD cards that want to be compliant to A1 performance standard must exceed at least random 1500/500 IOPS (read/write) with 4K blocksize. If you compare with the Samsung EVO numbers from this thread then 500 IOPS 4k random write look somewhat laughable but the good thing is that there's now a standard that defines random IO performance and not only sequential performance as it's today (the so called 'speed class'). So while Samsung today might produce EVOs that still fulfill 'class 10' performance criteria (10 MB/s sequential write performance) they could decrease random IO performance by factor 100 and still would meet 'class 10' criteria. It's long known that especially Android really sucks when running off slow storage (slow as in 'slow random write performance') which is why most Android devices are equipped with fast eMMC or older ones with raw NAND (showing high random IO numbers). Starting with Android 7.0/6.0 a concept called 'Adoptable Storage' has been introduced that tries to integrate storage on an additional SD card with the device's fast internal storage. Android does a performance test of the SD card in question before allowing 'adopted storage' and this is where this 'A1 performance' profile originates from. As most use cases for devices running with Armbian also benefit from high random IO performance you should watch out from now on for SD cards that show the A1 logo as can be seen here: http://heise.de/-3612646 But please keep in mind that meeting A1 specs means only 500/1500 IOPS (4k write/read). The 'DSP Memory' SD card shown in the Heise link is a pretty good example for a company known not for product quality but brilliant marketing (they grew with relabeled DRAM sold to clueless Apple users). This SD card (most probably some cheap noname chips with Phison or Silicon Motion controller) claiming A1 conformance slightly exceeds the A1 criteria and the 32GB variant will be sold for 23€ while at the same time you can get a Samsung EVO+ that costs only half as much and is multiple times more performant (combining quality NAND flash dies with quality controller!): http://geizhals.de/?cat=sm_sdhc&asuch=EVO&asd=on&v=e&hloc=at&hloc=de&hloc=pl&hloc=uk&hloc=eu&filter=aktualisieren&xf=1032_Samsung~2456_10~298_microSDHC~298_microSDXC~307_32~5950_2&sort=p If you compare with performance numbers from 1st post of this thread at least my Samsung EVO/EVO+ with 32 and 64 GB all show ~7500 read and +3000 write IOPS at 4k -- that's 5 times the A1 read performance and 6 times the write performance! 1
hojnikb Posted February 10, 2017 Posted February 10, 2017 Are evos still good at random performance ? can anyone test newly purchased cards ?
tkaiser Posted February 10, 2017 Author Posted February 10, 2017 Are evos still good at random performance ? can anyone test newly purchased cards ? While I highly appreciate that you don't do a full quote of mostly unrelated stuff someone else wrote before as usual why not just looking a few posts above? There are numbers of 'brand new' EVOs bought just recently. Maybe it's important to keep in mind what I wrote in the very first post of this thread: I used three of these cards the first time and they all show identical behaviour especially regarding writes with small record sizes: pretty slow in the beginning and getting faster over time (the Samsung Pro for example started with only 1400KB/s 4K writes and 3 runs later the same test showed 3270KB/s... 1
duyfken Posted March 29, 2017 Posted March 29, 2017 (edited) For comparison, a 32GB Sandisk Extreme Pro 95MB/s MicroSD, a 32GB Samsung EVO+ MicroSD and the Orange Pi PC Plus eMMC speeds: (using 'iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2') 32GB Sandisk Extreme Pro 95MB/s MicroSD random random bkwd record stride kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread frer ead 102400 4 2891 2973 8610 8609 8378 4764 102400 16 7914 8058 16077 16097 16064 11118 102400 512 20219 20331 21772 21724 21764 19582 102400 1024 20450 20479 22279 22277 22269 20117 102400 16384 21585 21620 22957 22967 22956 21358 32GB Samsung EVO+ MicroSD random random bkwd record stride kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread frer ead 102400 4 1510 2426 7017 7380 7496 3326 102400 16 10629 11033 14575 14563 14579 11048 102400 512 20020 20167 21592 21594 21584 20095 102400 1024 20283 20370 22130 22129 22128 20340 102400 16384 21053 21198 22906 22905 22903 21190 Orange Pi PC Plus eMMC random random bkwd record stride kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread frer ead 102400 4 6204 6558 15817 15812 13358 6422 102400 16 21499 22650 38919 38938 34894 21409 102400 512 35602 35566 64903 64908 64912 34244 102400 1024 36147 36359 69756 70083 69976 35780 102400 16384 38198 38697 78658 78768 78775 38495 MicroSD clearly bottlenecks (particularly obvious with one of the fastest UHS-1 MicroSD cards in the Extreme Pro) and eMMC is definitely the better option for OS by the looks. Edited March 29, 2017 by tkaiser Formatted for better reading
hojnikb Posted March 29, 2017 Posted March 29, 2017 Pretty good random performance from sandisk. Even better than sammy.
tkaiser Posted March 29, 2017 Author Posted March 29, 2017 10 hours ago, duyfken said: 32GB Sandisk Extreme Pro 95MB/s MicroSD Thank you for the numbers, though both 4K and 16K Sandisk numbers look very suspicious (random IO faster than sequential? Hard to believe since this would mean a crippled controller 'optimizing' something terribly wrong). In case you have the time it would be great if you could re-test, ensure that you've chosen performance governor and provide 'armbianmonitor -u' output or at least the MMC info block as can be seen here: https://forum.armbian.com/index.php?/topic/990-testers-wanted-sd-card-performance/&do=findComment&comment=8172 There a SanDisk Extreme Plus with 32GB was tested. Since I got a 16GB SanDisk Extreme Plus 2 days ago for just €17 here are the numbers (debug output -- you might want to search for '### mmc0' in there) that look quite comparable to the above from last year: random random kB reclen write rewrite read reread read write 102400 4 3331 3305 7823 7834 6722 2905 102400 16 8308 8672 16278 16290 16290 7830 102400 512 20892 22040 22913 22951 22949 22038 102400 1024 22346 22455 23074 22594 23071 22127 102400 16384 22206 22633 23289 23290 23289 22624 This was the fourth run and numbers remained stable (I don't trust in single 'fire and forget' benchmark runs and numbers with factory new Samsung cards from last year prove this attempt to be right since they seem to do some sort of initial 'calibration' at least they get significantly faster after some short time of use -- see post #1 from this thread for details if interested) As you already noticed the maximum throughput is limited to 22/23 MB/s anyway (this is an SDIO limitation unfortunately most boards Armbian supports share, one known exception is the ASUS Tinkerboard now where +50MB/s are reported with capable SD cards) 10 hours ago, duyfken said: Orange Pi PC Plus eMMC O yes, Xunlong seems to be the only vendor using pretty fast eMMC also on the 'small' or better say inexpensive boards (according to Aliexpress pictures same eMMC is also used on 'Zero Plus 2' now). FriendlyELEC and SinoVoip for example chose both pretty slow 8GB eMMC modules on their NanoPi and BananaPi (at least FriendlyELEC responded and said they will do some investigations so maybe we get faster eMMC here too)
hojnikb Posted March 29, 2017 Posted March 29, 2017 1 hour ago, tkaiser said: Thank you for the numbers, though both 4K and 16K numbers look very suspicious (random IO faster than sequential? Hard to believe since this would mean a crippled controller 'optimizing' something terribly wrong). In case you have the time it would be great if you could re-test, ensure that you've chosen performance governor and provide 'armbianmonitor -u' output or at least the MMC info block as can be seen here: https://forum.armbian.com/index.php?/topic/990-testers-wanted-sd-card-performance/&do=findComment&comment=8172 which results are you looking at ? They look fine to me (bigger the block size, faster performance).
hojnikb Posted March 29, 2017 Posted March 29, 2017 Quote This was the fourth run and numbers remained stable (I don't trust in single 'fire and forget' benchmark runs and numbers with factory new Samsung cards from last year prove this attempt to be right since they seem to do some sort of initial 'calibration' at least they get significantly faster after some short time of use -- see post #1 from this thread for details if interested) that seems very interesting. I guess there is a chance sd card is doing some internal garbage collection (old data from burn in testing ?) which can obviously yield worse results at the beginning.
tkaiser Posted March 29, 2017 Author Posted March 29, 2017 46 minutes ago, hojnikb said: which results are you looking at ? They look fine to me (bigger the block size, faster performance). First 2 lines of SanDisk numbers. Random writes being 60 percent faster than sequential at 4K blocksize is simply strange: 102400 4 2891 2973 8610 8609 8378 4764 102400 16 7914 8058 16077 16097 16064 11118 19 minutes ago, hojnikb said: I guess there is a chance sd card is doing some internal garbage collection (old data from burn in testing ?) which can obviously yield worse results at the beginning. Whatever it is... it happened above also, just look at the 4K results for the EVO: 102400 4 1510 2426 7017 7380 7496 3326 1.5MB/s when writing the 1st 100 MB, then 2.4MB/s when re-writing 100 MB and finally 3.3MB/s when writing the next 100 MB with a random pattern. Doing benchmarks is not just producing numbers without meaning but interpreting/understanding results (which leads in many or even most cases to the tester throwing away results and starting again). And now please let's remember that this here is not a chatroom but a research/educational thread trying to collect valuable information to be used by people coming from anywhere. This whole threads gets useless when flooded with posts!
![m][sko](https://forum.armbian.com/uploads/n2a/n2a-180904eb9612afa23a4c3358385bd86f384613e3.avatars_letters_gd.png){kind=avatar}
Recommended Posts