Jump to content

NanoPi NEO2: USB Disk + rsync = I/O errors


TCB13

Recommended Posts

Hello,

 

I'm running rsync to backup a VPS to an USB hard drive on the neo2 and sometimes rsync just stops with "read error from input device (I/O error)". I believe this issue happens when rsync is causing high load in the USB disk, writing or removing a large number of files or trying to write a big file.

 

Notes:

  • The hard drive is on a self powered enclosure - so power failure is not really an option here
  • I also have Apache and mariadb storing data on the USB disk and I never had this kind of I/O errors with it
  • OS: Debian GNU/Linux 9.4 (stretch)
  • Drive is ext4

 

Whenever rysnc stops I get this under /var/log/messages:

sd 0:0:0:0: [sda] tag#1 data cmplt err -32 uas-tag 2 inflight: 
sd 0:0:0:0: [sda] tag#1 CDB: opcode=0x28 28 00 38 80 0a 68 00 00 a0 00
sd 0:0:0:0: [sda] tag#0 data cmplt err -32 uas-tag 1 inflight: CMD 
sd 0:0:0:0: [sda] tag#0 CDB: opcode=0x2a 2a 00 57 50 28 78 00 03 00 00
sd 0:0:0:0: [sda] tag#1 uas_eh_abort_handler 0 uas-tag 2 inflight: CMD 
sd 0:0:0:0: [sda] tag#1 CDB: opcode=0x28 28 00 38 80 0a 68 00 00 a0 00
sd 0:0:0:0: [sda] tag#2 uas_eh_abort_handler 0 uas-tag 3 inflight: CMD 
sd 0:0:0:0: [sda] tag#2 CDB: opcode=0x2a 2a 00 19 47 7f 20 00 00 90 00
sd 0:0:0:0: [sda] tag#0 uas_eh_abort_handler 0 uas-tag 1 inflight: CMD 
sd 0:0:0:0: [sda] tag#0 CDB: opcode=0x2a 2a 00 57 50 28 78 00 03 00 00
scsi host0: uas_eh_device_reset_handler start
usb 5-1: reset high-speed USB device number 2 using ehci-platform
scsi host0: uas_eh_device_reset_handler success
sd 0:0:0:0: [sda] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
sd 0:0:0:0: [sda] tag#0 Sense Key : 0x2 [current] 
sd 0:0:0:0: [sda] tag#0 ASC=0x3a ASCQ=0x0 
sd 0:0:0:0: [sda] tag#0 CDB: opcode=0x2a 2a 00 57 50 28 78 00 03 00 00
sd 0:0:0:0: [sda] tag#1 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08
sd 0:0:0:0: [sda] tag#1 Sense Key : 0x2 [current] 
sd 0:0:0:0: [sda] tag#1 ASC=0x3a ASCQ=0x0 
sd 0:0:0:0: [sda] tag#1 CDB: opcode=0x2a 2a 00 19 47 7f 20 00 00 90 00
EXT4-fs warning (device sda1): ext4_end_bio:323: I/O error 10 writing to inode 13001563 (offset 0 size 73728 starting block 53014518)

 

My fstab:

LABEL=USBHD /mnt/storage ext4 defaults,nofail 0 0

 

Why does it happen? Thank you.

 

PS: can someone move this to the Technical support section? Posted on the wrong place!

Link to comment
Share on other sites

Looks like this is some bug related to the UAS module of the kernel.  The answer here by Cinos Modnar seems to be a nice workaround: https://unix.stackexchange.com/questions/239782/connection-problem-with-usb3-external-storage-on-linux-uas-driver-problem however looks like the device is not blacklisted.

 

I also tried to blacklist uas, but after this the disk isn't mounted.

Link to comment
Share on other sites

So If I blacklist UAS with "blacklist uas" into /etc/modprobe.d/blacklist-uas.conf I get:

lsusb -t
/:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=ehci-platform/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Mass Storage, Driver=, 480M

 

Looking at Class=Mass Storage, Driver=, 480M => seems like the systems doesn't load any other way to deal with the drive.

 

If I just try to disable for specific USB device, like the post said it still loads as uas:

echo options usb-storage quirks=174c:55aa:u | tee /etc/modprobe.d/blacklist-uas.conf
update-initramfs -u
reboot

(...)

dmesg | grep sda
[    2.488105] sd 0:0:0:0: [sda] 2930277168 512-byte logical blocks: (1.50 TB/1.36 TiB)
[    2.488584] sd 0:0:0:0: [sda] Write Protect is off
[    2.488592] sd 0:0:0:0: [sda] Mode Sense: 43 00 00 00
[    2.489335] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    2.539288]  sda: sda1
[    2.543875] sd 0:0:0:0: [sda] Attached SCSI disk
[    6.898109] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,data=ordered

lsusb | grep ASMedia
Bus 005 Device 002: ID 174c:55aa ASMedia Technology Inc. ASM1051E SATA 6Gb/s bridge, ASM1053E SATA 6Gb/s bridge, ASM1153 SATA 3Gb/s bridge

lsusb -t
/:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=ehci-platform/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Mass Storage, Driver=uas, 480M

 

Any tips here? Thank you.

 

 

Link to comment
Share on other sites

By using lsmod I found out that usb-storage in this kernel is not loaded as a module, it is builtin. Due to this we need to add the usb-storage quirks into the boot options.

 

I went into /boot/armbianEnv.txt and at the end of "usbstoragequirks=" I added my device by appending ",174c:55aa:u". Then I re-run update-initramfs -u and rebooted.

 

After this, the device not shows as using usb-storage instead of uas:

lsusb -t
/:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=ehci-platform/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Mass Storage, Driver=usb-storage, 480M


As you can see here, uas is not properly blacklisted for the device:

dmesg | grep "usb 5-1"
[    2.308569] usb 5-1: new high-speed USB device number 2 using ehci-platform
[    2.467087] usb 5-1: New USB device found, idVendor=174c, idProduct=55aa
[    2.467106] usb 5-1: New USB device strings: Mfr=2, Product=3, SerialNumber=1
[    2.467117] usb 5-1: Product: ASM1153E
[    2.467127] usb 5-1: Manufacturer: Inateck
[    2.467137] usb 5-1: SerialNumber: 12345678910E
[    2.468297] usb 5-1: UAS is blacklisted for this device, using usb-storage instead

 

So my problem is fixed.

 

Link to comment
Share on other sites

19 minutes ago, TCB13 said:

So my problem is fixed

 

Seriously? Please report back in a day whether it's really fixed or if you now just get different error messages reporting the same (underlying) problem (see issue 7 here). The ASM1153E with original ASMedia firmware is known to be fully UAS capable.

 

If your problem is not related to UAS please also correct the various posts on stackexchange.com. One reason so many Linux users think UAS would always cause issues is the vast amount of wrong reports on the Internet people stumble accross once they google for the error messages they run into.

 

 

Link to comment
Share on other sites

I guess you are right, at least for what I tried with Rock64 and Armbian

 

I have USB3.0 problems mentioned elsewhere on this forum and on 

https://forum.pine64.org/showthread.php?tid=5137&pid=37535#pid37535
I used tips above for an el cheapo USB3.0 JMS597 SATA adapter cable 0x152d:0x0578

 

In Armbian 5.59 I had read/write speed around 80MBps ish using samba (varying between 109 and 70)

But with read, after a few seconds full speed, it froze for 10-15 sec, no errors in the transferred data, but with "ERROR Transfer event for disabled endpoint or incorrect stream ring" in dmesg every time it froze.

 

With above tips I don't have freezes anymore, and using samba as server and W10 client, write speed still 80MBps ish, but read speed down to 50MBps.

 

And as you predicted, I have now another error : "reset SuperSpeed USB device number 3 using xhci-hcd"

But not in red :-) and without any noticeable delay.

 

Hopefully the USB "ERROR Transfer event for disabled endpoint or incorrect stream ring" gets solved soon.


Ill do some more testing on Pine and Armbian.

 

Also always interested if you have better fixes.

Link to comment
Share on other sites

Guest
This topic is now closed to further replies.
×
×
  • Create New...

Important Information

Terms of Use - Privacy Policy - Guidelines