DAS Mode and Capture One

[alanjames]

Had anyone tested DAS mode performance with Capture One? Or could someone explain to me how it works? Also, would C1 show the DAS mode and NAS mode as seperate volumes or is it smart enough to show up as the same one?

 

Background: Capture One doesn't work well with network drives. Editing images, moving raw files, even saving catalog backups is dramatically slower on a NAS than a DAS or internal drive. It doesn't matter how fast the drive or network is. Its a long standing bug they're always quick to say they aren't prioritizing fixing. I've looked at other NAS's that do a DAS mode by running a 10GBE connection over thunderbolt and that's a great solution in every situation except for working with C1. Has anyone tried this?

[lalaw]

This is my exact use case.  My hope is to get DAS mode running for my main machine, while keeping photos accessible to other devices and backing up offsite.

 

My reading seemed to indicate that using Helios64 as a Mass Storage Gadget was still a work in progress; I'd love to know if anyone knows otherwise. 

[gprovost]

In case you haven't looked up this section of our wiki : https://wiki.kobol.io/helios64/usb/#helios64-as-direct-attached-storage-das-device

[lalaw]

Bunch of random questions:

 

1. Any ETA on when DAS mode would be rolled into kernel 5.x?  

2. Any ideas on how I can get around the 1 post per day limitation?

 

If I can take the liberty, I'd like to describe my setup and solicit suggestions.  

 

I have am running Capture1 on a Mac connected via USB-C.  I'd love to use the Helios64 as a r/w DAS for the Capture1, while still making it available (perhaps ro) to enable network access and backup.

 

Some issues that I foresee:

1. Because the DAS mode is using the Mass Storage Gadget, it seems like my choices are to expose the entire disk/block device or to have a file that simulates the block devices.  The problem with the file is that it would no longer appears as a filesystem in NAS mode, correct?

2. If my assumption in (1) is true, then I have to expose the entire disk.  For the Mac to read the disk, I assume I need to use exFat or Fat32 (as ext4, ZFS, etc. support on Mac sound spotty for r/w)

3. exFat/Fat32 don't appear to be first class citizens in the world of OMV, but it sounds like I can cobble together support and mount those disks manually

4. Once sorted out, would it be possible to combine the Capture1 disk with other NAS-dedicated disks in a SnapRaid group for redundancy?  And expose the Capture1 disk as a (r/o) share on the NAS?

 

My hope is that I can have Capture1 see as a local DAS for work, I can get some level of disk failure redundancy though period SnapRaid parity syncs, and I can create some sort of rsync job to maintain an offsite backup in case I have a bigger failure.

[aprayoga]

On 1/9/2021 at 5:21 AM, lalaw said:

1. Any ETA on when DAS mode would be rolled into kernel 5.x? 

You can use DAS mode on Kernel 5.9 using device tree overlay

It is not as seamless as in LK 4.4 where you can just change to cable to switch the usb role but it work.

On 1/9/2021 at 5:21 AM, lalaw said:

2. Any ideas on how I can get around the 1 post per day limitation?

I gave you a like, it should lift the limitation.

 

On 1/9/2021 at 5:21 AM, lalaw said:

Some issues that I foresee:

1. Because the DAS mode is using the Mass Storage Gadget, it seems like my choices are to expose the entire disk/block device or to have a file that simulates the block devices.  The problem with the file is that it would no longer appears as a filesystem in NAS mode, correct?

2. If my assumption in (1) is true, then I have to expose the entire disk.  For the Mac to read the disk, I assume I need to use exFat or Fat32 (as ext4, ZFS, etc. support on Mac sound spotty for r/w)

3. exFat/Fat32 don't appear to be first class citizens in the world of OMV, but it sounds like I can cobble together support and mount those disks manually

4. Once sorted out, would it be possible to combine the Capture1 disk with other NAS-dedicated disks in a SnapRaid group for redundancy?  And expose the Capture1 disk as a (r/o) share on the NAS?

1. correct

2. Yes, you'd need to use filesystem that can be recognized by both, Linux and Mac.

With FAT32 there is maximum partition size and maximum file size (4GB) limitation.

With exFAT, it might put some load to cpu during NAS because of FUSE but it might be change in future where in kernel module used instead of FUSE.

4. I don't have experience with SnapRaid.

 

 

There is another method you can explore to use the DAS mode, expose the system as MTP device. It should not have issue with exclusive access to block device and the filesystem type.

[lalaw]

Quick update here, I was running some benchmarks today.  Off the top of my head:

 

1. local SSD was getting something like 1200 MB/s write and 1500 MB/s read

2. USB connected HDD was getting ~175 MB/s write and 175 MB/s read

3. NAS (not helios) connected at 1 Gbps was getting ~100 MB/s read/write

4. Helios in DAS mode and on Raid 10 was getting approximately 170 write and 250 read.

5. Helios in DAS mode on a single drive was getting 190 write and 220 read. 

 

For the capture 1 use case, I think a solid contender.  Unfortunately, as I described in the other thread, I can't get my whole array presented in DAS mode (it wants to give me 200GB for a 18TB array).

 

 

Updated: also ran on a single drive instead of the array.  Slightly better write numbers with slightly worse read numbers.  Also, frustratingly, drive capacity was off -- this time showing 1.2 TB instead of 10.

[alanjames]

That's good news! Thanks for testing that out, I'll keep an eye on the capacity issue

[aprayoga]

@lalaw  unfortunately i never use Mac, i have no idea why it always recognized only 210 GB.