ROCKPro64 as NAS?

[wolf7250]

Hi,

 

I am looking at investing in a RockPro64 NAS setup using the Pine64 Desktop casing and other necessary accessories. I'm hoping to run OpenMediaVault on it and have 2 3.5" HDDs attached. The entire setup would be configured to be a backup of my operational NAS (Helios4) in a different location (i.e. different room of the house).

 

My reading of the forum and the armbian download section has me a bit confused. The download section indicates that the board is supported, but the forum indicates it's a WIP.

Just trying to understand the status/stability of the board for what I'm planning on using it for and whether it would be a good investment or a waste of money.

 

Also wondering whether this post regarding updating the CPU governor is related -  and will be a benefit - to the RockPro64 as both the NanoPi M4 and RockPro64 share the same SoC RK3399.

 

Thanks

Edited February 11, 2021 by TRS-80
change title from "Stability of RockPro64"

[soerenderfor]

In my opinion it is a fine board, I run my ROCKPRO64 as a NAS. Runs 24/7 - wirh a sata card and 4 harddrivers in the nas case. OMV works fine, but there can be some probs with sata cards. 

I run krenel 5.7.13 focal, no problems what so ever. And it runs very stable.

[wolf7250]

Thanks very much for your feedback!

16 minutes ago, soerenderfor said:

OMV works fine, but there can be some probs with sata cards. 

Are there known SATA cards that should be avoided? Are you using the one from the Pine64 store?

Did you need to make any adjustments to the CPU governor?

[soerenderfor]

@wolf7250 - That card from pine64 should be avoided in my opinion, i did only have trubles with it.  I have only cards with Marvell chipset, and i found out buy a simple card not a fancy raid card - less problems. I can later today find my different cards and name them for you.

[wolf7250]

Thank you @soerenderfor! I really appreciate your help

[soerenderfor]

@wolf7250 - here you go, here is a couple i got home.

Delock 4 port SATA PCI Express Card - Marvell 88SE9215 - Works out of the box.
 

StarTech.com 4 Port PCI Express SATA III 6Gbps RAID Controller Card with Heatsink (PCIe SATA 3 Controller Adapter (PEXSAT34RH), Styreenhed til lagring (RAID), 4 Kanal, SATA 6Gb/s lavprofil, 600 MBps, RAID 0, 1, 10, JBOD, PCIe 2.0 x2, for P/N: ST521PMINT - Marvell 88SE9230) - Some problems. But i did mange to get it working.

DeLOCK - Lagringskontrol - SATA 6Gb/s - 600 MBps - RAID 0, 1, 10 - PCIe 2.0 x4 (Producent: DeLOCK | Model-nr.: DEL-89395 - Marvell 88SE9215) - Some probs. But did mange to get it working.

Ziyituod PCIe SATA Card, 4 Port (SA3014 - Marvell 9215) Works out of the box.

[wolf7250]

Thank you!

Struggling to find most of those options in Australia .

57 minutes ago, soerenderfor said:

Ziyituod PCIe SATA Card, 4 Port (SA3014 - Marvell 9215) Works out of the box.

I think this is the same one....
https://www.amazon.com/Ziyituod-Controller-Expansion-Profile-Non-Raid/dp/B07SZDK6CZ

[soerenderfor]

I have tested the Ziti... card for 1 week now, and runs very nice. So go for it, works right out of the box with new kernel also.

[Werner]

1 hour ago, soerenderfor said:

@wolf7250  not a fancy raid card

From my feel most of those are crap anyways and provide some weird/useles soft-raid solutions which can be realised via mdadm as well or even rely on that...

[soerenderfor]

One of the cards i have costed around 80euro, and it is almost useless.. The cheap cards is the best here, in my opinion.

[soerenderfor]

With new kernels the fan spins at "255" always, you can find the fan with this command 

sudo find /sys -name pwm1 | grep hwmon

and then change the output to your setup - But on the new kernels the fan point can jump, i am not sure of what reason.

[wolf7250]

Thanks again.

12 minutes ago, soerenderfor said:

change the output to your setup

Any recommendations for the output?

[soerenderfor]

@wolf7250 - I did cut a bigger hole in the orginal pine nas case, and fitted a 92mm Noctua fan. Complete silent, runs with the valuve "170" but as i say it is not standard fan..

[wolf7250]

On 8/7/2020 at 4:53 PM, soerenderfor said:

With new kernels the fan spins at "255" always

Ahh I didn't initially understand what you meant when you said this. But doing some research "255" refers to the PWM settings, correct? So with the new kernel if the fan spins always at "255" that means it always spins at it's maximum RPM correct?

 

On 8/8/2020 at 7:30 AM, soerenderfor said:

I did cut a bigger hole in the orginal pine nas case, and fitted a 92mm Noctua fan

OK. So Noctua fans appear to be very well respected and reviewed, and very high quality. The Pine64 case allows for an 80mm fan (which is also what they sell on their website). Did you feel that this was not big enough? I see that I could buy an 80mm Noctua fan and not need to cut a bigger hole in the NAS case.....

On 8/8/2020 at 7:30 AM, soerenderfor said:

Complete silent, runs with the valuve "170" but as i say it is not standard fan

So did you create a new PWM ramp for which the maximum was 170 (i.e 0 to 170)? Would I be correct in understanding that this would set the maximum fan speed at 66.66% of the maximum RPM?

 

Finally, from what I've found it seems as though all Noctua fans use a 3/4 pin connector. Whereas the RockPro64 header for the PWM fan is only 2-pin. How did you get around that?

[soerenderfor]

20 hours ago, wolf7250 said:

Ahh I didn't initially understand what you meant when you said this. But doing some research "255" refers to the PWM settings, correct? So with the new kernel if the fan spins always at "255" that means it always spins at it's maximum RPM correct?

Yes, "0" off and "255" full speed. You can change anything inbetween.

 

20 hours ago, wolf7250 said:

OK. So Noctua fans appear to be very well respected and reviewed, and very high quality. The Pine64 case allows for an 80mm fan (which is also what they sell on their website). Did you feel that this was not big enough? I see that I could buy an 80mm Noctua fan and not need to cut a bigger hole in the NAS case.....

I have 4 disk in the nas case, the 80mm noctua i did have installed before it was enough to cool it down, CPU temp around 40degress.  The orginal 80mm fan that "belongs" to the case is only around 10mm in thickness, and the noctua is 25mm. So to fit in the small space 90degree sata power cables is a very great idea. But you can fit normal sata power cables with some "violence" ;-)
 

 

20 hours ago, wolf7250 said:

So did you create a new PWM ramp for which the maximum was 170 (i.e 0 to 170)? Would I be correct in understanding that this would set the maximum fan speed at 66.66% of the maximum RPM?

No, I like always on, so i did find a number to fit my demands. But many fan scripts is avaible, but you have to fit it the fan controller (FAN name)

 

20 hours ago, wolf7250 said:

 

Finally, from what I've found it seems as though all Noctua fans use a 3/4 pin connector. Whereas the RockPro64 header for the PWM fan is only 2-pin. How did you get around that?

Just cut the cables and fit the plus and minus wire to the same cables at the fan. Very easy. Hope it was a little help, just ask again if there is something i can help with. Thanks.
The reason i did fit a 92mm noctua in the case was to get it silent, nothing else.

[wolf7250]

11 hours ago, soerenderfor said:

Hope it was a little help, just ask again if there is something i can help with

You have been an enormous help! Thank you.

I'll ask one more question if that's OK?

 

I notice that you mentioned you have 4 HDDs in the case, and the PCI-e Adapter you helped me with has 4 SATA connections. The power supply on the Pine64 store is 12V 5A. I'm pretty sure this is not powerful enough to power both the RockPro64 and 4 x 3.5" HDDs as the power supply for my Helios4 is 12V 8A. How are you powering your 4 HDDs and the board? How do I know how much and what sort of power supply I would need?

 

Thanks again 

[soerenderfor]

@wolf7250 - Just so you know, if you plan to use that NAS case from pine. In that case there is only room for 2x 3.5" and 2x 2.5". But back to your question, i have found a universal PSU from 7.5volt to 22.5volt and 10amp, so i did just use that one. 

Use 12V always to ROCKPRO64, and around (8A on the safe side) or so to power it all.

[wolf7250]

9 minutes ago, soerenderfor said:

Just so you know, if you plan to use that NAS case from pine. In that case there is only room for 2x 3.5" and 2x 2.5"

Yep, thanks. I knew that. It was more a question for down the track if I decided to use an old PC case or something to allow me to have more HDDs.

 

11 minutes ago, soerenderfor said:

Use 12V always to ROCKPRO64, and around (8A on the safe side) or so to power it all.

So the DC out for the SATA drives on the board can power 4x3.5" drives if needed if it has a 12V 8A PSU? You would just need a 4 way splitter to go from the one DC output to the 4 drives correct?

[soerenderfor]

3 minutes ago, wolf7250 said:

So the DC out for the SATA drives on the board can power 4x3.5" drives if needed if it has a 12V 8A PSU? You would just need a 4 way splitter to go from the one DC output to the 4 drives correct?

Did you buy the sata power cable from pine also? It splits the power into 2 sata cables and on both mount a splitter. Or you can like you say just split one cable to 4 with a sata splitter. I do power my rockpro64 with 4hdd in that way with a 12v 10a psu. It works like charm, no problem (but 8A is enough).

[wolf7250]

I haven't bought anything yet . I'm too cautious and wanted to make sure I'm doing the right thing and understanding what I'm doing first.

Thank you again, you've been very patient and very helpful!

[pfeerick]

I have been running the rockpro64 as a NAS for over a year now - since somewhere around 6 months after it's release. During it's time as a NAS, I have only had downtime due to maintenance on the power system it runs on (runs on a 12v solar system). I run a pair of USB Seagate 4TB Expansion HDDs (plus another pair I periodically network connect for offline backups), both shared out via minidlna and samba. It's running one of the ayufan-OMV images, so is still on OMV4.

 

I'm waiting to see if the 20.08 refresh of the images will bump the stable kernel for debian from 4.x to 5.x, and then I will migrate to Armbian + OMV5. I did try bumping the Buster 'stable' image kernel from 4.4 to 5.4 via `armbian-config` on a second rockpro64, but that resulted in what appears to be a no-boot scenario (no HDMI, no network), so I won't be trying that again in a hurry.

[TRS-80]

Like you guys, I have also zeroed in on this board for NAS usage.  When compared to other RK3399 based boards, I really appreciate the fact that Pine64 included a standard PCIe (x4) slot.  Everyone else is doing PCIe through GPIO/HAT which means custom parts (which are often unavailable) and/or dealing with fiddly ribbon cables or other DIY solutions.  It will be so much easier to source standard PCIe SATA boards now and in the future.

 

Anyway, I wanted to add my little bit of research and thoughts to the thread.  Please refer to following table (prices current as of date of post on AliExpress in USD and including shipping to US, except ASM1061 which is the one in Pine64 store and does not include shipping):

 

|-------+----------+-----+------------------+-------|
| Ports | PCIe gen | Phy | Chipset          | Price |
|-------+----------+-----+------------------+-------|
|     2 |      2.0 | x1  | ASMedia ASM1061  |  9.99 |
|     4 |        3 | x1  | ASMedia ASM1064  | 15.39 |
|     4 |        2 | x4  | Marvell 88SE9230 | 30.66 |
|     4 |      2.0 | x1  | Marvell 88SE9215 | 18.44 |
|-------+----------+-----+------------------+-------|

 

I very nearly bought (nowadays commonly available on AliExpress and cheap) ASM1064 based board, only to realize that it is PCIe gen 3 and ROCKPro64 only support PCIe gen 2.1.  Therefore, it would have likely become limited by x1 Phy (assuming >2 HDD, or SSD(s)), if it were even recognized at all.

 

Now because I am looking for 4 port board (I will probably add (2) 3.5" HDD now and 2 more later on), I think Marvell 88SE9230 based board is ideal for me.  If you are pretty sure you won't need more than 2 ports, there is also Marvell 88SE9215 which is cheaper.  I seem to recall a number of people (and not only in this thread) recommending to stay away from adapter they sell in Pine64 store (this is ASM1061 I included above for comparison).

 

Some more resources: Pine64 wiki ROCKPro64 Hardware Compatibility (PCIe devices) where they list a few more devices (and I also encourage everyone to add their experiences / data to the table there, I plan on doing so later on).

 

I actually ordered, then cancelled ASM1064 (after making above realization), but given its cheap price, now I am thinking I may order again strictly for science.

 

@wolf7250,

Since it's been a while now, did you ever get the project together?  What was your experience?

 

@pfeerick,

Which PCIe to SATA adapter have you been using?

 

It does seem that ROCKPro64 Armbian Stable is by now indeed on 5.x, which I am really excited about as now ZFS is also being packaged in Armbian and this is what I really want to try (and have for a few years now, in fact)!

 

On 8/7/2020 at 5:36 AM, soerenderfor said:

StarTech.com 4 Port [...] - Marvell 88SE9230) - Some problems. But i did mange to get it working.

 

As this is the same chip I purchased (but different board), I was curious what sort of problems and what did you do to get it working?

 

Of course I will eventually report back my experiences, too, although it will likely be a 2-3 months from now, unfortunately...

Edited February 11, 2021 by TRS-80

[soerenderfor]

@TRS-80 - I  do use i 5 port card now, works out of the box, and very cheap.. any ways.

If i remember correct, i did something like this:
 

cd /etc/udev/rules.d
sudo nano 99-9230.rules

in the file add this line
 

ACTION=="add", SUBSYSTEM=="pci", ATTR{vendor}=="0x1b4b", ATTR{device}=="0x9230", RUN+="/bin/bash -c 'echo %k > /sys/bus/pci/drivers/ahci/bind'"

reboot, and the card should be visible, in OMV

Hi, i have inserted link from a older thread, where we talked about rockpro64 as a NAS
 

 

[soerenderfor]

@TRS-80 - Got it working?

*Did forget your last line, with the reporting back info. 

[TRS-80]

Finally had time to play with this today.  I had same problem as you, @soerenderfor.  I did not recognize it at first however, because the card showed up in lspci right out of the box, without me having to do anything:

 

trs80@rockpro64:~$ lspci
00:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd RK3399 PCI Express Root Port
01:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9230 PCIe SATA 6Gb/s Controller (rev 11)

 

But the HDD(s) were not showing up in lsblk.  So I search internet again for "88SE9230 linux" which led me to this helpful thread at Pine64 forums discussing similar thing.  It was only then I remembered what you had said, and I came back here. 

 

So, one helpful thing from Pine64 thread above, is you can do this right away to test in your current session:

 

echo "0000:01:00.0" | sudo tee /sys/bus/pci/drivers/ahci/bind

 

After I did that, the drive showed up immediately in lsblk.

 

Of course, now I know that's the problem, I will need to make permanent settings as discussed in both these threads.

 

FWIW, someone (foresto) at end of Pine64 thread said (2020-08-20):

 

Quote

For the record, Debian unstable now supports the Marvell 88SE9235, so I expect the 9230 is probably supported as well. Tested with kernel 5.7.10. No custom udev rule required.

 

Now I just today installed Armbian_21.02.3_Rockpro64_buster_current_5.10.21.img and I still had to do this workaround, so I guess YMMV.

 

I am really happy to have this working finally.  I have been wanting to build a 64-bit ARM / RK3399 based NAS (so I can do ZFS) for a long time now.   

[soerenderfor]

@TRS-80 - Nice found, i have been testing around 10 sata cards, all of them 4 or 4+ ports.. And all "dirt" cheap cards have all been working, also out of the box.. But the more expensive cards have problems, all of them..

I now use some "no name or so" PCI-E 3.0, 5 port card, with 5 SSD disks. And that card also can boot the system (Not all the cards can)

@TRS-80 - If i should test something for you with the 5 port JMicron card i have, just tell and i will do so.

Did look just now, the card is with JMicron chipset.

[TRS-80]

I guess, as long as we can find these cheap cards on AliExpress (or wherever), and they are supported in Linux, the only other thing to test will perhaps be longer term reliability?

 

Like you, I also plan on building more of these ARM based NAS (eventually).  This one is just a bit of a prototype.  So I may get different cards in future.  In which case I will try and remember to report back results here (or perhaps even better, add them to Pine64 wiki).

 

I did not think about a 5 port card, I guess it would be useful for some 2 + 2 mirrors plus a hot spare.  Not a bad idea, now that I think about it.  Maybe next time...

 

@soerenderfor,

 

Are you using any ZFS perchance?  I have been reading all the relevant threads and issues (I think), but I am still a bit confused if it actually is ready to go in Armbian or not yet, whether we still need to compile, or what.  I was going to make a new thread about that, but I guess I try to ask you here first. 

[soerenderfor]

@TRS-80 - I haven't tried ZFS, but sounds interesting - Is the ZFS in  OMV good enough? I can install and try it out? The card i use now has been running for months now, also heavy read and write tasks, many different kernels, no problems at all. So stabil for sure. But i can try the ZFS out, you just say the word.

[TRS-80]

Well, my interests were more in a practical "how do I install it", assuming regular tools and command line, etc. (personally, I don't prefer GUI stuff like OMV).

 

ZFS is the cat's meow, if you are concerned about reliable filesystems.  If you are not aware of what it is and how it works, do yourself a favor and spend a few minutes reading about it.  It's very impressive!  And amazing to me that mere mortals like us can nowadays obtain industrial grade filesystems like this, thanks to the power of F/LOSS!

 

I do appreciate the offer though, mate.   

[soerenderfor]

@TRS-80 - Ohh yes, i know your opinion about GUI - I will check it out, i have a ROCKPRO64 with 5 SSD just sitting on the shelf.. What do you want to do with it? RAID BACKUP?

what about

apt install spl-dkms zfs-dkms

it it fails you can check the status with

 

dkms status