NanoPi M4V2: Standard PCIe Devices Anyone?


Recommended Posts

Hello,

 

I've been using a M4V2 since its release mostly as a NAS (disks over UBS 3.0). I also have it booting from a Samsung NVMe with the official NVMe hat. My setup works flawlessly however I was thinking about ways to add more storage with reasonable performance/price so I thought, what if I managed to add a standard PCIe x1 port from the pins on the board and use standard hardware with it.

 

The idea was to add a cheap PCI switch + NVMe adapter (to boot from) and a PCI to SATA controller such as the following:

 

1) PCI Switch: https://www.ebay.com/itm/PCI-Express-1X-to-4X-PCI-E-4-Ports-Switch-Multiplier-Expansion-Riser-Card-New/293564888755

2) PCI > NVMe Adapter: https://www.ebay.com/itm/NVMe-PCIe-x4-x2-M-2-NGFF-SSD-to-PCIe-x1-converter-card-adapter-PCIe-x1-to-M-00e/143915406968

3) PCI > Sata: https://www.ebay.com/itm/PCI-Express-1x-To-4-Port-Sata-3-0-6G-Expansion-Controller-Card-Marvell-88SE9215/233219670613

 

final.thumb.jpg.510598b383706470e90f9237698e6939.jpg

 

Did anyone had success with a setup like this? It looks like there isn't any adapter for the board to add a standard PCI interface, however I could probably solder something. Software wise would it work? I believe the Marvell 88SE9215 is supported by Armbian from what I read on espressobin threads. Any tips?

 

Thank you!

 

Link to post
Share on other sites

Armbian is a community driven open source project. Do you like to contribute your code?

So, after some research I decided to tune down a bit my project. I decided to ditch the NVMe as boot device because I believe a regular SATA SSD will be enough for my use case.

 

My goal was:

  • Boot from a SATA SSD;
  • Have around 10 SATA ports for 3.5" hard drives.

I ended up buying the following:

 

1332114199_Screenshot2021-05-16010251.png.d335020ee602351ce2e0a4875b95d3e2.png

 

I chose this PCIe to SATA because I already had the NVMe adapter for my M4v2 and it uses the JMB585 chip is used by other SBC manufacturers in hats thus compatible with Armbian. This chip also features "Cascaded mode: support up to x75 SATA devices with JMB575" that allows me to pair it with the other board (based on JMB575) to have more SATA ports.

Link to post
Share on other sites

Here, my testing setup with both boards. Everything powered up by a standard ATX power supply.

 

678089741_Screenshot2021-05-16011628.png.e7f3f51c87fb48c132b0054171a37607.png

 

Now I did run some benchmarks on the disks.

 

System/Kernel: Armbian 21.05.1 Buster with Linux 5.10.35-rockchip64
Hard Drives: Seagate Barracuda 7200rpm 1TB. ST31000524AS (hard drives from 06/2012)

https://www.seagate.com/docs/pdf/en-GB/datasheet/disc/barracuda-7200-12-ds1668-6-1101gb.pdf

1599652280_Screenshot2021-05-16013205.png.55545bb9489f34618376491a6fa58006.png

 

Scenario: Amrbian installed to the SATA hard drive using the armbian-config helper tool. BTRFS filesystem used.

Benchmark Command: dd if=/dev/zero of=test bs=100MB count=100 conv=fdatasync

Notes: I enabled the PCIe x2 overlay.

 

- HDD Attached to the SATA to M2 Board (HDD > SATA to M2 > NanoPI M4v2)

10000000000 bytes (10 GB, 9.3 GiB) copied, 27.248 s, 367 MB/s
10000000000 bytes (10 GB, 9.3 GiB) copied, 24.6339 s, 406 MB/s

 

- HDD Attached to the SATA Expansion Board (HDD > SATA Expansion Board > SATA to M2 > NanoPI M4v2)

10000000000 bytes (10 GB, 9.3 GiB) copied, 25.1813 s, 397 MB/s

 

Conclusion: I run the test multiple times I was getting around 350-400MB/s regardless of having the disk connected to the SATA Expansion board or to the SATA to M2 board directly.

 

I run the same test by while booting from an eMMC card. The performance is the same if the the disks are mounted with:

 

mount -t btrfs -o defaults,noatime,commit=600,compress=lzo,x-gvfs-hide /dev/sda1 /mnt/HDD

 

Interestingly the performance of ext4 was very bad (tested both as boot device and simple mount):

 

Quote

10000000000 bytes (10 GB, 9.3 GiB) copied, 79.0775 s, 126 MB/s

 

So it seems I'm able to build a low power and high capacity NAS with this setup. Please note that I'm not after the best performance and parallel wiring to disks. I also seem to be able to write simultaneously to two disks without a performance drop and that's enough for my use case.

 

Does anyone has a similar setup?

 

Link to post
Share on other sites

vor 15 Stunden schrieb TCB13:

Does anyone has a similar setup?

Well - not similar from assembly, but similar in use case

I have Sata Tower from radxa. I have 4 ssd attached, but I didn't run any benchmarks. I love the small form factor, and its going to be useful after I ported driver from radxa.

Link to post
Share on other sites

3 hours ago, tparys said:

But before you reinvent the wheel, have you considered that you basically made a Helios64 without the nice enclosure?

 

Yes I did, however the limitation on the number of bays x price didn't make me very happy. My current setup with more SATA expansion boards can go up to 20 bays without cascading and 75 with cascading.

 

And.. I already had a M4v2 with a the NVMe adapter so... No more costs there.

Link to post
Share on other sites

2 hours ago, tony013 said:

Well - not similar from assembly, but similar in use case

I have Sata Tower from radxa. I have 4 ssd attached, but I didn't run any benchmarks. I love the small form factor, and its going to be useful after I ported driver from radxa.

 

Are you going to use it with the NanoPI?

Link to post
Share on other sites

vor 7 Stunden schrieb TCB13:

Are you going to use it with the NanoPI?

No - but look:  from what I read, your nanopi is a big size nanopi and is a rk3399 device.

I have a rockpi - normal size - and a rk3399 device too. So I think, both have more in common, than they differ.

I guess, we build the same image but may have different dtb and uboot.

Some pi software (like pigpio) does not work on my rockpi - and I guess, same is true for yours.

I try many os images, to find out, what the hardware supports, and what is software limitations.

For example - I build LE images on my own, but failed to build armbian. Images supported by radxa are pretty old. Twister is the most recent one, that seems to fulfill my needs. But my driver only works on LE. With armbian the button does not work and with twister the pwm does not work. So I have to start mixing my own flavour.

Link to post
Share on other sites

6 hours ago, tony013 said:

I have a rockpi - normal size - and a rk3399 device too. So I think, both have more in common, than they differ.

 

Indeed they've more in common than they differ.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
 Share

2 2