So, you want to run a file server?

[TRS-80]

This is not so much a 'Tutorial' or a 'Review' as it is an overview (and/or starting point for further research) of some things to consider when setting up a file server, some in general and then some more particular to SBC.

 

In the simplest case, you could attach almost any drive to almost any board, by any number of ways, and you have yourself 'a file server.'  However with even a small amount of effort, there are so many better ways to do things nowadays.  We have a lot more interesting hardware choices, advanced filesystems, etc. and so those are mostly what I want to talk about.

 

But first, some context.

 

Dedicated distros vs simple directory sharing

 

Personally I am not a fan of dedicated distros like FreeNAS, OMV, etc. as I consider them to be too 'heavy weight' and single solution specific.  I mean, if you need some/all of those features, or you like them or whatever, then great, use them.

 

However, a 'file server' is one of the simplest use cases of a 'server' and I am sure one which almost everyone buying a SBC will do at some point.  IMO, all one needs to do is simply share some folder(s) via either NFS (if your network is mostly GNU/Linux machines) or Samba/SMB (if your network is mostly Windows machines).  And that's it, you're done!  Simple, lightweight, and using standard protocols (well, a little less 'standard' in case of Windows, but I digress ).

 

You could also add any of the software included in the 'all in one' solutions in first paragraph on a 'piece by piece' basis, if/when you need them.  Personally I am more of a fan of starting small (vanilla Armbian base) and then adding things you need one by one.  But you are of course free to do whatever you want.

 

Setting any of that up, or debating about it, is considered off-topic for this thread, as there are tons of resources already around the Internet about those things.  So I want to focus on other, (IMO) more interesting, aspects.  You are welcome to make your own thread with a different focus, of course.

 

Important note about hardware RAID

 

RAID, in case you were unaware, means "Redundant Array of Independent Disks."

 

For a looooooong time, I (and I think many others with SOHO NAS perspective) thought that 'hardware RAID was the only proper way to do it' but in the course of researching ZFS I eventually became convinced otherwise.  In fact, apparently, historically 'big-iron' filesystems (Solaris, others) were always run with software RAID in the enterprise as there are a lot of advantages to that.

 

Just think about it.  If your hardware RAID card dies, you need to replace it with exactly the same model, before you can even begin to recover your data.  With software RAID, you just re-connect drives and you are back in business (with ZFS, you can even connect them in any random order, amazing!).

 

In particular if you are using ZFS, hardware RAID controllers are not recommended.  I am not sure how much this applies to other similar software RAID systems, but it would stand to reason that it does (IMO).

 

Hardware controllers are also typically very expensive and proprietary.  Where software RAID uses readily available and inexpensive commodity hardware (while actually increasing reliability).  All of which is why I have become such a big fan of it, whether we are talking about ZFS (which I like) or some other software RAID solution you may prefer instead.

 

Now, with that out of the way, onward!

 

Filesystem

 

Assuming you came to similar conclusions as I did in previous section already, this is where our decision making process really begins in earnest IMO.  As the way I see it, choice of file system will dictate topology, which will in turn dictate choice of hardware (which SBC to buy).

 

Of course if you already bought some SBC, you are in a different boat (and working backwards, from my point of view ).  Anyway, you will have to adapt accordingly (and/or buy different hardware).  However, if you are already there, go to 'Historical Overview' section, and maybe you can find some solution there which will work with the hardware you already have.

 

Now, back on track, we are really lucky nowadays, thanks to F/LOSS, to have some industrial grade, software based filesystems available to us mere mortals! 

 

I like ZFS, but certainly, opinions will differ.  Some people like btrfs, in fact I got into a lengthy (but IMO informative) debate with @tkaiser about that.

 

Then there are other distributed things like Ceph (and others) which I will admit to knowing less about.  If you do, kindly add your thoughts to the thread when you have a moment.

 

Point being, hardware setup for someone like me (using ZFS) or even someone using btrfs will be perhaps quite different than someone using something distributed like Ceph.  Just keep that in mind.

 

Historical Overview (SBC Hardware)

 

I want to cover some of things which came before, as I think they are relevant to understanding where we are today.

 

In the beginning, there was darkness.  Wait, let me fast forward a bit. 

 

I will start with the creation of Armbian.  Which happened, in fact, because Igor was trying to set up a file server on a cubietruck.  True story (and like I said, you are not alone, this is a common use-case)!    Anyway, many of us bought (A20 based) cubietruck back then, because they sold a small power adapter board which made this easy and they also had a SATA connector right on the board.  I think they were one of first to do this in fact (I could be wrong).

 

After that, a lot of us used (more powerful Samsung Exynos5 based octo-core) ODROID-XU4, along with a good quality USB to SATA adapter with UASP.  That one in particular was recommended by tkaiser a long time ago, however -- as of time of writing this post -- PINE64 still sell it in their store(!).  And it may be an 'okay' solution if you have some (especially, USB3 capable) board laying around that you just want to use.  Some people are still using this today.  But we have a lot better options now.

 

Then RK3399 came on the scene.  Which was very exciting, because these support not only PCIe, but they are also 64-bit (which is required for ZFS).  Unfortunately, it took literally years for these to become really stable.  Lucky for you, this is all in the past by now.  And so, currently, many people will probably recommend RK3399 as a file (or other) server.

 

There are a lot of other boards out there, which I do not have as much experience with.  But I also try to note only historically significant developments here (at least the way I see them).  If you have any to add, please comment below or contact me somehow (PM or IRC) and I will edit this post and add them.

 

Usage (other than file server)

 

As I already stated, almost any cheap board can be a (minimal) 'file server.'  From that point of view, more powerful boards (like RK3399, or others) are overkill.  However maybe you want better throughput.  Or more disks.  Or, like me, you want to run advanced filesystem like ZFS (also with more disks).

 

Another thing which happens is that, once you handle the simple case of serving files, before you know it you will probably want to start doing more than that.  You may already have some more demanding requirements in mind beforehand.

 

For all these reasons, a slightly more powerful (and more expensive) board may be a better choice, as you will not be limited in the future.  But if you are very sure about your needs and wants, and have perhaps other boards on your network to handle processing intensive tasks, this may not apply to you.  Just something to consider.

 

Hardware Recommendations (RK3399 based)

 

For reasons mentioned at end of 'Historical Overview' section, many of us are fans of RK3399 based devices nowadays.

 

Within this subset, you will get a lot of opinions.    I know because I spent literally years asking people (including developers), while I was waiting for RK3399 to mature.  Some developers think certain vendors make crappy hardware (and maybe they are right).  Or they prefer the boards they know and develop on (naturally, IMO).  Some vendors support the project a lot more than others (PINE64, notably, do not support Armbian at all to my knowledge).  So you will have to do some amount of your own research, and form your own opinion here.

 

I don't want to discount any of the above criteria too much.  However, in the end, at least for me, one criteria bubbled to the top.  And that was direct PCIe (or SATA) support.  Since we are talking about RK3399 here, this means PCIe support.  Which of course is in the SoC, but that is not what I am talking about.  I am talking about putting a standard PCIe physical interface directly on the board.

 

This way, you can buy any standard cheap/dumb PCIe to SATA adapter card.  You can keep spares.  You will be able to obtain them for the foreseeable future.

 

By contrast, you can certainly rig up your own adapters, cables, etc. from GPIO.  But personally, I find that fiddly and it doesn't strike me as the most reliable.  I know some people are doing this just fine though.

 

Another way, is that many vendors sell some sort of HAT, which provides SATA connectivity.  However I find this short-sighted and -- quite frankly -- self-serving.  What if they are out of stock, or stop making them?  You are out of luck.  Or you would need to revert to the above option (rigging up cables and adapters).  It's good for them selling add-on hardware though I guess -- hence my 'self-serving' comment -- but maybe not the best for you and me in terms of keeping our important data available and up and running, on a long term basis.

 

Therefore I say, skip all of that and go with a board that has a standard physical PCIe right on the board.

 

ROCKPro64

 

And AFAIK, there is only one (standalone SBC) RK3399 that meets that description, which is ROCKPro64.  Of course, I am not the first to come to same conclusion, in fact there is a whole thread where myself and some other like-minded folks discuss using ROCKPro64 as NAS.

 

I say 'standalone SBC', because I almost forgot...

 

Helios 64

 

These of course are no longer made, and Kobol, our once partners, have unfortunately closed up shop.  However they still come up for sale on the forums from time to time (usually in Kobol Club but could be anywhere).

 

Here you would be getting a more integrated solution, often including a nice case, for perhaps a bit more money.

 

Some people have had problems with stability though, which I am not sure were ever fully worked out.  Have a look through Kobol club (linked above) and form your own opinion.

 

Hardware Recommendations (not RK3399 based)

 

ODROID HC4

 

ODROID HC4 was also brought up today, when this subject came up in IRC, by @rpardini as that is what he uses.  This is an interesting alternative to the above, except Amlogic (S905X3) based (like a lot of TV boxes).  Which to me means two things:

 

1. It almost certainly requires blobs.

2. It's probably cheaper than RK3399.

 

Well in fairness ROCKPro64 probably requires some blobs, too.  But somehow I get the impression that Amlogic are really obnoxious with their blobs, and I am not a fan of them as a company because of this.  Surely a developer will need to correct me here. 

 

Anyway, I am not sure how it compares in processing power.  I suspect less than RK3399(?).  However for media transcoding, it may actually be better (perhaps hardware decoders(?), speculation on my part).

 

Also for simple 'file server' use-case, in particular with Ceph or something like that, it becomes more interesting IMO.  Again, quoting rpardini from IRC today: "scale out, not up."

 

Future

 

There are newer SoCs, like RK3566 and others.  At this time, they are a lot like RK3399 was early on.  Very exciting, but not able to realize their full potential due to lack of (software/Linux) development.

 

The consensus among developers seems to be that RK3566 development should go faster/easier than RK3399 development, because of all the ground work that was laid in the course of the latter.  Maybe that's true, but we are still talking about some time.  Maybe less years, but still significant time.

 

If/when this changes, I will try and remember to update the post.  Feel free to ping me by posting, I should get notified.

 

Others?

 

Possibly anything in NAS category.  Which I mostly covered already (above, in more detail) but can always change in future.

 

Certainly I forgot (or don't know about) some others.  I have shared what I learned from my research the last several years while looking for solution for myself.  However I do want to make this a more general resource that we can all point to, so please discuss, and I will be happy to edit this post later on and add relevant info.

 

To clarify, I am not talking about 'any SBC which can be made into a file server' which is almost any of them, but rather particular hardware which is interesting for some specific reason(s), especially currently (or perhaps historically, which I would add to that section).

 

Cheers!