Self-hosting micro- (or regular) services, containers, homelab, etc.

[TRS-80]

I have been having some long running, on and off discussion with @lanefu in IRC (and elsewhere), because I know this is his area of expertise.  However he is busy, and I can appreciate he might not want to spend his free time talking about stuff he gets paid to do M-F. 

 

Then I also realized, certainly I am not the only one interested in Armbian in order to be doing such things.  So I thought I would make a thread about this, instead, to open the discussion to a larger group people here in the broader community.

 

I have my own thoughts, concerns, things I want to do, and have already done, but I thought I would make a more general thread where like minded people can discuss their experiences, what has worked (or not), what services you might be running (or would like to), etc.

 

I guess I will begin by saying I have been running some services for family and friends, first on a Cubietruck since maybe 2017 (or earlier?) and then later on some ODROID-XU4 in addition.  I run XMPP server (Prosody) which has been very hassle free "just works" as well as some experiments into Home Automation and other things which have had some, well let's just say mixed results. 

 

Currently my interest is in containers and like technologies, but I am very new to it.  I been reading a lot, but there is so much to know and I can't help but feel I am a bit late to the game.  That's by design though, as I don't like being (too) early of a technology adopter, generally speaking.  Well, certainly not with such "hyped" technologies like containers, anyway.

 

In fact I thought it was a bunch of baloney and hype for a long time, but even cranky old cynic like me now sees some advantages of abstracting things, changing servers/services from "pets" to "cattle" if you will, for reproducibility, redundancy, and other reasons.

 

Anyway, I am prone to walls of text of idiotic ramblings, so I will stop here and give some others a chance.    Hope everyone is enjoying their holiday weekend so far. 

[jeanrhum]

I don't see real questions in your post, but here is my personal experience.

I run armbian on sbcs since 2016 and used it on one of them for home server. My current server is based on a s912 tv box with a 4.20 kernel installed maybe two years ago (I don't remember exactly ):

Welcome to ARMBIAN 5.75 user-built Debian GNU/Linux 9 (stretch) 4.20.5-aml-s912
System load:   0.03 0.02 0.00  	Up time:       135 days

 

The system is amazingly stable and the only improvement I'm looking at, is maybe upgrading to a board/sbc having usb3 or sata(but since my local network is only 100MB, no real need for now). On my next install, I plan to use docker for several applications that I installed the classic way. Why:

- no more problems with package dependencies not available or not update to date: I use home-assistant and I'm stuck on v 0.96 since after python 3.5 is mandatory and it is not easily available on stretch.

- easily upgrade the applications: remove the container, upgrade the image and launch it again!

- easily install and remove applications you want to test.

The main drawback I see, is the disk space and the memory consumption that may be more important.

 

I've used several test configs to replace my current server, but they mainly failed on the stability criterion, but my last try with pineh64-b seems promising. I also want to switch from armbianTV's Balbes images to official Armbian, since support for amlogic SoCs stops and also upgrade seems more robust with official Armbian builds (maybe true before but now armbianTv builds are more similar than original armbian). What I've installed on docker is:

- home-assistant (official image)

- zigbee2mqtt (official image)

- pyload (linuxserver.io image)

- jellyfin (linuxserver.io image) (previously plex)

- airsonic (linuxserver.io image)

I'm looking to use again openmediavault, but no official container image seems available and I didn't take time to look for it. I'm also looking at linuxerver.io images to check possible additional services to host, since their images seems well "packaged".

For me now, the main objective is to use a board that can be "basic" (but with enough memory and GBE), then to install and configure a "basic" and robust host system easy to maintain, finally to launch containers for which I have just the necessary config to set in dedicated folders (easy to save).

[TRS-80]

On 11/30/2020 at 9:09 AM, jeanrhum said:

I don't see real questions in your post

 

Yeah just trying to get a cozy thread going.

 

I am still reading and studying about it.  Which I tend to to extensively, perhaps to a fault even some times.

 

On 11/30/2020 at 9:09 AM, jeanrhum said:

s912 tv box with a 4.20 kernel installed maybe two years ago (I don't remember exactly ):

Up time:       135 days

 

Amazing, I would never have thought such things were possible with one of (oft maligned) "TV Boxes"!    Well done! 

 

On 11/30/2020 at 9:09 AM, jeanrhum said:

On my next install, I plan to use docker for several applications that I installed the classic way.

 

This is pretty much right where I am at, and mostly for all the same reasons.  In addition I think I became convinced of the wisdom of viewing the services I am running as "cattle" and not "pets" as they say.

 

My current Cubietruck is definitely a "pet."  Bespoke manual configuration, I spent a lot of time setting each of those services up, etc.  Therefore to set that up all again would take a lot of time.

 

Now I think, well if I have service A, B, or C defined as containers, they are more like "cattle."  They can die off and just be re-created.  If one SBC (hardware) goes down, I just start the services on a different SBC.

 

Taking that philosophy to the next level, if I have some SBC at friend/family house, even if my Internet goes down, I could re-start services over there and just re-direct DNS.  Potentially even automatically (i.e., failover).  But that gets much more complicated, I am aware.  So baby steps.  But I am excited at possibilities of thinking in a different way.

 

On 11/30/2020 at 9:09 AM, jeanrhum said:

they mainly failed on the stability criterion

 

By doing my homework and sticking with well supported devices (Cubietruck and ODROID-XU4, but there are others) I have had very good "luck."  But I also take care of all important details like good brand name sdcards (or more recently, eMMC), well tested upon arrival, from known suppliers, high quality (Mean Well) power supplies, etc...  In case all of that fails, I have multiple physical spares as well (not only for SBC, but all my "infrastructure").  You probably know all this.  And I am still amazed at your "TV Box" results which fly in the face of everything I think "I know." 

 

My issue is learning all this stuff, time and implementation.  I only started with GNU/Linux around the time I bought my Cubietruck, so I think I came a pretty good way so far.  Just time now I think to take it to next level which will be to start playing with containers.

 

I fired up my first container just the other day, the official Docker (which is now recommended) version of Tiny Tiny RSS.  I was able to log in and add some feeds.  It worked a treat and was a piece of cake.  Last night I play around just launching containers with docker run and exploring shell in Alpine or other OS on command line.  Real sh is a bit different for me, coming from bash.  All those "bash-isms" I read about at Wooledge FAQ over the years coming back to bite me now. 

 

But a lot of other things I want to do, I know will be more complicated.  So I keep studying, and thinking about architectural decisions.  And picking @lanefu brain when he's not too busy. 

 

[lanefu]

Ready fire aim

[TRS-80]

I wanted to share an article I came across, I thought was pretty good.  Keep in mind I am starting from zero with Docker recently, if you are already familiar, this article might be too basic for you.  The article walks you through several of basic Docker tasks (create, run, inspect, ssh into, etc.) on command line:

 

Getting started with Docker for the Inquisitive Mind

 

They have other related content on their site as well which I have only skimmed so far, but it looks pretty good.  I always appreciate websites with clean looking CSS that work well (including mouseover effects) without requiring JavaScript to be enabled (which I block by default). 

 

What's everyone else been up to?  Come on, don't be shy now. 

[forestj]

I have been self-hosting my blog (ghost), git server (gogs), and a few other small things for years: https://sequentialread.com/.  I'm not using an SBC for it, rather a very old dell desktop, but I think the principle is the same. I use docker-compose to run the apps on my server.  I worked as a DevOps engineer for a couple years so I am intimately familiar with docker and linux containers in general.   So to me, setting up containers and what not (anything which is done inside the OS of the server) is the super duper easy part, its highly automatable, it can all be defined with code, like with ansible for example. 

I think there are 2 main pain points for self hosting, both of which don't really have any great solutions out there on the world wild web:

1.  Configuring your router is a pain in the butt, especially for non-technical people & may not be possible for many users.
2.  If you want to use your self-hosted services for something "mission critical" like email, you probably are going to want monitoring, observability, automated failover, etc.  These things are really complex and hard to set up, there's no "ez mode" button or framework that one can use, especially not anything designed for self-hosting.

I actually started a project to try to address those pain points, starting with #1.  I wrote a bit about it years ago here: https://sequentialread.com/pragmatic-path-towards-non-technical-users-owning-their-own-data/

I also decided to try to coin a term to describe what I'm trying to create, a "personal datacenter", basically like a personal computer, but instead of having a screen, keyboard, and mouse, it's a web server or group of web servers that are accessible on the public internet, and one can easily run various "apps" on these servers to make a blog, email server, online store, git server, etc.   I know  there are already other projects out there like nextcloud, freedombox, yunohost, etc, but I don't think any of those projects really address the hard pain points of self hosting, the network routing setup & failover.   I don't wanna go too much in depth with my project here, I'll probably make my own thread about it at some point. I don't quite have enough done to show it off properly yet
 

Edited December 11, 2020 by forestj

[forestj]

Someone in IRC brought this up regarding self-hosting when I lamented that there aren't any user friendly projects to make self-hosting easier:

https://syncloud.org/setup.html

 

However it sounds like they still expect the user to configure port forwarding on the router as a pre-requisite, which is a big no-no in my opinion:

 

https://github.com/syncloud/platform/wiki/Things-to-know-before-buying-a-device
https://github.com/syncloud/platform/wiki/External-access

 

 

[lanefu]

Im a fan of the linuxserver.io project.  All of their containers are multiarch.  Lots of good turnkey stuff there

[DevShanky]

I have used docker containers on Armbain in past that hosted .Net 5 Blazor applications and they run just fine. You can also remotely update your docker images which makes a good case for software updates on those systems.

 

-R

 

[TRS-80]

On 12/13/2020 at 2:45 PM, lanefu said:

Im a fan of the linuxserver.io project.  All of their containers are multiarch.  Lots of good turnkey stuff there

 

I have heard lots of people saying this, actually.

 

I did not immediately realize the importance of your "multiarch" comment (especially for us here at Armbian!), until I was reading something at r/docker about someone who was trying to use x86 container on ARM which of course is not going to work.  As containers are, after all, just some upper part of filesystem overlay + some isolation, but still relying on underlying hardware of course.

 

I went to linuxservers.io and was checking out their blog last night, some good and relevant articles there, IMO.  Seem like pretty cool bunch of dudes.    A much smaller (and less slickly marketed) group than I was expecting, to be honest, based on their very outsized awareness / reputation in container circles around the Internet already.  Good for them!

 

---

 

Another article I came across some time ago, which I thought was a pretty good exposition of some of these various technologies and products I keep reading about around (and under) containers, including which companies they came from, along with a bit of history:

 

You Don’t Have to Use Docker Anymore

 

That sort of thing is very valuable for someone like me that is new to it all and may not have been following the evolution of it all along as some of you have.  I think it's also important to understand what companies are behind which technologies and why, as an important metric before investing any significant amount of my own precious personal resources (time, energy) into learning any more about said technologies (much less using or becoming more "invested" in them).

 

One of main things I have been picking lanefu brain about is why he like certain tools he does in preference to Docker.  I don't think they mention his preferred tools in the above article, however I suspect he may agree with the general gist of it.

[lanefu]

Yeah linuxserver.io thing was started by one of the guys on self-hosted podcast.  No real commercial motives.

 

Id probably still start with docker.   I personally look forward to making the switch to podman because there's no docker daemon, but my ability to live in a world of edge cases causes me to save that battle for another time.

 

Another thing i look for in for in self-hosted stuff are programs written in go-lang.  Often there arm binaries ready to go, and if not typically its not too challenging to compile or cross compile.