Hey guys. I’ve been considering maybe moving to another OS for my home lab. Do you have have any suggestions? Especially former Unraid users? Mostly just for arrs though I would like to run reverse proxy/file hosting as well. Proxmox seems pretty trendy can I use it for arrs as well as backups?
Rant/extra info:
Tap for spoiler
I’ve been using Unraid for a couple years now, even paid for basic registration. I’ve largely used it to run all my arrs in docker, pihole and had a HASSIO VM running.
I recently tried setting up nextcloud, during the set up (which like nearly everything, I followed a video guide for) I ran into a novel error. So I deleted the nextcloud docker and got it from the official repo instead. Now my nextcloud share is gone and I can’t create new shares??
Stuff like this happened when I set up guac. Weird errors, plenty of which have little documentation or explanation. Plenty of which I need to ssh in or use Linux commands to fix. Which lead me to, “I’m having to learn this stuff anyway, why not spin up a Linux server and learn properly”.
Should I just rebuild/give Unraid a bit more time, it is young OS wise right?
OS: NixOS (high learning curve but its been worth it). Nix (the config language) is a functional programming language, so it can be difficult to grok. Documentation is shit as its evolved while maintaining backwards compatibility. If you use the new stuff (Nix Flakes) you have to figure what’s old and likely not applicable (channels or w/e).
BYOD: Just using LVM. All volumes are mirrored across several drives of different sizes. Some HDD volumes have an SSD cache layer on top (e.g., monero node). Some are just on an SSD (e.g., main system). No drive failures yet so can’t speak to how complex restoring is. All managed through NixOS with https://github.com/nix-community/disko.
I run stuff on a mix of OCI containers (podman or docker, default is podman which is what I use) and native NixOS containers which use systemd-nspawn.
The OS itself I don’t back up outside of mirroring. I run an immutable OS (every reboot is like a fresh install). I can redeploy from git so no need to backup. I have some persistent BTRFS volumes mounted where logs, caches, and state go. Don’t backup, but I swap the volume every boot and keep the last 30 days of volumes or a min of at least 10 for debugging.
I just use rclone for backups with some bash scripts. Devices back up to home lab which backs up to cloud (encrypted with my keys) all using rclone (RoundSync for phone).
Runs Arrs, Jellyfin, Monero node, Tor entry node, wireguard VPN (to get into network from remote), I2C, Mullvad VPN (default), Proton VPN (torrents with port forwarding use this), DNS (forced over VPN using DoT), PiHole in front of that, three of my WiFi vlans route through either Mulvad, I2C, or Tor. I’ll use TailsOS for anything sensitive. WiFi is just to get to I2C or Onion sites where I’m not worried about my device possibly leaking identity.
Its pretty low level. Everything is configured in NixOS. No GUIs. If its not configured in nix its wiped next reboot since the OS is immutable. All tracked in git including secrets using SOPS. Every device has its own master key setup on first install. I have a personal master key should I need to reinstall which is tracked outside of git in a password manager.
Took a solid month to get the initial setup done while learning NixOS. I had a very specific setup of LVM > LUKS encryption /w Secure Boot and Hardware Key > BTRFS. Overkill on security but I geek out on that stuff. Been stable but still tinkering with it a year later.
I have seen Nix come up quite a bit and have been tempted to try it. I’ve rolled with Arch before so I was considering going back to it but maybe something new be go.
Something like this has always interested me. I remember reading about doing similar with Windows. Not so much it being immutable so much as having a decent starting image that you load on any device you want with all your programs ready to go.
Do you have a guide or ten you used for all this perchance? Unraid has stuff like trashguides and space invader one. Especially the DNS part onwards? If not it’s cool I have Mullvad set up and Pihole with my current setup so I’ll be able to work it out. This is all very compelling for me to try out (I should really have learned about wireguard by now). Thanks a lot for such an interesting and informative write up!
Nixos’ weakness is definitely it’s documentation. There’s often configuration snippets you can copy and paste, though. If you go with NixOS, make sure to come back with questions, the community is very helpful.
Best resource I’ve found is searching GitHub.
My setup closely follows https://github.com/Misterio77/nix-config.
For servarr I just translated someone else’s docker compose setup to nix. There are some ready made nix ones you can look at like https://github.com/rasmus-kirk/nixarr/tree/main/nixarr.
The complex networking I just picked up over time once I knew my way around a little bit.
GitHub is your best resource.
lang:nix search terms.For the networking I found some repos with Nix and Gluetun (OCI containers). I don’t see them in my bookmarks, so it was probably a day project when I set up and didn’t keep the references.
That part is still in docker / podman. So any docker network guide just needs to be translated to nix.
You might want to read the recent blog post (linked at top) and discussion on Hacker News first.
I wouldn’t run NixOS in a container. With native nix containers I’m pretty sure they share the store. For docker I’d use images built with nix (doesn’t run nix itself) or pull from docker hub.