Go with used & refurb business PCs right out of the gate instead of fucking around with SBCs like the Pi.
Go with “1-liter” aka Ultra Small Form Factor right away instead of starting with SFF. (I don’t have a permanent residence at the moment so this makes sense for me)
Ah, but now you have a stack of PiS to screw around with, separate from all the stuff you actually use.
I would documented everything as I go.
I am a hobbyist running a proxmox server with a docker host for media server, a plex host, a nas host, and home assistant host.
I feel if It were to break It would take me a long time to rebuild.
Ansible everything and automate as you go. It is slower, but if it’s not your first time setting something up it’s not too bad. Right now I literally couldn’t care less if the SD on one of my raspberry pi’s dies. Or my monitoring backend needs to be reinstalled.
IMO ansible is over kill for my homelab. All of my docker containers live on two servers. One remote and one at home. Both are built with docker compose and are backed up along with their data weekly to both servers and third party cloud backup. In the event one of them fails I have two copies of the data and could have everything back up and running in under 30 minutes.
I also don’t like that Ansible is owned by RedHat. They’ve shown recently they have zero care for their users.
if by “their users” you mean people who use rebuilds of RHEL ig
I didnlt realize that about ansible. I’ve always thought it was overkill for me as well, but I figured i’d learn it eventually. Not anymore lol.
Buy an actual NAS instead of a rats nest of USB hub and drives. But now it works so I’m too lazy and cheap to migrate it off.
I haven’t built one myself, but you could look into TrueNas.
Instead of a 4-bay NAS, I would have gone with a 6-bay.
You only realize just how expensive it is to expand on your space when you have to REPLACE HDDs rather than simply adding more.
Yes, but you’ll be wishing you had 8 bays when you fill the 6 :) At some point, you have to replace disks to really increase space, don’t make your RAID volumes consist of more disks than you can reasonably afford to replace at one time. Second lesson, if you have spare drive bays, use them as part of your upgrade strategy, not as additional storage. Started this last iteration with 6x3tb drives in a raidz2 vdev, opted to add another 6x3tb vdev instead of biting the bullet and upgrading. To add more storage I need to replace 6 drives. Instead I built a second NAS to backup the primary and am pulling all 12 disks and dropping back to 6. If/when I increase storage, I’ll drop 6 new ones in and MOVE the data instead of adding capacity.
This. And build my own instead of going with synology.
I’ve been pretty happy with my Synology NAS. Literally trouble-free, worry-free, and “just works”. My only real complaint is them getting rid of features in the Photos app, which is why I’m still on their old OS.
But I’d probably build a second NAS on the cheap, just to see how it compares :)
What OS would you go with if you had to build one?
I’m happy with synology too for the most part. But I like a bit more flexibility I’d probably build one and use truenas or unraid.
I ended up getting a Raspberry Pi 4 and the Argon Eon case. It all goes through one USB 3 channel however, and for some reason I am stuck at 10MB/s transfer speeds even though USB 3 standards support much more.
I would like a SBC which supports SATA. I suppose there is a the Raspberry Pi CM4, although there’s no cases for it to support multiple drives
I’ve got the argon one v2 with a m2 drive. Works well haven’t tested speeds. Not used as a nas though.
For me:
- Document things (configs, ports, etc) as I go
- Uniform folder layout for everything (my first couple of servers were a bit wild-westy)
- Choosing and utilizing some reasonable method of assigning ports to things. I do not even want to explain what I need to do when I forget what port something in this setup is using.
I’d plan out what machines do what according to their drive sizes rather than finding out the hard way that one of them only has a few GB spare that I used as a mail server. Certainly document what I have going, if my machine Francesco explodes one day it’ll take months to remember what was actually running on it.
I’d also not risk years of data on a single SSD drive that just stopped functioning for my “NAS” (its not really a true NAS just a shitty drive with a terabyte) and have a better backup plan
I wouldn’t change anything, I like fixing things as I go. Doing things right the first time is only nice when I know exactly what I’m doing!
That being said, in my current enviroment, I made a mistake when I discovered docker compose. I saw how wonderfully simply it made deployment and helped with version control and decided to dump every single service into one singular docker-compose.yaml. I would separate services next time into at least their relevant categories for ease of making changes later.
Better yet I would automate deployment with Ansible… But that’s my next step in learning and I can fix both mistakes while I go next time!
I do the same. I use caddy reverse proxy, and find it useful to use the container name for url, and no ports exposed
What is the benefit for making changes with separate files?
If you have relevant containers (e.g. the *arr stack) then you can bring all of them up with a single docker compose command (or pull fresh versions etc.). If everything is in a single file then you have to manually pull/start/stop each container or else you have to do it to everything at once.
This. In addition, I’ve read that it’s best practice to make adding and removing services less of a pain.
You’re not messing with stacks that benefit from extended uptime just to mess around with a few new projects. Considering my wife uses networks that the homelab influences, it would be a smarter choice for me long term to change things up.
I’d use Terraform and Ansible from the start. I’m slowly migrating my current setup to these tools, but that’s obviously harder than starting from scratch. At least I did document everything in some way. That documentation plus state on the server is definitely enough to do this transition.
I’d put my storage in a proper nas machine rather than having 25tb strewn across 4 boxes
More ram 🐏
That was my mistake when I tried to host literally everything on an Orange PI which has only 2 GB of RAM
I have ended up with 6x 2TB disks, so if I was starting again I’d go 2x10TB and use an IT mode HBA and software RAID 1. I’d also replace my 2x Netgear Switches and 1x basic smart TP-Link switch and go full TP-Link Omada for switching with POE ports on 2 of them - I have an Omada WAP and it’s very good. Otherwise I’m pretty happy.
Not go as HAM on commercial server hardware. iLO is really nice for management though…
I already have to do it every now and then, because I insisted on buying bare metal servers (at scale way) rather than VMs. These things die very abruptly, and I learnt the hard way how important are backups and config management systems.
If I had to redo EVERYTHING, I would use terraform to provision servers, and go with a “backup, automate and deploy” approach. Documentation would be a plus, but with the config management I feel like I don’t need it anymore.
Also I’d encrypt all disks.
Also I’d encrypt all disks.
What’s the point on a rented VPS? The provider can just dump the decryption key from RAM.
bare metal servers (at scale way) rather than VMs. These things die very abruptly
Had this happen to me with two Dedibox (scaleway) servers over a few months (I had backups, no big deal but annoying). wtf do they do with their machines to burn through them at this rate??
I don’t know if they can “just” dump the key from RAM on a bare metal server. Nevertheless, it covers my ass when they retire the server after I used it.
And yeah I’ve had quite a few servers die on me (usually the hard drive). At this point I’m wondering if it isn’t scheduled obsolescence to force you into buying their new hardware every now and then. Regardless, I’m slowly moving off scaleway as their support is now mediocre in these cases, and their cheapest servers don’t support console access anymore, which means you’re bound to using their distro.
I’d encrypt all disks. Nevertheless, it covers my ass when they retire the server after I used it.
Good point. How do you unlock the disk at boot time? dropbear-initramfs and enter the passphrase manually every time it boots? Unencrypted
/boot/and store the decryption key in plaintext there?I run openbsd on all my servers so I would be entering the passphrase manually at boot time. Saving the key on unencrypted
/bootis basically locking your door and leaving the key on it :)
I would use terraform to provision servers, and go with a “backup, automate and deploy” approach. Documentation would be a plus
Yea. This is what I do. Other than my Synology, I use Terraform to provision everything locally. And all my pi holes are controlled by ansible.
Also everything is documented in trillium.
Whole server regularly gets backed up multiple times, one is encrypted and the other via syncthing to my local desktop.
Terraform is the only missing brick in my case, but that’s also because I still rent real hardware :) I’m not fond of my backup system tho, it works, but it’s not included in the automated configuration of each service, which is not ideal IMO.
I would have gone with an Intel CPU to make use of iGPU for transcoding and probably larger hard drives.
I also would have written down my MariaDB admin password… Whoops
I would have taken a deep dive into docker and containerised pretty much everything.
Same for me. I’ve known about Docker for many years now but never understood why I would want to use it when I can just as easily install things directly and just never touch them. Then I ran into dependency problems where two pieces of software required different versions of the same library. Docker just made this problem completely trivial.
Converting my environment to be mostly containerized was a bit of a slow process that taught me a lot, but now I can try out new applications and configurations at such an accelerated rate it’s crazy. Once I got the hang of Docker (and Ansible) it became so easy to try new things, tear them down and try again. Moving services around, backing up or restoring data is way easier.
I can’t overstate how impactful containerization has been to my self hosting workflow.
Same, but I’ve never once touched Docker and am doing everything old skool on top of Proxmox. Others may or may not like this approach, but it has many of the benefits in terms of productivity (ease of experimentation, migration, upgrade etc)
I’m mostly docker. I want to selfhost Lemmy but there’s no one-click Docker Compsoe / Portainer installer yet (for Swag / Nginx proxy manager) so I won’t until it’s ready
Same here. Now I’m half docker and half random other stuff.
