What’s up, what’s down and what are you not sure about?
Let us know what you set up lately, what kind of problems you currently think about or are running into, what new device you added to your homelab or what interesting service or article you found.
I finally finished my first iteration of my Minilab including a very smooth migration from the old server yesterday so I can go to the service side of things again. I plan to get some kind of selfhosters VPN for external access to stuff that’s not exposed to the internet, I’ll have to investigate which one.
I recently setup a full matrix server. What I am currently worried about is my server. I am currently shopping for a used dual Xeon server. I am hosting close to 40 docker containers on 2 1 liter PCs with very low specs. I would love to bring it all in house to a single server with a separate NAD which I do have currently holding 60 terabytes of storage space.
I’m currently trying to figure out why my email server got blocked by Proofpoint and they refuse to talk to me. Really about ready to give up on email after self-hosting it for a decade with few problems.
There is still the relay through the cloud route (SES, but also at least Scaleway)
Part of me thinks if I have to pay for a relay service, I should just pay for hosted email. But I’ve definitely been considering it!
Check RBLs a lot of times services just use one of those, and they can be flaky. Usually, you can fill out a form and get reinstated.
I’m not on any of those blacklists, luckily. I guess Proofpoint doesn’t publish theirs. At least iCloud and Gmail both use them. I saw one hint that they may require mail servers to literally have the word “mail” as the subdomain, so I’m working up the courage to mess around with my perfectly working DNS.
Oh that sucks! One would think that after that long, it’d be somewhat established.
RIGHT?!
Up: My unraid server with media library, emby and my kids Down: the fiber internet line into the house that the contractor working on our siding snapped. No one is upset so the system is working.
I have a question on top of my matrix setup. Has any one integrated VoIP? I am trying to bring all communication in house.
My biggest shortcoming at the moment is my NAS is also my gaming PC. It’s pretty inefficient to have that on all the time. But I haven’t had the time to build a dedicated NAS.
Yeah I had your idea back when I wanted a nas. I didn’t have the time and just bought a synology knowing it wasn’t the best option and was aware of the possibility of enshitification. Now that they’ve enshitified, I can’t really recommend them any longer. So far it’s been good but I’m still looking for options that are quick and easy to set up. Or maybe I’ll grit my teeth and start building one from scratch.
I’m perfectly happy to build my own NAS with NixOS and ZFS on it. I think it’s mostly a matter of getting the right hardware.
I’m putting together a pretty simple one this week. Got a used HP Elitedesk G4 SSF for around $150, already have 2 8TB external drives lying around that are easy enough to shuck and slap into it. Should be pretty easy to just slap TrueNAS Scale onto it, set up a mirror with the 2 drives, and be good to go for a while.
I’ll definitely need more space down the road and this thing can’t fit more than 2 drives without some modifications (3 is doable, but 4 will take some 3D printed parts which I believe someone’s still working on fine-tuning). But it’s good enough for me for now, still got 2.5TB I’m not using.
If I thought about storage a bit more before starting this project, I probably would’ve just gotten the same SSF but with some slightly better specs to use as the entire server, rather than running 2 different machines, but oh well.
Edit: Slight change of plans, got a 12tb drive free through a program at work, so gonna go with UnRAID instead. The license fee is a bit disappointing but it seems to suit my needs better, and being able to mix and match drives of any size at will is pretty nice
Having electric stability issues this week in Bangkok - several 2-3 hour outages, which are too long for a UPS to cover the gap. I have several mid range but older PCs running docker, virtualbox, etc for various things including a postfix server for the family email, immich, QBittorrent, pihole, paperless, huly, postiz, a Minecraft bedrock server, a flightradar24 ads-b collector, and a variety of other homegrown projects.
Thinking about getting some or most of this over to a service like hetzner, perhaps even splurging on a baremetal dedicated system.
Recently I’ve been reading about/trying to learn qemu and proxmox, but don’t understand them yet. Is that where it’s at for managing a bunch of your own VMs? Or kubernetes/k8s?
I’ve been a little out of the loop for a few years and of course coming back up to speed IT wise judge take weeks. Looking for recommendations on offloading my home stuff to a cloud that I control.
proxmox
You will enjoy Proxmox. When you get it all jammy, check out the Proxmox Helper Scripts: https://community-scripts.github.io/ProxmoxVE/
Hey that’s awesome! thank you for the share. Planning to install proxmox this weekend and give it a try.
Thinking about getting some or most of this over to a service like hetzner, perhaps even splurging on a baremetal dedicated system.
If I may, I find LUXVPS to be quite capable and responsive hosts.
Black Luxury Deal #1
4 vCores (Xeon Gold 6150) 26 GB DDR4 RAM 150 GB Raid 1 NVMe 1 Gbit internet speed | 40 TB Traffic 1x IPv4 1x /64 IPv6 3.2Tbit Premium DDoS Protection 24/7 Ticket Support 4 Backups For ONLY 10€/Mo (recurring)I’ve never used Hetzner, and I don’t know what you are hosting, but I’m sold on LuxVPS. I also use Contabo, and Ethernet Services. The latter would indeed be bare-bare-metal as there are no frills. However, for a test server and for $35 a year, it works.
Proxmox runs Qemu under the hood. It’s the current favorite for VM management.
I wouldn’t bother with k8s unless you’re deploying services in high availability, or groups of related containers.
I’ve used a RV/Marine deep cycle battery attached to a UPS before, that would certainly give you enough for 2-3 hours on most setups.
K8S is a whole different approach and I find it to be a lot more complex, but you would not need virtual machines. If all your applications are running in containers anyways, you could consider it. Finding a good solution for persistent storage is probably the most important design decision.
I finally bought a tiny PC to replace my aging APU border router/firewall (OpenBSD), so I’m trying to wrap my head around building a router currently inside the network that it will be protecting.
I have Debian installed as hypervisor, Incus, and sticking with OpenBSD for the firewall.
pfmakes too much sense to me too switch to firewalld. I’ll also move the network-related containers off my main lab host once this is up and running.I’m still trying to get a good backup strategy. I am currently using Duplicati but I cannot get the before script execution to work. I will eventually look at Kopia.
What kind of hardware are you using for a mini lab? I want to switch from a raspberry pi 5 to a small form factor Intel based system so I can run Proxmox. I was looking at the Lenovo m920q or an Optiplex 79xx series machine.
Do you have any recommendations for backups or the hardware switch I mentioned?
If you do make a switch to Proxmox, then Proxmox Backup Server is where it’s at for backups. Its de-duplication feature is incredible. I backup all my Proxmox VMs/LXCs with it, as well as my non-Proxmox hosts (laptop, etc.), with
proxmox-backup-client.Personally, I’m using a few of those tiny Beelink PCs (a couple Mini S12 and an EQ12) with the N100 processor, as well as a couple larger rackmount PCs I built for situations where I needed to add an HBA or some other PCI-Ex device. I do recommend something like a Beelink before building, though - they run Proxmox fine, they’re inexpensive, efficient, quiet, and each one can run a handful of VMs.
Yeah, I heard about Proxmox backup and that sounds really nice. Love the idea of being able to take a snapshot before any major changes to a VM and then if it goes south restore from snapshot very quickly.
I don’t know Duplicati or Kopia, im mostly just using VM snapshots as backups. I store them in an NFS Share on my NAS.
I just posted my Minilab, check my history - I’m also using tiny Lenovos. m920q should be able to do anything you want it!
I use cron schedules to run scripts that backup my important stuff to s dedicated backup drive, then copies the backups to a different external drive, then upload the backups to a dedicated backup cloud storage account. Then it deletes any backups older than a month.
Have a look at Backrest for Restic. It works great with pre/post scripting and supports healthchecks for monitoring status and stats.
Also is a nice easy to use WebUI which is great for servers.
I’ll look at this again. I had it before and did not stick with it though I don’t remeber why now.
I need to get a new VPN setup. Been using OpenVPN through OPNsense for years but I’m fed up with the abysmal performance of the OpenVPN client on iOS. Open to suggestions but it has to be fully self hosted.
wg-easy is what you want
I have running OPNsense as well and was looking for OPEN VPN on it as well, but I’m not decided yet. I wonder if Android clients are any better.
I’m running OpenVPN on pfsense and am using the android app.
I’ve got a stable 150mbit/s, depending on carrier coverage.
Wireguard is where it’s at.
Good on iOS too, albeit a bit battery hungry if you route 0.0.0.0/0 and ::/0
I dunno if there’s an iOS equivalent but on my Android phone I use the WG Auto Connect app so it’s only active when not on my home wifi.
The iOS app has this, based on SSIDs
Latest thing is my server was hard locking up randomly every couple days. Finally thought to check IPMI and it was triggering a correctable ECC error on a specific stick of RAM.
I figured maybe the first couple errors were correctable by the ECC RAM but then they just got worse and caused the lock up.
Pulled the 2 sticks in that pair and so far so good. I’ll survive just fine with the remaining 192GB of RAM lol.
Also switched from my old Dell box with Opnsense to a Linksys MX4300 running OpenWRT, saves me about 20W and its fun to try something different.
Was able to put calibre web on nixos. Still trying to build a package that’s not available (piped), but boys is it hard to package java stuff for nixos…
For some reason Grafana started to sync roles with my IdP (google) and now my own user keeps getting a read only role, so I decided to take this opportunity to finally move away from google and start hosting keycloak instead.
It was a busy week so I could not get the time to finish it yet.
You may also have a migration path by hosting keycloak and add Google as an Identity Provider. Gives you much more flexibility and control this way
Thanks for the tip! I didn’t know that setup was possible.
Currently rewriting my homelab into terraform and adding some redundancies using cloud environments, in case of power outages or network issues.
Trying to get navidrome routed through Traefik.
I think it’s rejecting it as an untrusted proxy because forwarding the ports locally works.
Also working on getting Traefik up and running on a TuringPi cluster to eventually move my workloads over to it.
I have navidrome running in k8s behind Træfik. Do you want to take a look at my environment variables for navidrome? I haven’t configured anything on Træfik
Sure thank you 🙂
Sorry, I can’t help you. I configured only the TZ and the schedule env vars…
spec: containers: env: - name: TZ value: Europe/Rome - name: ND_SCANSCHEDULE value: 0 * * * * image: deluan/navidrome:latest imagePullPolicy: IfNotPresent name: navidrome ports: - containerPort: 4533 hostIP: null hostPort: null name: http protocol: TCP volumeMounts: - mountPath: /data mountPropagation: null name: config-volume - mountPath: /music name: music readOnly: true
I finally finished setting up my Nebula network! An overlay network, as opposed to a true VPN, but excellent for flexibility and remote access. For anyone wanting maximum control over your network with excellent performance, I highly recommend it.
Check out apalrd’s blog for a great tutorial if you’re interested.
