Actual public services run there, yeah. In case if any is compromised they can only access limited internal resources, and they’d have to fully compromise the cluster to get the secrets to access those in the first place.

I really like garage. I remember when minio was straightforward and easy to work with. Garage is that thing now. I use it because it’s just co much easier to handle file serving where you have s3-compatible uploads even when you don’t do any real clustering.

I’ve dealt with exactly the same dilemma in my homelab. I used to have 3 clusters, because you’d always want to have an “infra” cluster which others can talk to (for monitoring, logs, docker registry, etc. workloads). In the end, I decided it’s not worth it.

I separated on the public/private boundary and moved everything publicly facing to a separate cluster. It can only talk to my primary cluster via specific endpoints (via tailscale ingress), and I no longer do a multi-cluster mesh (I used to have istio for that, then cilium). This way, the public cluster doesn’t have to be too large capacity-wise, e.g. all the S3 api needs are served by garage from the private cluster, but the public cluster will reverse-proxy into it for specific needs.

I would not recommend unifi for a mature solution. It sure works nice as a glass panel, but it will get limiting if you will have a desire to hack around your network. Their APs are solid, though, it’s just the USG/Dream machine that I wouldn’t recommend.

Mikrotik software is very capable and hackable and you can run it in a vm if you feel like bringing your own hardware.

Apparently traefik might be better if you run docker compose and such, as it does auto-discovery, which reduces the amount of manual configuration required.

and swap Prometheus for VictoriaMertics, or your homelab ram usage becomes Prometheus ram usage.

I’ll second conduit. You can tune up its caching, reducing the ram usage significantly. It does become a bit painful to sync the mobile clients, but at least it’s not gigabytes of ram wasted.

In the context of my comments here, any mention of “S3” means “S3-compatible” in the way that’s implemented by Garage. I hope that clarifies it for you.

Clearly I mean Garage in here when I write “S3.” It is significantly easier and faster to run hugo deploy and let it talk to Garage, then to figure out where on a remote node the nginx k8s pod has its data PV mounted and scp files into it. Yes, I could automate that. Yes, I could pin the blog’s pod to a single node. Yes, I could use a stable host path for that and use rsync, and I could skip the whole kubernetes insanity for a static html blog.

But I somewhat enjoy poking the tech and yes, using Garage makes deploys faster and it provides me a stable well-known API endpoint for both data transfers and for serving the content, with very little maintenance required to make it work.

S3 storage is simpler than running scp -r to a remote node, because you can copy files to S3 in a massively parallel way and scp is generally sequential. It’s very easy to protect the API too, as it’s just HTTP (and at it, it’s also significantly faster than WebDAV).

I remember when minio just started and it was small and easy to run. Nowadays, it’s a full-blown enterprise product, though, full of features you’ll never care about in a homelab eating on your cpu and ram.

Garage is small and easy to run. I’ve been toying with it for several months and I’m more than happy with its simple API and tiny footprint. I even run my (static html) blog off it because it’s just easier to deploy it to a S3-compatible API.

That’s exactly my point, though.

It absolutely does. Think of lemmy like of email – your mail server has all the email you received.

Looking at the resource usage of mine, a tiny cheap VPS for $4/mo would be enough, sans the image store. But it’s not a hard requirement unless you expect to have lots of local communities posting pictures.

Lemmy’s issue is that it’s non-trivial to deploy and oftentimes painful to upgrade.

Is there anything interesting at all reported in /proc/spl/kstat/zfs/dbgmsg?

I did ran out of pcie, yeah :-( the network peaks at about 26gbit/s, which is the most you can squeeze out of pcie 3.0 x4. I could move the nvmes off the pcie 4.0 x16 (I have two m2 slots on the motherboard itself), but I planned to expand the nvme storage to 4x SSDs and I’m out of the pci lanes on the other end of the fiber either way (that box has all x16 going to the gpu)

I run 3900X with a 40Gbit fiber, packed with HDDs and nvmes. The box fluctuates around 90-110W use.

Lots of files. I’d offload old projects that I worked on with synology drive so they aren’t stored locally, only remotely (but are easily accessible).

Or just slap a GPL and subsume everything within a vortex of FREEDOM, and thusly become a true FOSS dude

Yeah, no. I suppose this is sarcasm, but just in case: not every license is compatible with GPL, GPL has a few versions, and not everything is GPL-3-and-above.

Personally, I prefer Apache-2.0. It just seems more fair.

Fediverse generally runs on ActivityPub, which uses HTTP as a transport, so you’ll be good. The problem is that the clients don’t talk to fediverse, it’s more of a server-to-sever protocol; you’d look into the specific server APIs. But you’re good there, too - all the big fediverse players use RESTful HTTP for their client-facing API.

Your requirements sound a lot like Chrome Remote Desktop and it’s pretty trivial to install, which might be a handy thing for family members that aren’t tech-savvy.