I need
It’s just fun to play with, there is no “need”.
Yeah, I enjoyed my time with k3s setup at home as well, but right now I don’t really want nor need that 😄
I need a kubernetes cluster with high availability, load balancing and horizontal pod autoscaling, because that is something I want to learn. I don’t care that it’s just for wife’s home-made dog collars webshop.
This is the way
You can do it on a handful of Raspberry Pis rather than one, then.
Imagine, if you will, a Beowulf cluster of Raspberry Pis!
A man of culture, I see!
Switched from a raspberry pi 3 to a second hand x86 thin client (lenovo thinkcentre m920q) because raspberry pi 4 were not available at the time. Made me learn proxmox and a bunch of other cool stuff my raspi couldn’t handle.
I’m rooting for ARM / RISC-V to become more popular in desktop computing / servers though.
Waiting for proxmox-arm becoming a thing (I know there’s some community versions trying it but I’m not sure how reliable they are)
Is there RISC-V hardware already? I thought the specification was still under development.
Very much so, not quite ready for prime time maybe, but you can play with it, StarFive is quite well-known for their chips in this space for example
Ok great! Time to get me some presents.
There are some Raspi competitors offering SBCs with RISC-V chips, there is even a RISC-V Mainboard for the framework laptops, but the last time I checked they sadly didn’t reach the performance levels of comparable ARM chips.
Psh if I’m writing at that level I don’t need the performance yet. Thanks!
This struggle usually takes place over a weekend.
This guy selfhosts
i think the best choice is a cheap used pc or laptop, or server. Reduces electric waste. I also host my own server on a 19 year old Dell Insprion 1300
Think centre tiny here
Low consumption, two ddr4 slots, one 2.5" slot and one nvme slot! Lots of outside slots.
Costed less used than a new pi too. They have gotten too expensive IMO.
Pi has gotten crazy expensive.
Yes, but also no. Older hardware is less power efficient, which is a cost in its own right, but also decreases backup runtime during power failure, and generates more noise and heat. It also lacks modern accelerated computing, like ai cores or hardware video encoders or decoders, if you are running those appd. Not to mention lack of nvme support, or a good NIC.
For me a good compromise is to recycle hardware upgrades every 4-5 years. A 19 year old computer? I would not bother.
Bro, I am just hosting a WordPress backup, an RSS reader, and a few Python scripts
Wordpress needs the latest GPU these days, didn’t you know? /s
I have a Lenovo M710q with a i3 7100T that uses 3W at idle. I’m not mining bitcoin, server is idle 23h a day if not more.
my 19 year old laptop runs the web server just fine, and only needs 450 mb ram even with many security modules. it produces minimal noise
Reduces electric waste
A lot of older equipment actually wastes more electricity.
But it will cut down on electronic waste.
not always. especially laptops
not always a lot of
Yeah, I expect there is a compromise somewhere but 10+ year old stuff which I can easily find on eBay probably just isn’t worth it for anything that is going to be on most/all of the time. Better to go for more modern lower end stuff to use less power. 250w will cost you like £50 a month to run
Not necessarily.
A i5-6500 has a TDP of 65W while a i5-13600K has a TDP of 150W.
If you get something modern that has the performance of a i5-6500 it will be a little bit more efficient. The key is that more performance uses more power.
not necessarily
a lot of
TDP ≠ power draw. TDP is literally the Thermal Design Power aka what is the amount of thermal load a system designer should account for. Yes it can give you a rough and dirty idea of maximum power draw, but real world power draw can be entirely different because that depends on load.
For example, if your i5-6500 runs at 50-70% load while the newer processor only runs at 20-30% load due to IPC and instruction improvements the newer processor might very well use less power over the course of month than the older one despite the newer one being capable of drawing more
You’re also comparing a 4c4t part to one with 14c/20t not to mention comparing a mass market part to a gaming specific part. The 6600k (which is targeting the same market segment as the 13600k) has a 91w TDP. Go compare your 6500 to the i5-13500 except again it’s still comparing apples to oranges when you just look at raw specs and TDP ≠ real world power consumption
Yeah what I’ve always done is use the previous gaming/workstation PC as a server.
I just finished moving my basic stuff over to newer old hardware that’s only 6-7 years old, to have lots of room to grow and add to it. It’s a 9700k (8c/8t) with 32GB of ram and even a GTX 1080 for the occasional video transcode. It’s obviously overkill right now, but I plan to make it last a very long time.
They take up so much space though.
This is why rack mounts were made. Hell, I’ve seen a lot of custom builds where people have mapped out the server on their wall and it takes up no floor space. Something like this: https://i.xno.dev/kG9Wx.jpg
A rack takes up as much space as a fridge though, and mounting things to the wall is risky. You better make sure you really got it into the stud in the wall. Also, don’t do that if you live in an earthquake zone.
Only full size racks. You don’t need to buy a full size rack. You can get very small racks these days that are smaller than a little chest cooler. And why are you under the impression that you have to mount it on the wall?
You shared a picture of a computer mounted on a wall.
Correct. If space is such a big problem for you that it’s unconscionable to use a 4U mini rack (which again, like what the fuck), then mounting hardware on the wall is a completely valid option. It’ll take up zero floor space.
That is very cool
A mini PC is a good middle ground. Mostly for the video transcode and machine learning power.
Yeah, a mini PC… or if you already have one, why not 5 mini PCs?
I think any mini-pc/old laptop is better, and probably cheaper than a raspberry pi nowadays
I would have disagreed with you when Pis were like $50 and chaining 3 Pis together with a hard drive was a fun project to do self hosting.
Now to get to the beefiest raspberry pi, it’s $120. And in the range, yeah, for price and reliability, use a mini-pc/laptop.
I think the pi zero might have a place, don’t have one but at least it is actually cheap. Used an old laptop before, now running stuff on my PC which I feel was a bit of a mistake in a way but it started as one process and more appeared over time.
At some point would like to move it into a VM tbh, then I could copy the VM to a mini PC at a later date. Or easily copy it when reinstalling the OS on my main PC. VM for convenience and separating it from my general PC usage.
Pi zero 2 runs my home automation smoothly and few of it’s sisters are in a desk drawer in case of equipment failure. It has quite enough oomph to run Home Assistant with zigbee hat.
Yeah honestly i can’t get anything done with a raspberry. Maybe i host too many services ?
Nah, they are underpowered for their current price. When you could get your hand on a raspberry pi 3 for 35€ 10 years ago it was a bargain. But now it’s not the case.
Honestly Raspberry Pis are pretty underpowered as hosts for more than a handful of super basic services, and given they consume 20-30w at minimum you’re easily getting into used office desktop territory where you can get a ton more performance right out of the gate.
The real value in the raspberry pi is in the GPIO and the cohesive ecosystem of accessories to plug into said GPIO. You can do so many cool automations and controls using just an RPi (especially if combined with something that can’t be accomplished more cheaply and easily with an ESP32) but as a server host they’re pretty crap in comparison to a decade old business PC off of eBay
I’ve got 3x RPi 4/5 with external drives and other USB running ~10 containers at 4-6W each. How did you make them draw 20W?
I’ve discovered that there are a lot of medium-tier software engineers who immediately will go straight to horizontal scaling (i.e: just throw hardware at it), and I’ve seen instances where very highly skilled engineers just write their code better, set things up on a bare metal server, cache things, etc. and manage with just a single badass server
Even just the choice of programming language makes a big difference. Running a JVM language or NodeJS, Python, Ruby etc., you can be bottlenecked by a Pi. Meanwhile, Rust or C/C++ will use barely a fraction of those resources.
Yup, a pi is enough for me.
Well… 5 Pis and an ancient NUC running proxmox are enough for me. And a DS920+… and an old laptop running docker are enough for me.
See, I don’t pay for the electric bill to keep my collection of old enterprise equipment running because I need the performance. I keep them running because I have no resistance to the power of blinkenlights.
I spend all day at work exploring the inside of the k8s sausage factory so I’m inured to the horrors and can fix basically anything that breaks. The way k8s handles ingress and service discovery makes it absolutely worth it to me. The fact that I can create an
HTTPProxyand have external-dns automagically expose it via DNS is really nice. I never have to worry about port conflicts, and I can upgrade my shit whenever with no (or minimal) downtime, which is nice for smart home stuff. Most of what I run tends to be singleton statefulsets or single-leader deployments managed with leases, and I only do horizontal for minimal HA, not at all for perf. If something gives me more trouble running in HA than it does in singleton mode then it’s being run as a singleton.k8s is a complex system with priorities that diverge from what is ideal for usage at home, but it can be really nice. There are certain things that just get their own VM (Home Assistant is a big one) because they don’t containerize/k8serize well though.
Yup, same here. being able to skip all the networking and DNS hassle and have it automated for you is so nice.
Having databases fully managed with cnpg is AMAZING
I just have renovate set to auto update my argocd, so everything just runs itself with zero issues. Only the occasional stateful container that has breaking changes in a minor version.
If something OOMs or crashes, it all just self heals, I never need to worry about it. I don’t have any HPAs (nor cluster scaling obv), though I do have some HA stuff set up just to reduce restart times and help keep the databases happy.
The main issue with Kubernetes is that a lot of self-hosted software makes bad design decisions that actively make kubernetes harder, eg sqlite instead of postgres and secrets stored in large config files. The other big issue is that documentation only supports docker compose and not kubernetes 90% of the time so you have to know how to write yaml and read documentation.
Moving my hass from a statefulset to kubevirt sounds tempting. Did you have better reliability/ergonomics? I have been looking into moving my Hass automation to NodeRed, so that I can GitOps it all, since NodeRed supports git syncing.
Ha ha
Under-complicated -> over-complicated -> under-complicated.
There’s a ‘just right’ that I think you skipped through.
So close. Started on raspberry pi. Went for a cluster with dpckrt swarm. Finished with a nas and a 10years old game computer as a mediacenter. (That the electricity bill whoch made me stop the cluster)
The only problem I’ve had with Raspberry Pi is that some apps want to write a lot of stuff to “disk”, and the default “disk” on a Pi is a MicroSD card which dies if you keep writing things to it. Sure, you can always plug something into a USB slot, but that adds a bit of friction to the whole process.
Oh, also, I wish it were easy to power a whole bunch of Pi units. Each one needing its own wall wart is a bit annoying, and I’ve had iffy results using weaker, less steady power supplies with multiple ports intended for things like phones.
Most SD cards aren’t really suitable for the kind of workload an operating system generates (that being mostly random i/o). Make sure to get a reputable A2 (application class 2) rated card, they aren’t that expensive but perform way better.
Raspberry Pi themselves launched a card recently, I haven’t tried that one but it’s probably a good choice too.
I think the Raspberry Pi Linux releases mount things onto a ram drive, so the typical IO doesn’t touch the SD Card. But, if you run another OS (which sometimes is the easiest way to get other software running) it tends to just treat the SD Card like an HDD/SSD.
That’s definitely not true, Raspberry Pi OS works and acts like a normal Debian installation per default - with root mounted rw and all.
Other than that, there isn’t much “treating like an HDD/SSD” going on, it just writes to flash when an application requests it does. If the underlying storage is an eeprom, an sdcard nvme storage doesn’t really change anything here.
rPi with k3s installed
None of the power, all of the hassle 💪💪
k3s
kbnts?
That’s not a typo: https://k3s.io/
It’s basically a Kubernetes cluster, which you can run locally on your PC. Really useful for playing around with Kubernetes before you move to a ‘proper’ environment.
That’s neat, and also gives me another reason to hate numeronyms 😭
Only the good ones for me
- Interoperability (i14y)
- Accessibility (a11y)
- Localization (l10n)
- Internationalization (i18n)
- Observability (o11n)
- Kubernetes (K8s)
- Andreessen Horowitz (a16z)
How does that compare to microk8s? I have been using that for a while and like the plugins.
