This is a continuation of my other post
I now have homeassistant, immich, and authentik docker containers exposed to the open internet. Homeassistant has built in 2FA and authentik is being used as the authentication for immich which supports 2FA. I went ahead and blocked connections from every country except for my own via cloudlfare (I’m aware this does almost nothing but I feel better about it).
At the moment, if my machine became compromised, I wouldn’t know. How do I monitor these docker containers? What’s a good way to block IPs based on failed login attempts? Is there a tool that could alert me if my machine was compromised? Any recommendations?
EDIT: Oh, and if you have any recommendations for settings I should change in the cloudflare dashboard, that would be great too; there’s a ton of options in there and a lot of them are defaulted to “off”
By not making them publicly accessible. With Wireguard there’s really no reason.
Setup service to be active on a subnet, enable Wireguard to VPN into the subnet and use the services.
Yeah, I’m not gonna tell the 50 users of my plex server to set up wireguard on their devices so they can request movies and TV series on my overseer, when I can instead just use NPM to make it publically accessible with a password prompt
Your use case, and OPs, are completely different scenarios. I can’t tell if you’re being purposefully disingenuous or just flippantly stupid.
I agree with WG however I need https for a few locally hosted items like actual budget so I have that through nginx proxy manager. I was debating adding Authelia in front with some of my others (audiobook shelf, home assistant and music assistant) as sometimes I disconnect from my home network and forget to reconnect.
Why not swap from nginx-proxy-manager to Caddy2, which can handle everything? SSL and reverse_proxy?
So there is https://en.wikipedia.org/wiki/Fail2ban which helps already to some degree.
But what are you trying to prevent? You have your services in a docker container, hopefully not running as root, which already makes it difficult to break out even if through a bug someone would be able to get access to the docker container.
I mean its not like your stuff is very important for someone to break in like the pentagon, you probably just have some photos from your phone on it, some lights can be switched on and off and some temperatures read.
I’m not trying to say that you should not care about it but I’m trying to figure out what your threat model is.
I feel weird about having those apps on the internet and basically being blind to threats. I mean yeah, I’m not a target on anyone’s list and most IPs visiting the site are bots but I would still like to know what’s going on.
I don’t work in tech for a living, this is just a hobby for me so I have limited time to work on this stuff and do research. It’s very possible I fucked something up and don’t know it. I figured if I at least got an alert that said “hey, your immich server db was dumped and sent to <insert IP>”, I could at least turn it off
I would put this stuff behind VPN.
I’ve tried different approaches with fail2ban, crowdsec, VPNs, etc. What I settled on is to divide the data of my services in two categories: confidential and “I can live with it leaking”.
The ones that host confidential data is behind a VPN and has some basic monitoring on them.
The ones that are out in the public are behind a WAF from cloudflare with pretty restrictive rules.
Yes, cloudflare suck etc., but the value of stopping potential attacks before they reach your services is hard to match.
Just keep in mind: you need layers of different security measures to protect your services (such as backups, control of network traffic, monitoring and detection, and so on).
has some basic monitoring on them.
What monitoring software are you using?
I feel like the other measures you talked about (backups, condom of network traffic, etc) I’m doing ok on. Its really just the monitoring where I’m stuck. There’s so many options
There are so many monitoring tools with various degrees of complicated setup / configuration or the amount of information you get. And honestly, I’ve looked into various tools: checkmk, monit, Prometheus… And realised that I rarely look into that information anyway. Of all “fancy” tools, I liked the ease of Netdata to set up and the amount of information that you get. However, beware that their in the process to make their free / homelad offering worse. I’ve been eyeing beszel and don’t forget CLI based tools that are avaible such as atop, btop, htop or glances.
If you want to delve deeper into the rabbit hole of monitoring, I can recommend you to read this article below: https://matduggan.com/were-all-doing-metrics-wrong/
Wazuh
Active responses are like fail2ban but better
I’ll look into it, thank you
We expose about a dozen services to the open web. Haven’t bothered with something like Authentik yet, just strong passwords.
We use a solid OPNSense Firewall config with rather fine-grained permissions to allow/forbid traffic to the respective VMs, between the VMs, between VMs and the NAS, and so on.
We also have a wireguard tunnel to home for all the services that don’t need to be available on the internet publicly. That one also allows access to the management interface of the firewall.
In OPNSense, you get quite good logging capabilities, should you suspect someone is trying to gain access, you’ll be able to read it from there.
I am also considering setting up Prometheus and Grafana for all our services, which could point out some anomalies, though that would not be the main usecase.
Lastly, I also have a server at a hoster for some stuff that is not practical to host at home. The hoster provided a very rudimentary firewall, so I’m using that to only open necessary ports, and then Fail2Ban to insta-ban IPs for a week on the first offense. Have also set it up so they get banned on Cloudflare’s side, so before another malicious request ever reaches me.
Have not had any issues, ever.
Have also set it up so they get banned on Cloudflare’s side, so before another malicious request ever reaches me.
How did you end up setting that up?
Fail2ban allows you set different actions for different infringements, as well as multiple ones. So in addition to being put in a “local” jail, the offending IP also gets added to the cloudflare rules (? Is that what its called?) via their API. It’s a premade action called “cloudflare-token-multi”
that’s awesome. thanks!
Auth portal for VPN tunnell -> Authelia -> fail2ban -> VLAN with services only.
ELK stack monitors the LAN. (Including VLAN)
Keep that VLAN segmented. You’re good unless you’re a DOGE employee, then I’d recommend quite a bit more security.
I’ve seen a bunch of people recommend Authelia. Do you mind if I ask why you went with it over other software? I only went with authentik because I found a tutorial on it first
So among my services I self host, a few need to be publicly accessible for work. For those I wish to remain private, Caddy only allows private IP ranges, plus then Authelia as auth which is set to 30 days. There is then the login of each service behind Authelia as well. It’s as good as it needs to be for my needs.
If I were only self hosting private services, then as others have said, I would put all access through a VPN.
Edit: I should add that of course the private services are then only accessed via VPN to the router (part of the private IP ranges). Caddy as reverse proxy also obfuscates the subdomain names I use.
Caddy only allows private IP ranges
Do you mind telling me more about this? How does that work; a VPN?
Sure, so I use Caddy as a reverse proxy for all my subdomains, the public ones direct straight to whatever service(s) are on IP:port etc, then the private ones only allow private IP ranges of which one is my VPN subnet, therefore only allowing LAN and VPN access. I then also have a section for each of the private subdomains with Authelia authentication which is omitted here in the caddyfile example:
(allowed) { @allowed client_ip 192.168.1.0/24 192.168.10.0/24 192.168.20.0/28 } sub.domain.com { import allowed handle @allowed { reverse_proxy 192.168.80.8:8080 } handle { abort } }
Some of these you’re already doing, but writing a complete* list. *almost garuanteed not to be complete, suggestions welcome
- Have everything behind the same reverse proxy, so that you have only one endpoint to worry about. Run it through ssllabs or similar to check your config.
- On your reverse proxy, add one or more layers of authentication if possible. Many possibilities here: If one app supports client certificates, while another has limited capabilities, you could probably tie together something where IPs are whitelisted to the ither services based on that certificate auth.
- Geoblock all countries you won’t be accessing from
- crowdsec is pretty nice, this detects/blocks threats. kinda like fail2ban but on steroids.
- if you use one of those 5$/month VPSes, with a VPN tunnel to your backend services, that adds one layer of “if it’s compromised, they’re not in your house”.
lastly consider if these things need to be publically avilable at all. I’m happy with 95% of my services only being available through Tailscale (mesh VPN, paid service with good enough free tier, open source+free alternatives available), and I’ve got tailscale on all my devices
-
check
-
check
-
check
-
I saw someone else recommend crowdsec. I’ll look into it, thanks
if you use one of those 5$/month VPSes, with a VPN tunnel to your backend services, that adds one layer of “if it’s compromised, they’re not in your house”.
I’ve heard this mentioned before but I don’t really understand how this works in practice. If the VPS was compromised, couldn’t they use the VPN to then connect to my home?
-
