I know, I know, clickbaity title but in a way it did. It also brought in the situation in the first place but I’m just going to deliberately ignore that. Quick recap:
- I came home at 3pm from the city, my internet at home didnt work.
- checked multiple devices, phones worked out of wifi, I figured I need to restart the router
- I login to the router and it responds totally normal but my local network doesnt. (Its always dns, I know)
- I check the router log and see 100s of login attempts over the past couple of days.
- I panic and pull the plug, try to get into my server by installing an old monitor, works, many errors about dns
- Wife googles with her phone, seems I had https login from outside on and someone found the correct port, its disabled now
- Obviously, local network still down, I replug everything and ssh into the server which runs pihole as dns
- pihole wont start dns, whatever I do
- I use history and find I "chmod 700"ed the dns mask directory instead of putting it in a docker volume…
- I check the pihole.log, nothing
- I check the FTL log, there is the issue
- I return it to 777, everything is hunky dory again.
Now I feel very stupid but I found a very dangerous mistake by having my lan fail due to a less dangerous mistake so I’ll take this as a win.
Thanks for reading and have a good day! I hope this helps someone at some day.
Are you joking? Why would I start fresh?
Because you don’t have a way to know what’s been compromised. Take your data only and make sure to verify nothings been tampered with.
Trust me it will be better in the long run.
Yeah, I dont feel like setting up a whole cloud infrastructure on a hunch. I‘m running like 15 different services and they are all compartmentalized. It would take weeks to reset all this. So far nobody got anywhere from what I can see.
One word of advice. Document the steps you do to deploy things. If your hardware fails or you make a simple mistake, it will cost you weeks of work to recover. This is a bit extreme, but I take my time when setting things up and automate as good as possible using ansible. You don’t have to do this, but the ability to just scrap things and redeploy gives great peace of mind.
And right now you are reluctant to do this because it’s gonna cost you too much time. This should not be the case. I mean, just imagine things going wrong in a year or two and you can’t remember most things you know now. Document your setup and write a few scripts. It’s a good start.
I get your point. Ansible is quite interesting too. I do document most of the things I do but I have to admit I have been slacking a bit, recently. There is just so much stuff that needs doing and a lot of interesting projects to learn about that sometimes stuff gets forgotten.
My personal impression of the linux space is still that folks get dumped on by the community for not being immersed in the nitty gritty though.
Thats neither fun nor will it work to get more people interested in linux. People make mistakes, learn to help without judging.
Have a good one.
Just be careful
Wow, a lot of people would set up a new server because of intrusion attempts in a log i guess. If I did that in a job I‘d get fired for doing nothing else but resetting everything every week.
As an admin, you have to keep the CTO from using „master“ or „admin“ as the ssh password on a production server. Just so you know what level of stupidity makes the big bucks out there.
As an admin I’d question why the CTO has a login on a production server.
You would do well listening more when you ask for advice.
You would do well keeping your condescension to yourself. Blocked.
Oh no, entitled asshole who talks down to everyone who has an answer for them blocks me. Whatever am I to do.
You’re saying you see a bunch of login attempts on your router, but you don’t think they actually got into it?