I’m kind of curious as to what people these days are doing on a UPS front, to keep systems running through power outages and provide a clean shutdown prior to batteries becoming exhausted.
It used to be common to see UPS systems sold to give desktop computer systems time to shut down cleanly.
The UPS market seems pretty stale to me. There have been changes over the past twenty years or so that I’d guess have caused some of that:
-
A move to filesystems structured so as to not risk corruption at the filesystem level from power loss at an arbitrary time.
-
Many people using laptops. Doesn’t change the situation much for servers, but I think that it reduces volume of the market that might want some kind of UPS.
I had expected that, with the drop in cost of lithium batteries and rise in tremendous rise in use of large batteries, that one would see new lithium-ion UPS units with large capacity.
But in practice, that doesn’t seem to be the case. UPS units are still around, but basically only provide a small amount of power, enough time to shut down. They aren’t normally geared up to keep systems running for hours.
There are lithium battery-based “home power backup” systems that provide loads of storage and automatic switching over to battery power if the mains power drops. However, these have some serious drawbacks that limit their use in a UPS role:
-
Some of these aren’t rated to switch over to battery power within a sharply-bounded amount of time, to avoid risk of momentary power interruption. For many devices, a momentary power interruption isn’t a huge deal, but for computers, it matters. I understand that on the order of 10 ms is expected for reliable UPS use, to keep computer power supplies happy.
-
One thing that one would like from a UPS is a clean shutdown prior to the battery becoming exhausted. For that to be done, the UPS needs to report its current charge capacity, so that software on the system can predict remaining runtime before exhaustion. Network UPS Tools is a widely-used Linux UPS-interfacing software package that does this shutdown. But looking at its hardware support grid, there isn’t support for these power stations, and I suspect that if there were reasonable charge-level reporting support anywhere, there would be.
-
USB has device classes that permits charge-level reporting, and looking at the USB spec, that appears to be true of USB PD. I have wireless headphones, for example, that make use of this. However, as best I can tell from looking through the kernel source, Linux doesn’t provide a way to treat these as a
power_supply
-class device, the way laptops have aBAT0
,BAT1
, etc, which would let the OS provide a clean shutdown itself when the time-remaining drops to a critical level. And even though power stations typically provide USB charging, I have not been able to find any that actually report their charge level via that USB in such a fashion.
I can think of at least three viable ways to do provide a large amount of backup power and a clean shutdown, based on what I’ve seen:
-
I’m sure that there are people who have rigged up some kind of ad hoc system off a full-blown grid-tie power system, with separate batteries, inverter, charge controller, etc. In that case, all one needs is a voltmeter linked to the batteries prior to the voltage-regulation stuff, knowing what battery type is involved, and one could give a capacity estimate. Doing this ad hoc is going to have some drawbacks that I’d hope that a vendor-provided battery management system wouldn’t, like having to calibrate to one’s batteries and not automatically dealing with battery aging.
-
Simply run a UPS and a “big-battery” lithium backup power station. Plug the UPS into the power station and the computer into the UPS. The UPS provides the rapid changeover time and provides the computer with a warning prior to shutdown. This uses systems that should work out-of-box, but doesn’t really seem ideal to me in that one’s buying extra hardware and doesn’t have a unified view of time remaining on the battery – the computer thinks that everything’s normal until the power station is drained and the UPS kicks on.
-
Some people use old laptops as servers. For those, you can already use the OS’s built-in power management to deal with laptop batteries. If you have a power station extending the runtime, great, though in that case, you run into the same “you don’t have a unified view of the laptop and power station battery charge” situation.
I’m pretty sure that people out there doing self-hosted servers have thought about this, and I’m curious as to what people out there are doing in terms of the options out there. Do you just not worry about it, given the fact that corruption at a filesystem level isn’t such a big deal? Do you just use a UPS for a handful of minutes prior to a clean shutdown, and not try to keep your systems running through longer power outages?
I also don’t know how resillient home Internet connections are in the presence of power outages, whether typical cable, fiber, and DSL connections remain functional from the telco’s standpoint. I know that cell towers typically have some sort of generator setup, as I’ve read about those in the past, and believe that I’ve read that they typically can run for at least several days without power even without technicians driving out. I don’t know to what degree that is also true of wired communications hardware. I’m curious as to what the experiences of people who have put their server and network hardware on some form of backup power is. If you keep your on-premises hardware powered, have you retained Internet connectivity in power outages that you’ve experienced?
You probably don’t need a UPS especially if it is more than a small fraction of the budget.
Use a modern filesystem like ZFS and you will be fine.
Its good for keeping your firewall or other services running, like homeassistant, during powered outages. But UPS also corrects power when it is dirty and protects your equipment.
I agree that it’s less-critical than it was at one point. Any modern filesystem, including ext4 and btrfs, isn’t at risk of filesystem-level corruption, and a DBMS like PostgreSQL or MySQL should handle it at an application level. That being said, there is still other software out there that may take issue with being interrupted. Doing an
apt
upgrade is not guaranteed to handle power loss cleanly, for example. And I’m not too sanguine about hardware not being bricked if I lose power during anfwupd
updating the firmware on attached hardware. Maybe a given piece of hardware has a safe, atomic upgrade procedure…and maybe it doesn’t.That does also mean, if there’s no power backup at all, that one won’t have the system available for the duration of the outage. That may be no big deal, or might be a real pain.
I’ve never had an issue personally. Actually repeated power loss is one of my tests when I make changes to my setup. Repeated power loss quickly reveals issues with startup and fail migration.