Where are you buying OptiPlex’s for 50? Get one for 30.
I would say look into i3 (more power efficient) and get some cheap budget SSDs or dirt cheap HDD (it will be plenty fast).
Because win11 requires TPMs there businesses had to replace office drone email terminals, so there are plates of those getting scrapped. Maybe less, so most probably have been turned to shredded metals.
You can host Wireguard or any other tunnel that you want inside of a container in the VPS.
I use VPNs inside of a container because they do not grant access to my network to host machine. Then on VPS you can also host something like traefik and that would apply to the VPN container.