Read about site-to-site VPN configuration, it’s what you’re looking for.

You have several options:

1. Add DNS records to point to your firewall, forward the ports to the machine hosting the apps, secure your firewall by limiting access only from Cloudflare Proxy servers’ subnet (it’s publicly accessible here: https://www.cloudflare.com/ips/),
2. Use Cloudflare Tunnels to make your apps inside your LAN accessible publicly without opening the ports on your firewall. I recommend to host the Cloudflare Tunnel inside a docker container, because it automatically connects to Cloudflare and once you configure the apps you want to host in the Cloudflare web GUI, the tunnel will automatically set up a proxy for you.
3. Use VPN, you either set it up on a firewall or on some other machine and connect directly to your network. I recommend Wireguard, it’s stupid fast.

I have three VMs on Proxmox VE:

1. OPNsense as a router/firewall for my whole network,
2. “Zeus” (god from Greek religion, I usually name VMs as gods from Greek religion) - VM with multiple Docker containers: Jellyfin, Plex, Nextcloud etc.
3. UniFi - VM with just UniFi Controller in a Docker container. I wanted to separate the UniFi controller and Zeus’ apps, so that’s why they’re two VMs.
4. Other VMs, I use Proxmox VE to spawn more VMs and test some solutions at work.