How to securely run multiple external services in your single public IP home

Many of us have this challenge. We want to run multiple services (web sites) and access them remotely. It might be plex, homeassistant, a blog, etc. The traditional way to to this is to use 1 IP per server. The traditional method to do this in a NAT/home environment is port-forwards, and just remember http://myhouse:80 == plex, http://myhouse:81 == homeassistant, etc. But there is a better way, we can use vhost/SNI to achieve this with nginx and Let's Encrypt, and have all sites be HTTPS, secure, on port 443. (Note, I'm ignoring the specifics of IPv6 here, that would give a method to have each service have an IP, but the reverse proxy makes the SSL configuration simpler).

So, what are the steps? First, pick a spot to run the reverse proxy (nginx). The other services can run on this server, or others.

Next, we port-forward port 80 and 443 to this server. We need port 80 at a minimum for the Let's Encrypt, but also its nice to 301 redirect your users if they accidentally type http://service to https://service.

Next, go to your dns provider. You may as well be fancy and register your own domain name, its cheap, but you can use a dynamic dns site if you prefer. Create a name for each service. In my case, I have 'plex.donbowman.ca', 'git.donbowman.ca', 'blog.donbowman.ca', 'ha.donbowman.ca', and 'cloud.donbowman.ca' (amongst others). Give all of them the same IP, that of your external side of your router (you can find this by going to http://icanhazip.com). You can either make them CNAME's (aliases) for your router's name, or give them the IP directly as A records.

OK, now lets go and install nginx on this server. And configure it thusly. My underlying hostname that runs nginx is 'nas'. For each service (here i just show 'blog' and 'plex'), add a paragraph as shown to the 'default' config. What this achieves is allows the special /.well-known URL (used by Let's Encrypt) to hit the filesystem, and the rest are redirected to the same hostname but HTTPS.

server {
    listen 80;
    server_name nas.donbowman.ca;

    root /var/www;

   location ~ /.well-known {
        allow all;
    }

    location / {
        return 301 https://nas.donbowman.ca$request_uri;
    }
}
server {
    listen 80;
    server_name blog.donbowman.ca;
    root /var/www;
    location ~ /.well-known {
        allow all;
    }
    location / {
        return 301 https://blog.donbowman.ca$request_uri;
    }
}
server {
    listen 80;
    server_name plex.donbowman.ca;
    root /var/www;
    location ~ /.well-known {
        allow all;
    }

    location / {
       return 301 https://plex.donbowman.ca$request_uri;
    }
}
 . . .

OK, now we are going to make an include file for our TLS configuration. Mine is don-tls.conf, and it looks as below. I have configured this to achieve an A+ score on Qualys SSL Test.

    ssl_protocols TLSv1.3 TLSv1.2;

    ssl_certificate /etc/letsencrypt/live/donbowman.ca/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/donbowman.ca/privkey.pem;
    ssl_trusted_certificate /etc/letsencrypt/live/donbowman.ca/fullchain.pem;

    ssl_dhparam /etc/ssl/certs/dhparam.pem;
    ssl_ecdh_curve secp384r1;
    ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH:!AES128';
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m;
    ssl_session_tickets on;
    ssl_stapling on;
    ssl_stapling_verify on;
    resolver 8.8.8.8 8.8.4.4 valid=300s;
    resolver_timeout 5s;
    add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload";
    add_header X-Frame-Options SAMEORIGIN;
    add_header X-Content-Type-Options nosniff;

OK, great, now lets go on to the config of each service and the creation of the SSL keys. Lets create a file called 'plex.conf' as below:

upstream plex {
        server 127.0.0.1:32400;
}

server {
    listen 0.0.0.0:443 ssl http2;
    server_name plex.donbowman.ca;

    include don-tls.conf;

    access_log /var/log/nginx/a-plex.log;
    error_log /var/log/nginx/e-plex.log;

    location / {
        proxy_pass http://plex;
        proxy_http_version 1.1;
        proxy_request_buffering off;
        proxy_set_header Connection "";
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "$connection_upgrade";
        proxy_read_timeout 36000s;
        proxy_pass_request_headers on;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
    }
}

OK, what this does, it says if the server_name (which is the SNI field of TLS) is 'plex.donbowman.ca' (regardless of IP), activate this file. We include the common TLS config, give a separate log file per service (this is optional), and then redirect all locations (location /) to the 'real-port' plex runs on, which is 32400 on the same host. The proxy_set* etc allow us to use WebSockets if needed, and to make it a bit more efficient (we disable buffering since its local pass through). OK, we are done our nginx config after setting one of these up per service.

OK, that leaves us one component, the certificates from Let's Encrypt. We will create them once by hand, and then set a cron-job to refresh them if needed.

First, run

certbot-auto certonly --webroot --webroot-path /var/www --rsa-key-size 4096 -d donbowman.ca -d nas.donbowman.ca -d rtr.donbowman.ca -d cloud.donbowman.ca -d plex.donbowman.ca  -d blog.donbowman.ca ...

e.g. add a -d name for each name you want a certificate for. These will all be encoded as altnames into the same cert. The certbot-auto will put a file into /var/www/.well-known for each, and then check it can reach it from externally (to prove you own the domain). So make sure your IP/DNS is set up from above!

OK, once this is done, all that remains is to put a

certbot/certbot-auto renew --rsa-key-size 4096 

in a daily cron, and we are renewing when needed. At this stage, reload your nginx. You may find you need to give permissions to nginx (www or www-data) to the certs, so you might do:

find /etc/letsencrypt/live /etc/letsencrypt/archive -print0 | xargs -0 chmod 755

or add the nginx user (www or www-data) to a 'ssl' or 'cert' group, and chgrp the files. Up to you.

At this stage, anyone in the world can start typing 'https://mysite.mydomain' and hit it directly! So make sure they are all secure. You may choose to put http-basic on in front of a few things that are not well secured. But, it works well because now you can access your plex from anywhere (configure the custom ssl cert in the plex server config), use your homeassistant from an app, etc. All due to the power of the 'SNI' and the reverse proxy.

 

Leave a Reply

Your email address will not be published. Required fields are marked *

*