This guide is opinionated, primarily aimed at [[Homelab]]s with a preference to [[LXC]] (system container) workloads, instead of application container workloads ([[Docker]], [[Kubernetes]], etc).
# Requirements
1. [[Internet]] access
2. [[Cloudflare]] account
3. [[Proxmox]] VE ([[PVE]])
4. [[DNS]] server ([[Pi-hole]] or similar)
[[Traefik-Proxy]] requires prior planning to deploy securely and successfully.
For our guide we will be using the following resources and hosts ([[KVM]]/[[LXC]] servers):
mdn = my domain name
| Host | Resource name | IPv4 | Notes |
| ----------------- | ----------------------- | --------------- | -------------------------------- |
| [[Cloudflare]] | `
[email protected]` | N/A | Cloudflare account email address |
| [[Cloudflare]] | `mdn.com` | N/A | an ICANN registered domain name |
| [[PVE]] | `prod.pve.mdn.com` | `192.168.0.111` | [[PVE]] [[BM]] host |
| [[Pi-hole]] | `pihole.mdn.com` | `192.168.0.222` | [[PVE]] [[LXC]] host |
| [[HAOS]] | `haos.mdn.com` | `192.168.0.99` | [[PVE]] [[KVM]] host |
| [[Traefik-Proxy]] | `traefik-proxy.mdn.com` | `192.168.0.11` | [[PVE]] [[LXC]] host |
For hosts which support secure communications using certificates, we will use [[Let's Encrypt]] ([[LE]]) to automate certificate issuance and renewal. For [[LE]] certificate issuance to work, [[LE]] needs to verify we somehow own the domain. How [[LE]] verifies we own the domain name (`mdn.com`), is by being able to write and then delete a TXT entry in the DNS zone of our domain. We will achieve this goal using the [[Cloudflare]] [[DNS]] services, but you can also choose a [different provider](https://doc.traefik.io/traefik/https/acme/#providers).
# Preparation - [[Cloudflare]]
Login to your [[Cloudflare]] account.
It is assumed you already have a domain name registered. If you do not you can purchase one from Cloudflare, which is the easiest way to get started. In this guide our domain name is: `mdn.com`
Go to your Account Profile.
![[traefik-proxy-cf-0.png]]
<button class="navigate">Create Token</button>
![[traefik-proxy-cf-1.png]]
<button class="navigate">Get Started</button>
![[traefik-proxy-cf-2.png]]
Name the token, add Permissions and select Zone Resources, as provided in the image below. I suggest, even if you have one domain registered under your account, to still specify the zone you want the permission to be applied to (Specific Zone).
<button class="navigate">Continue to Summary</button>
![[traefik-proxy-cf-3.png]]
<button class="navigate">Create Token</button>
![[traefik-proxy-cf-4.png]]
Ensure you **Copy** and save the token in a safe and secure place (e.g. [[Vaultwarden]]), as there are no tools to allow you to see generated tokens with the exception of Global and Origin API Keys. We need to use this key in the [[Traefik-Proxy]] config, to allow [[LE]] automatically manage our certificates through [[Cloudflare]]. If you lose the generated token the only way to recover is to create a new one and apply the new one in the [[Traefik-Proxy]] config.
![[traefik-proxy-cf-5.png]]
Click **View all API tokens** to ensure the newly generated key is available and active.
![[traefik-proxy-cf-6.png]]
# Setup [[Traefik-Proxy]]
[[Traefik-Proxy]] supports a static config and a dynamic config. Any change in the static config requires a manual restart of [[Traefik-Proxy]] to activate, while changes in the dynamic config are enacted instantly, while [[Traefik-Proxy]] is running.
![[traefik-proxy-architecture.png]]
## Static config
> [!cli]+ <code class="g-code">nano /etc/traefik/traefik.yaml</code>
>
>```YAML
>################################################
>################################## observability
>################################################
>
>#----------------------: https://doc.traefik.io/traefik/contributing/data-collection/
>global:
checkNewVersion: true
sendAnonymousUsage: true # send anonymous usage data
>
>#----------------------: https://doc.traefik.io/traefik/operations/api/
>api:
> dashboard: true
> insecure: false # access to http://traefikIPv4:8080/dashboard/ is disabled
> debug: false
> disableDashboardAd: true
>
>#----------------------: https://doc.traefik.io/traefik/observability/access-logs/
>accesslog:
> addInternals: true
> filePath: "/var/log/traefik-access.log"
> bufferingSize: 128
>
>#----------------------: https://doc.traefik.io/traefik/observability/logs/
>log:
> filePath: "/var/log/traefik.log"
> level: INFO # TRACE DEBUG INFO WARN ERROR FATAL PANIC
> maxAge: 48
>
>#----------------------: https://doc.traefik.io/traefik/observability/metrics/overview/
>metrics:
> addInternals: true
>
>#----------------------: https://doc.traefik.io/traefik/observability/tracing/overview/
>#tracing:
># addInternals: true
># otlp: {}
>
>################################################
>#################################### environment
>################################################
>
>#----------------------: https://doc.traefik.io/traefik/routing/entrypoints/
>entryPoints:
> http:
> address: ":80"
> http:
> middlewares:
> - internal-hosts-endorsed
> https:
> address: ":443"
> http:
> middlewares:
> - internal-hosts-endorsed
>
>#----------------------: https://doc.traefik.io/traefik/providers/overview/
>providers:
>#----------------------: https://doc.traefik.io/traefik/providers/docker/
># docker:
># exposedbydefault: false
>#----------------------: https://doc.traefik.io/traefik/providers/file/
> file:
># filename: /etc/traefik/dynamic.yaml # when a specific file is desired
> directory: /etc/traefik/dynamic
> watch: true
>
>#----------------------: https://doc.traefik.io/traefik/https/acme/
>certificatesresolvers:
> cloudflare:
> acme:
> caServer: https://acme-v02.api.letsencrypt.org/directory # prod
># caServer: https://acme-staging-v02.api.letsencrypt.org/directory # test
> email:
[email protected] # valid Cloudflare-account email
> storage: /etc/traefik/acme.json
> dnschallenge:
> provider: cloudflare
> resolvers:
> - "1.1.1.1:53"
> - "1.0.0.1:53"
>```
>We protect the *entryPoints* with middleware which narrows down the IPv4 address space allowed to access our backend infrastructure. You will see how middleware *internal-hosts-endorsed* is configured in the dynamic config.
>
>We will use a multiple-files dynamic config, which are saved under directory `/etc/traefik/dynamic`. `core.yaml` will handle most of the configuration, except **routers** and **services** for actual backend hosts. `hosts-http.yaml` will handle all **HTTP** hosts, while `hosts-https.yaml` will handle all **HTTPS** hosts. You can choose any different arrangement to suit your environment, assuming all dynamic config files are saved in the same *watched* directory (`/etc/traefik/dynamic` in our guide).
>
>If you are in testing mode, comment the # prod *caServer* and uncomment the # test *caServer*, so you do not hit the [Let's Encrypt rate limits](https://letsencrypt.org/docs/rate-limits/)., which will disable the ability to generate certificates for days.
>
>For testing purposes,you can empty the content of the `acme.json` file at any time to get new certificates issued.
>
## Dynamic config
As we are securing our dashboard, we first need to decide what user names and passwords will be allowed. In our example we will use the following:
Username: dashuser1
Password: letmein
Username: dashuser2
Password: letmein
[[Traefik-Proxy]] accepts MD5 hashed passwords, and to generate them we will use `openssl` which comes standard with [[Debian]].
<code class="g-code">openssl passwd -1 "my-pass"</code> - password MD5 hash: `$1$M196zkpJ$vw.T7SKDkQknMO2D/vvRU/`
> [!cli]+ <code class="g-code">nano /etc/traefik/dynamic/core.yaml</code>
>
>```YAML
>http:
>
>#----------------------: https://doc.traefik.io/traefik/routing/routers/
> routers:
>
> # harden dashboard access: can only be accessed with a username/password
> dashboard:
> rule: "Host(`traefik-proxy.mdn.com`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))"
> service: api@internal
> middlewares:
> - auth
>
> # catchall rule, evaluated when no router exists for a request;
> # applicable to HTTP and HTTPS entryPoints only
> catchall:
> entryPoints:
> - "http"
> - "https"
> rule: "PathPrefix(`/`)"
> service: unavailable
> priority: 1
>
>#----------------------: https://doc.traefik.io/traefik/routing/services/
> services:
>
> # Service that will always provide a 503 Service Unavailable response
> unavailable:
> loadBalancer:
> servers: {}
>
>#----------------------: https://doc.traefik.io/traefik/middlewares/http/overview/
> middlewares:
>
> auth:
> basicAuth:
> users: # users and their MD5 hashed passwords, granted access to the traefik-proxy dashboard
> - "dashuser1:$1$M196zkpJ$vw.T7SKDkQknMO2D/vvRU/"
> - "dashuser2:$1$M196zkpJ$vw.T7SKDkQknMO2D/vvRU/"
>
> internal-hosts-endorsed:
> ipAllowList:
> sourceRange:
> - "192.168.0.0/24"
>
> http-only:
> redirectScheme:
> scheme: http
> permanent: true
>
> internal-http-hosts:
> chain:
> middlewares:
> - internal-hosts-endorsed
> - http-only
>
> https-only:
> redirectScheme:
> scheme: https
> permanent: true
>
> # chains are useful when multiple middleware needs to be applied to a route,
> # especially if the chain has to be applied to multiple routes
> internal-https-hosts:
> chain:
> middlewares:
> - internal-hosts-endorsed
> - https-only
>
>#----------------------: https://doc.traefik.io/traefik/https/tls/
>tls:
> options:
> default:
> minVersion: VersionTLS13 # change to a lower version if you expect to service Internet traffic from around the world
> curvePreferences: # below priority sequence can be changed
> - X25519 # the most commonly used 128-bit
> - CurveP256 # the next most commonly used 128-bit
> - CurveP384 # 192-bit
> - CurveP521 # 256-bit
> sniStrict: true # true if our own certificates should be enforced
># certificates:
># - certFile: /etc/traefik/domain.cert
># keyFile: /etc/traefik/domain.key
># - certFile: /etc/traefik/certificate.pem
># keyFile: /etc/traefik/private_key.pem
>#### Traefik uses its own default certificate for connections without SNI, or without a matching domain.
>#### However, we can provide our own default certificate, instead of using the Traefik default.
># stores:
># default:
># defaultCertificate:
># certFile: /etc/traefik/cert.crt
># keyFile: /etc/traefik/cert.key
>#### Alternatively, we can use an ACME generated default certificate.
> stores:
> default:
> defaultGeneratedCert:
> resolver: cloudflare
> domain:
> main: mdn.com
> sans:
> - "*.mdn.com"
>```
> [!cli]+ <code class="g-code">nano /etc/traefik/dynamic/hosts-http.yaml</code>
>
>```YAML
>http:
>
>#----------------------: https://doc.traefik.io/traefik/routing/routers/
> routers:
>
> pihole:
> entryPoints:
> - "http"
> rule: "Host(`pihole.mdn.com`)"
> middlewares:
> - internal-http-hosts
> service: pihole
>
> homeassistant:
> entryPoints:
> - "http"
> rule: "Host(`haos.mdn.com`)"
> middlewares:
> - internal-http-hosts
> service: homeassistant
>
>#----------------------: https://doc.traefik.io/traefik/routing/services/
> services:
>
> pihole:
> loadBalancer:
> servers:
> - url: "http://192.168.0.222"
> passHostHeader: true
>
> homeassistant:
> loadBalancer:
> servers:
> - url: "http://192.168.0.99:8123"
> passHostHeader: true
>```
>Home Assistant needs to trust [[Traefik-Proxy]] and therefore the following lines must be added to our [[HA]] instance:
>
>> [!cli]+ <code class="g-code">configuration.yaml</code>
>>```
>>http:
>> use_x_forwarded_for: true
>> trusted_proxies:
>> - 192.168.0.11
>>```
>
> [!cli]+ <code class="g-code">nano /etc/traefik/dynamic/hosts-https.yaml</code>
>
>```YAML
>http:
>
>#----------------------: https://doc.traefik.io/traefik/routing/routers/
> routers:
>
> proxmox:
> entryPoints:
> - "https"
> rule: "Host(`prod.pve.mdn.com`)"
> middlewares:
> - internal-https-hosts
> tls:
> certResolver: cloudflare
> domains:
> - main: "prod.pve.mdn.com"
> service: proxmox
>
>#----------------------: https://doc.traefik.io/traefik/routing/services/
> services:
>
> proxmox:
> loadBalancer:
> servers:
> - url: "https://192.168.0.111:8006"
> passHostHeader: true
> serversTransport: "proxmox"
>
>#------------: https://doc.traefik.io/traefik/routing/services/#serverstransport_1
>#------------: https://www.cloudflare.com/en-gb/learning/ssl/what-is-sni/
> serversTransports:
>
> proxmox:
> insecureSkipVerify: true # set to true if you get "Internal Server Error"
>```
>
Prior to testing if the above [[Traefik-Proxy]] config works, we need to take a few additional steps:
><code class="g-code">export CLOUDFLARE_DNS_API_TOKEN="KCH_OP_Uqd1DTho1JebZw-ijjAmCS5CdERjVyZZL"</code> - type this bash command to pass the [[Cloudflare]] [[API]] token we created at the beginning of this guide (replace the value with your own [[API]] key) to [[Let's Encrypt]]
> **Add the following DNS records in [[Pi-hole]].**
>
>Create an A record for [[Traefik-Proxy]]:
>![[traefik-proxy-pihole-0.png]]
>
>Add [[CNAME]]s for all servers behind [[Traefik-Proxy]]. We could add A records (e.g. `haos.mdn.com` - `192.168.0.11`), but if we ever have to change the [[IPv4]] of [[Traefik-Proxy]], it would be too much work to enact the [[IPv4]] changes, especially if we have to change lots of servers. Using [[CNAME]]s we only have a single point of reference/change, which is the [[Traefik-Proxy]] [[IPv4]], and magically with just one change all our servers point to the right [[IPv4]].
>![[traefik-proxy-pihole-1.png]]
With all of that out of the way we can start [[Traefik-Proxy]]. Note the console will not display any information, as everything from now on is saved into logs.
<code class="g-code">traefik</code> - start [[Traefik-Proxy]]
http://traefik-proxy.mdn.com/dashboard/ - access the dashboard; you can no longer use [[IPv4]] addresses
![[traefik-proxy-signin.png]]
To disable the Sign in challenge, change *insecure: true* in `/etc/traefik/traefik.yaml`
Ensure you can successfully access the servers behind [[Traefik-Proxy]]:
- http://haos.mdn.com
- http://pihole.mdn.com/admin
- https://prod.pve.mdn.com
If everything works as expected, stop [[Traefik-Proxy]]. We need to take one last step: enable [[Traefik-Proxy]] to automatically start when the [[LXC]] boots.
>Optionally, at this point you can also explore what has been recorded in the logs.
>
><code class="g-code">less /var/log/traefik.log</code> - check the system log
>
><code class="g-code">less /var/log/traefik-access.log</code> - check the dashboard access log
>
><code class="g-code">less /etc/traefik/acme.json</code> - check the [[LE]] certificates store
> [!cli]+ <code class="g-code">nano /etc/systemd/system/traefik-proxy.service</code>
>
>```BASH
>[Unit]
>Description=Start Traefik Proxy
>Documentation=https://go-acme.github.io/lego/dns/cloudflare/
>
>[Service]
># declare the Cloudflare API token so Let's Encrypt can verify the domain name
>Environment="CLOUDFLARE_DNS_API_TOKEN=KCH_OP_Uqd1DTho1JebZw-ijjAmCS5CdERjVyZZL"
>ExecStart=/usr/local/bin/traefik
>Restart=always
>
>[Install]
># installs a hook to use this unit file when the system boots or shuts down
>WantedBy=multi-user.target
>```
>
<code class="g-code">systemctl enable traefik-proxy</code> - enable the unit file to automatically run at boot time
>For reference, here are the commands to control the [[Traefik-Proxy]] unit file (troubleshooting purposes):
>
> <code class="g-code">systemctl status traefik-proxy</code> - show the status of the unit file
>
><code class="g-code">systemctl disable traefik-proxy</code> - disable the unit file from automatically run at boot time
>
><code class="g-code">systemctl start traefik-proxy</code> - manually start the unit file
>
><code class="g-code">stop restart traefik-proxy</code> - manually restart the unit file
>
><code class="g-code">systemctl stop traefik-proxy</code> - manually stop the unit file
<code class="g-code">reboot</code> - reboot the system to ensure everything works without manual intervention
http://traefik-proxy.mdn.com/dashboard/ - ensure you can successfully access & login in the dashboard;
As a reminder, there is no need to restart [[Traefik-Proxy]] to change any of the following dynamic config files; the changes take effect instantly.
- `/etc/traefik/dynamic/core.yaml`
- `/etc/traefik/dynamic/hosts-http.yaml`
- `/etc/traefik/dynamic/hosts-https.yaml`
![[Traefik-Proxy#References]]
# Follow or Support me -> <a href='https://ko-fi.com/S6S0K9U5Q' target='_blank'><img height='36' style='border:0px;height:36px;float:right; ' src='https://storage.ko-fi.com/cdn/kofi1.png?v=3' border='0' alt='Buy Me a Coffee at ko-fi.com' /></a>