DNS in the homelab is handled by four components providing ad-blocking, DoT/DoH, recursive resolution, in-cluster DNS, and automatic record management.
Technitium is deployed as raw Kubernetes manifests (no Helm chart) in the technitium namespace at sync wave 6.
Primary (technitium-primary):
| Setting | Value |
|---|---|
| Image | technitium/dns-server:14 |
| Namespace | technitium |
| Sync Wave | 6 |
| Admin Password | Vault: kv/technitium/admin |
| Service | LoadBalancer at 192.168.88.11 (externalTrafficPolicy: Local) |
| Ingress | https://dns.homelab.vyanh.uk (DoH on port 8443, web UI on port 5380) |
| Storage | longhorn 2Gi PVC |
| Resources | Deployed via raw manifests: primary-deployment.yaml, primary-pvc.yaml, primary-service-dns.yaml, primary-service-web.yaml |
Secondary (technitium-secondary):
| Setting | Value |
|---|---|
| Image | technitium/dns-server:14 |
| Service | LoadBalancer at 192.168.88.13 |
| Storage | longhorn 2Gi PVC |
| Anti-affinity | Runs on different node from primary |
| Ingress | https://dns2.homelab.vyanh.uk |
Tertiary (technitium-tertiary):
| Setting | Value |
|---|---|
| Image | technitium/dns-server:14 |
| Service | LoadBalancer at 192.168.88.16 |
| Storage | longhorn 2Gi PVC |
| Ingress | https://dns3.homelab.vyanh.uk |
MikroTik Instance (fourth instance):
| Setting | Value |
|---|---|
| Location | MikroTik container (app-technitium) on virtual IP 192.168.88.53 |
| Web UI | http://dns-mikrotik.homelab.vyanh.uk |
| Zones | homelab.vyanh.uk + vyanh.uk as Secondary AXFR from 192.168.88.11 |
| Forwarder | Primary (udp://192.168.88.11) + Cloudflare DoH fallback |
A configure-job.yaml ArgoCD PostSync hook (BeforeHookCreation) configures all four instances via the Technitium API:
dnsServerDomain per-instance (dns.homelab.vyanh.uk / dns2 / dns3 / dns-mikrotik)dnssecValidation=true)AllowOnlyForPrivateNetworkstechnitium-tls secret)enableDnsOverHttp+X-Forwarded-Proto does NOT work)qnameMinimization=true, randomizeName=true, eDnsClientSubnet=falsecacheMaximumRecordTtl=86400, cacheMinimumRecordTtl=60, serveStaleTtl=86400, prefetch enabled (cachePrefetchEligibility=2, cachePrefetchTrigger=9)clientQueryRateLimit=200 QPS per clientTechnitium has no native cache sync between instances. Instead, secondary/tertiary/MikroTik forward to the primary first — effectively sharing its cache.
| Instance | Forwarders | Protocol |
|---|---|---|
| Primary | cloudflare-dns.com/dns-query, dns.google/dns-query |
DoH (HTTPS) |
| Secondary | 10.99.188.56 (primary ClusterIP), cloudflare-dns.com/dns-query |
UDP + DoH |
| Tertiary | 10.99.188.56 (primary ClusterIP), cloudflare-dns.com/dns-query |
UDP + DoH |
| MikroTik | 192.168.88.11 (primary LB IP), cloudflare-dns.com/dns-query |
UDP + DoH |
How it works (concurrentForwarding=true): both upstreams are queried simultaneously. When primary has a cache hit it responds in <1ms over LAN — Cloudflare (~30ms) cannot win. On a cache miss, Cloudflare responds in ~30ms and caches locally on that replica; primary also caches the result independently the next time it's asked.
Note: Secondary/Tertiary use the ClusterIP (
10.99.188.56) — pods cannot reach MetalLB LB IPs due to hairpin NAT. MikroTik is external to the cluster so it uses the LB IP (192.168.88.11).
URI-scheme format: forwarders use
udp://andhttps://URI prefixes so Technitium resolves the mixed-protocol list correctly.forwarderProtocol=Udpis the base default (overridden per-entry by the URI scheme).
| Source | URL |
|---|---|
| StevenBlack/hosts | https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts |
| AdGuard DNS filter | https://adguardteam.github.io/AdGuardSDNSFilter/Filters/filter.txt |
| URLhaus | https://urlhaus.abuse.ch/downloads/hostfile/ |
| HaGeZi Light | https://cdn.jsdelivr.net/gh/hagezi/dns-blocklists@latest/adblock/light.txt |
Important: Each blocklist URL must be a separate
--data-urlencodeparam in the configure-job curl call. A single combined string is silently ignored by the Technitium API.
Installed from the Technitium app store (non-fatal if unavailable):
Technitium primary hosts the homelab.vyanh.uk authoritative zone. ExternalDNS manages records in this zone via rfc2136+TSIG. The secondary and MikroTik instances pull this zone via AXFR.
Public *.vyanh.uk hostnames exposed via the Cloudflare tunnel are also overridden locally so LAN clients reach the Istio gateway directly — bypassing Cloudflare entirely.
The configure-job.yaml PostSync hook runs an ensure_split_dns function on every ArgoCD sync. For each public hostname it:
192.168.88.12 (Istio Gateway VIP, MetalLB L2), TTL 300This runs against all three K8s Technitium instances plus the MikroTik tertiary.
| Zone | A Record | Backend via Istio |
|---|---|---|
vault.vyanh.uk |
192.168.88.12 |
Vaultwarden :8843 on NAS |
wiki.vyanh.uk |
192.168.88.12 |
WikiJS :3080 on NAS |
status.vyanh.uk |
192.168.88.12 |
Uptime Kuma :3001 on RouterOS container |
tracker.vyanh.uk |
192.168.88.12 |
LifeOps in K8s |
ntfy.vyanh.uk |
192.168.88.12 |
ntfy :2586 on NAS |
auth.vyanh.uk |
192.168.88.12 |
Ente museum in K8s |
accounts.vyanh.uk |
192.168.88.12 |
Ente accounts in K8s |
dns.vyanh.uk |
192.168.88.12 |
Technitium DoH endpoint |
tasks.vyanh.uk |
192.168.88.12 |
Vikunja in K8s (added 2026-04-30) |
Each Istio Gateway listener serves the *.vyanh.uk hostname with its own dedicated TLS cert (<name>-public-tls) issued via the letsencrypt-dns01 ClusterIssuer. The internal *.homelab.vyanh.uk hostnames continue to ride the wildcard homelab-wildcard-tls.
| Feature | Test | Result |
|---|---|---|
| DoT | openssl s_client -connect 192.168.88.11:853 |
LE cert chain ✓ |
| DoH | kdig +https @dns.homelab.vyanh.uk cloudflare.com |
HTTP/2-POST status 200 ✓ |
| Blocking | dig doubleclick.net @192.168.88.11 |
NXDOMAIN + EDE "Blocked" ✓ |
| DNSSEC | dig +dnssec google.com @192.168.88.11 |
flags: qr rd ra ad ✓ |
Warm cache (most queries in practice):
| Server | Avg latency |
|---|---|
| Technitium primary | 0–1 ms |
| Cloudflare 1.1.1.1 | ~30 ms |
| Google 8.8.8.8 | ~73 ms |
Cold cache (first-time lookup, no local cache):
| Server | Avg latency |
|---|---|
| Technitium primary | ~54 ms |
| Cloudflare 1.1.1.1 | ~100 ms |
| Google 8.8.8.8 | ~48 ms |
Technitium beats Cloudflare cold because concurrentForwarding=true races Cloudflare + Google simultaneously and takes the faster response.
Throughput (50 sequential queries):
| Run | Total | Per-query avg |
|---|---|---|
| Technitium cold (1st pass) | 2.8 s | 56 ms |
| Cloudflare cold | 6.2 s | 123 ms |
| Technitium warm (2nd pass) | 0.2 s | 4 ms |
DNSSEC: cloudflare.com → AD flag set ✓. dnssec-failed.org → SERVFAIL ✓ (validation working).
Blocklist test: doubleclick.net, ads.google.com, pagead2.googlesyndication.com all return NXDOMAIN at 0 ms.
Technitium v14 has no Prometheus /metrics endpoint — the built-in web dashboard at dns.homelab.vyanh.uk covers stats, query logs, blocklist hits, and top clients. No Grafana dashboard or vmagent scrape job is needed.
dns.vyanh.uk)Technitium's DoH endpoint is exposed publicly so phones, tablets, and laptops on cellular networks can use the homelab resolver (with adblock and split-DNS for *.vyanh.uk) instead of whatever resolver the local network hands out.
| Element | Value |
|---|---|
| URL | https://dns.vyanh.uk/dns-query |
| Istio listener | https-dns-public on homelab-gateway |
| TLS cert | dns-public-tls (Let's Encrypt DNS-01) |
| Backend | Technitium DoH on K8s — primary first, secondary on health-check eject |
| WAF | CrowdSec ext_authz disabled for this hostname (crowdsec-ext-authz-dns-disable EnvoyFilter) — DoH must not be rate-limited or scored |
Public DoH failover (2026-04-25): the gateway uses a Cloudflare-style "outlier detection" config that ejects the primary DoH backend after a few consecutive 5xx and routes to the secondary. A blackbox-exporter probe runs every 30s against dns.vyanh.uk/dns-query?dns=... (a DoH GET for cloudflare.com.) and pages on homelab-alerts if the response is anything other than a valid DoH binary body.
The
dns.vyanh.ukCloudflare DNS record is proxied: false (gray cloud). Cloudflare's HTTP-aware proxy strips the binary DoH content type. The record points straight at the WAN IP, with the MikroTik publishing port 443 to Istio.
Apple devices can be configured to use the homelab DoH resolver system-wide via a signed configuration profile served at:
https://dns.vyanh.uk/apple.mobileconfig
This is a small XML profile (Settings > General > VPN & Device Management > DNS) that pins the iPhone / iPad / Mac to https://dns.vyanh.uk/dns-query for all DNS, including queries from apps that try to bypass Wi-Fi-level DNS. Once installed it follows the device anywhere — coffee shops, cellular, hotel Wi-Fi — keeping adblock + LAN split-DNS active everywhere.
How it's served:
apple.mobileconfig (with the right Cloudflare proxy IPs / hostname) into the Technitium nginx html dir at startup.technitium app holds the config — see core-components/technitium/resources/configure-job.yaml.To install on iOS / macOS:
https://dns.vyanh.uk/apple.mobileconfig in Safari.https://dns.vyanh.uk/dns-query.https://1.1.1.1/help — should report "Using DNS over HTTPS (DoH): Yes" and the resolver should be the homelab one.Two gotchas worth remembering:
PayloadDescription, PayloadDisplayName, PayloadIdentifier, PayloadType, PayloadUUID, PayloadVersion — iOS silently rejects profiles missing any of these (no error UI, just nothing happens after the prompt).ServerAddresses array inside the DNSSettings dict needs the Cloudflare anycast IPs the proxied tunnel actually resolves to. Hard-coded 1.1.1.1 works as a fallback if dns.vyanh.uk is unreachable. Re-render the profile if Cloudflare's anycast set changes.| Setting | Value |
|---|---|
| Sync Wave | -10 (deploys first, before everything) |
| Namespace | kube-system |
| Type | ConfigMap (custom Corefile) managed by ArgoCD coredns-custom app |
CoreDNS uses split-horizon DNS with a dedicated zone block for homelab.vyanh.uk. This means:
*.homelab.vyanh.uk A queries return the Istio Gateway ClusterIP directly (see hairpin NAT note below)cluster.local is handled natively by the kubernetes plugin as alwaysPod DNS query
│
▼
CoreDNS (10.96.0.10)
│
├─ *.cluster.local ──→ kubernetes plugin (in-cluster service discovery)
│
├─ *.homelab.vyanh.uk ──→ template plugin → 10.102.56.238 (Istio Gateway ClusterIP)
│
└─ everything else ──→ Technitium primary → Technitium secondary
→ 1.1.1.1 → 8.8.8.8
An earlier version forwarded homelab.vyanh.uk queries to Technitium, which is authoritative and returns 192.168.88.12 (the MetalLB LoadBalancer IP). This broke all in-cluster SSO:
Pod → resolves authentik.homelab.vyanh.uk → 192.168.88.12
Pod → connect 192.168.88.12:443 → i/o timeout
Why the timeout? MetalLB L2 mode announces 192.168.88.12 on the physical LAN via ARP. kube-proxy sets up DNAT for external traffic arriving at the node, but does not DNAT pod-originated traffic destined for the LB IP — so packets from pods hit the node's physical interface and go nowhere.
Fix: The homelab.vyanh.uk:53 block uses a template plugin to return the Istio Gateway ClusterIP (10.102.56.238) for all A queries. ClusterIPs are always reachable from any pod on any node. Since every *.homelab.vyanh.uk service is fronted by the Istio gateway, the ClusterIP is always the correct in-cluster destination.
The ClusterIP is stable — it only changes if the homelab-gateway-istio Service is deleted and recreated (which ArgoCD would handle).
# All *.homelab.vyanh.uk services are fronted by the Istio gateway.
# Return the Istio Gateway ClusterIP (10.102.56.238) for all A queries from pods.
# Pods cannot reach the MetalLB LoadBalancer IP (192.168.88.12) due to hairpin NAT:
# the external LB IP is announced via ARP on the physical network but kube-proxy
# does not DNAT pod→LB traffic back into the cluster, causing i/o timeout.
homelab.vyanh.uk:53 {
template IN A homelab.vyanh.uk {
answer "{{ .Name }} 60 IN A 10.102.56.238"
}
template IN AAAA homelab.vyanh.uk {
rcode NOERROR
}
cache 60
errors
log . {
class error
}
}
.:53 {
errors
health { lameduck 5s }
ready
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure
fallthrough in-addr.arpa ip6.arpa
ttl 30
}
prometheus :9153
# Suppress AAAA lookups — cluster has no IPv6 pod routing
template IN AAAA { rcode NOERROR }
# Suppress gRPC service-config TXT lookups (_grpc_config.*)
template IN TXT {
match ^_grpc_config\.
rcode NXDOMAIN
}
# Route public queries through Technitium (blocklists, DNSSEC, cache)
# then fall back to public DNS if Technitium is unreachable
forward . 192.168.88.11 192.168.88.13 1.1.1.1 8.8.8.8 {
max_concurrent 1000
health_check 5s
}
cache 300 {
disable success cluster.local
disable denial cluster.local
}
loop
reload
loadbalance
}
hosts {} Block?An earlier version hardcoded 5 *.homelab.vyanh.uk hostnames in a hosts {} block pointing to the Istio Gateway ClusterIP. This had two problems:
nextcloud.homelab.vyanh.uk, vault.homelab.vyanh.uk, wikijs.homelab.vyanh.uk, etc. got NXDOMAINhosts {} does not support *.homelab.vyanh.ukThe template plugin fixes both: it matches all A queries for any subdomain of homelab.vyanh.uk and always returns the correct ClusterIP. New services added via ExternalDNS work automatically.
During initial cluster bootstrap, CoreDNS starts at wave -10, well before Technitium (wave 6) and MetalLB (wave 1). The template plugin has no external dependency — it answers entirely from config. DNS for homelab.vyanh.uk is available immediately on startup.
K8s nodes use systemd-resolved for system-level DNS (used by kubelet and containerd for image pulls). This is separate from CoreDNS (which pods use).
Problem discovered 2026-03-09: All 4 nodes had 8.8.8.8/1.1.1.1 hardcoded in their cloud-init netplan config. Containerd cannot resolve harbor.homelab.vyanh.uk via public DNS → image pulls from Harbor fail silently for any image not already cached on that node.
Fix: All nodes updated to use 192.168.88.1 (MikroTik) as primary system DNS, 8.8.8.8 as fallback. MikroTik is configured to forward to all three Technitium instances.
Config location on each node: /etc/netplan/50-cloud-init.yaml
nameservers:
addresses:
- 192.168.88.1 # MikroTik → Technitium (resolves homelab.vyanh.uk)
- 8.8.8.8 # Fallback for public DNS during MikroTik outage
Nodes fixed: k8s-node1 (192.168.88.249), k8s-node2 (192.168.88.248), k8s-node3 (192.168.88.247), k8s-controlplane
Verify on any node:
ssh n1 "resolvectl query harbor.homelab.vyanh.uk"
# Expected: harbor.homelab.vyanh.uk: 192.168.88.12
Warning:
netplan applyon these nodes requires a PTY (password prompt blocks non-interactive sudo). Usekubectl runwith a privileged nsenter pod to apply changes remotely without SSH password.
| Setting | Value |
|---|---|
| Chart | external-dns v1.19.0 |
| Namespace | external-dns |
| Sync Wave | 7 (after Technitium) |
| Provider | rfc2136 (TSIG HMAC-SHA256) |
| Policy | upsert-only (never deletes records) |
| Sources | ingress, service |
| Domain Filter | homelab.vyanh.uk |
| rfc2136 Host | 192.168.88.11, port 53 |
| Zone | homelab.vyanh.uk |
| TSIG Key | external-dns-key, alg: hmac-sha256 |
| Secret | Namespace | Vault Path | Notes |
|---|---|---|---|
technitium-tsig |
external-dns |
kv/external-dns/tsig |
TSIG key for rfc2136 updates |
Generate TSIG secret: openssl rand -base64 32
ExternalDNS watches Kubernetes HTTPRoute (Gateway API) and Service resources. When a new HTTPRoute hostname matches homelab.vyanh.uk, ExternalDNS creates the corresponding A record in Technitium via rfc2136 (dynamic DNS update with TSIG authentication), pointing to the Istio Gateway VIP 192.168.88.12.