letsencrypt with dns challenge - i.e. for non public pages and wildcard certificates

By default, letsencrypt's certbot uses a method where it verifies control over a webpage by prvoiding a challenge keyword via http to the letsencrypt server. If the webserver is however an intranet server which is not publicly accessible or if we want to generate a wildcard certificate, we need to use a DNS challenge, where certbot can prove to the letsencrypt servers, that we are in control of the DNS for our domain.

there are various DNS services out there who provide an API interface wiht support for certbot, but if you, like me, run your own bind based dns server, you can do this too by using rfc2136 dynamic updates (same as is used for dynamic dns services)

here is the documentation for the certbot side: https://certbot-dns-rfc2136.readthedocs.io/en/stable/

in the following I will describe how i have set this up for my domains, assuming my domain is called example.ch and I want to create a wildcard certificate for example.ch and www.example.ch

somehwere in your bind configs you should find your zone definition. we need to add a update-policy to the zone for example.ch that will allow us to update a TXT record for _acme-challenge.example.ch and _acme-challenge.www.example.ch as well. While you could use allow-update I highly discurage that, because if your TSIG key gets stolen, the attacker would have full control over your domain and could add new host entries as well as txt records to create SSH certificates, and we don't want that! By using a update-policy we can define more granular, waht a specific key is allowed to do.

here is the zone definition for example.ch

zone "example.ch" { 
    type master; 
    file "/etc/bind/db.example.ch"; 
    notify yes; 
    also-notify { 111.111.111.111; }; # ip of additional DNS server(s)
    update-policy {
        grant letsencrypt.example.ch. name _acme-challenge.example.ch. txt;
        grant letsencrypt.example.ch. name _acme-challenge.www.example.ch. txt;
    };
    allow-query { any; }; 
};

letsencrypt.example.ch. (notice the dot at the end) is the name of the key which will be used and what follows after name is the name of the dns entry that we grant permission to edit, txt is the type of entry we want to grant access to. certbot needs a txt entry.

somewhere before this zone definition, add an include statement to include our key file which will hold the TSIG keys used to grant such access. we will create the keyfile later on. So add this line somewhere above your zone definition:

include "/etc/bind/keys.conf";

next we create the key using the tsig-keygen command:

tsig-keygen letsencrypt.example.ch | tee -a /etc/bind/keys.conf

this should show something like this:

key "letsencrypt.example.ch" {
	algorithm hmac-sha256;
	secret "V0ncaFTw0uvlSC3NlreznNRqankXkPFIr3NWTDSe7B0=";
};

note down the key for later use.. in fact, copy the complete output to a file on your workstation to test this later on. this key should be editable by root only and readable to bind only, so lets change the permissions accordingly:

chmod 640 /etc/bind/keys.conf
chown root.bind /etc/bind/keys.conf

also we need to make sure that the directory where our zone file is located and the zonefile itself is writeable by bind:

chown bind.bind /etc/bind/db.example.ch /etc/bind
chmod 775 /etc/bind/db.example.ch /etc/bind

now lets reload the dns configuration

rndc reload 

we should now be able to set the text entry from a remote computer..

using the key we saved above, we can now use nsupdate to test our dns configuration. so run this on your local workstation where key.conf is the key you saved before

nsupdate -k key.conf -v
> server ns1.example.ch
> zone example.ch
> update add _acme-challenge.example.ch. 60 IN TXT "test123" 
> send
> quit

this should have updated the DNS entry, so let's query it using dig:

dig _acme-challenge.example.ch TXT @ns1.example.ch

you should now see your “test123” in the output.

now clean up and remove the test entry:

nsupdate -k key.conf -v
> server ns1.example.ch
> zone example.ch
> del _acme-challenge.example.ch.
> send
> quit

for certbot we need to create a credentials file which contains some info and our key. here is an example for our case:

# Target DNS server IP address
dns_rfc2136_server = 123.123.123.123
# Target DNS port
dns_rfc2136_port = 53
# TSIG key name
dns_rfc2136_name = letsencrypt.example.ch.
# TSIG key secret
dns_rfc2136_secret = V0ncaFTw0uvlSC3NlreznNRqankXkPFIr3NWTDSe7B0=
# TSIG key algorithm
dns_rfc2136_algorithm = HMAC-SHA256

pleas note,t hat the server address can only be an IP address there are no hostnames allowed here!(as of now.. there is an open issue to resolve this but it has been open for ages!

make sure this file is well protected, i.e. make it mode 600 and give it to the user that runs certbot for your renewals in the future.

now lets run certbot with this credentials file to request our certificate for example.ch and www.example.ch

certbot certonly --dns-rfc2136 --dns-rfc2136-credentials /tmp/credentials -d "www.example.ch" -d "example.ch" 

you can also request a wildcard certificate (needs the same line as the example.ch entry in the zone config of bind) like so:

certbot certonly --dns-rfc2136 --dns-rfc2136-credentials /tmp/credentials -d "*.example.ch"

that's it! now continue with your certbot setup as usual, i.e. add a cron job that regularly renews your certificate etc.