Reverse Proxy With Caddy

10 Dec 2024 - Ben

I’ve never set up a reverse proxy before. The only thing I’ve done with Nginx is set up a simple file server for local development, based straight on their docker documentation.

But I wanted to give my local development environments names. To make them fel like real websites. My backed could live at “backend” instead of “localhost:3256”. My frontend at “frontend” and so on.

There always felt like so many moving pieces. The easiest method, modifying /etc/hosts didn’t solve it all. It would be hard to maintain, impossible to share with my coworkers, no SSL certificate, and I still needed port numbers. I needed a process that was easy to stand up, share, and felt magical without being too complicated.

Part of the reason I’m writing this blog is to centralize some of my most interesting or frustrating technical moments. So I’m writing this to help me recall this set up and hopefully for you to learn something from.

My Journey

DNS

I started the day trying to run pihole to resolve my services’ names. I got pihole up and running in docker only to learn that port 53 is already used in a Mac. Turns out the Bonjour service uses it, so I can’t run my DNS locally! In the end, it didn’t matter. Web browsers will resolve anything ending in .localhost to the loopback interface anyway. Great, problem solved! Along the way, I realized all my services run at different ports. But web browsers only ever connect at one port, 80 or 443. And while I could maybe use SRV records that seemed overly complicated, when I couldn’t even get pihole running. That’s when I knew I needed a reverse proxy.

The Reverse Proxy

A reverse proxy could listen to ports 80 and 443, then send that traffic to the right service running on a different port. Since all the services will have names, the reverse proxy can use those to decide where to route the traffic.

I knew I didn’t understand Nginx’s configuration very well. Traefik’s auto discovery bewilders me. Then I saw a two line Caddyfile for a reverse proxy and thought, “hmm, that’s so easy!” It still took me an afternoon of tweaks though. The container networking had to be set to host so Caddy could proxy to ports on my local machine (since those services don’t run in docker). Great, everything can talk to each other now! Except now my web browser doesn’t understand I’m in a secure context (no longer localhost Host), so I need an SSL certificate…

SSL Certificate

And Caddy created a cert for me! One line in bash and I had it copied out of the docker container. It still took me an hour to figure out to trust the thing in Keychain Access on a Mac, but I didn’t have to generate it!

My final hiccup of the day was with Vercel serverless functions. Locally, vercel dev also proxies whatever it’s running, and it didn’t like the custom name I gave it (frontend.localhost). It thought it was still hosted at localhost. Eventually, I figured out how to rewrite the Host header in the upstream and it was all good. And it was only ~3 lines in a Caddyfile! I feel very accomplished! I was lucky vercel dev worked with the Host rewrite because that’s still about all I know about proxying.

The Next Day

I’d learn the next day that Node doesn’t read your local SSL certs, or resolve *.localhost names the way the browser does. At least for now I’m able to effectively access all my sites with their names, instead of localhost:<port number>. That alone feels like a win, even if the configuration of my services still need to reference the localhost address. The rest of this I’ll write about another day.

Caddy example

I’ve changed the port numbers from my local dev environment, but this is how my Caddyfile is laid out with the lines for Vercel serverless under frontend.localhost.

backend.localhost {
  reverse_proxy localhost:4321
}

frontend.localhost {
  reverse_proxy /api/* localhost:2345 {
    header_up Host {upstream_hostport}
  }
  reverse_proxy localhost:2345
}

admin.localhost {
  reverse_proxy localhost:5738
}