Tailscale Sidecar Tutorial

-Ben

Adding Tailscale to your self hosted containers lets you easily set up those services with a secure, fully qualified domain name. It is easy to remember and easy to set up. Tailscale has their own Docker docs, but I want to expand on that starting point and use a different container engine.

There’s a lot to Tailscale that facilitates this magic. First, the Tailnet name means you can give each container a name. It’s easy to identify on the Tailscale Dashboard, it’s easy to type into a URL bar, it’s as easy to use as any URL! Second, OAuth tokens make it simple to connect containers (not just machines!) into your Tailnet. Third, HTTPS certificates mean web services behave like you would expect. They look secure in the browser and have access to secure context APIs. Lastly, Tailscale Serve proxies you to the correct port in your container.

Those services together make the experience pretty seamless to connect, once you know how all those pieces fit together. This post will walk you through all the steps you need to set this up today. We’ll be using FreshRSS as the service we’re proxying. I’m not going to use docker compose. Instead, I’ll be using podman and a handful of bash scripts to connect our containers together. This way, the tutorial will be a little more generic and should work for any oci compliant engine (docker, podman, nerdctl, etc).

Prerequisites

Tutorial

Pull the container images

If you skip this step, podman will pull these the first time we run the container. Either way is fine!

# pull tailscale
podman pull ghcr.io/tailscale/tailscale:latest
# pull freshrss
podman pull docker.io/freshrss/freshrss:latest

Set up some environment variables

This .env file will hold a few configuration variables we can share across our bash scripts. You can parameterize more or less, but this is a good start. HOSTNAME is how we’ll reference it on our tailnet at the end.

# file: .env
SIDECAR_NAME="freshrss_tailscale"
CONTAINER_NAME="freshrss"
HOSTNAME="rss"

Next, create a .ts.env file. This one will hold all the environment variables our tailscale sidecar container needs to connect to our tailnet. We can fill in everything in this file but our TS_AUTHKEY argument. We’ll generate that shortly.

# file: .ts.env
TS_AUTHKEY=tskey-client-notareal-tailscaleoauthtoken
TS_EXTRA_ARGS=--advertise-tags=tag:container
TS_STATE_DIR=/var/lib/tailscale
TS_USERSPACE=false
TS_SERVE_CONFIG=/config/ts.json

Pick a Tailscale Tailnet Name

In your Tailscale dashboard, go to DNS and find your Tailnet name. Tailscale uses yak-bebop as their example. You can’t pick the one you want (only generate it), and you also can’t change it once you turn on HTTPS. So take some time to generate one that you like. I like a fun name, but you can pick the default too!

We can make most of this tutorial work without HTTPS, so we won’t turn it on until the end.

Generate an OAuth client

An [OAuth client] gives you a more secure way to connect to your tailnet versus and Auth Key. Before we can generate the token, we need a tag to give our container. The official docs give a good reference for defining a tag. My short version: go to Tailscale Access Controls tab and add an entry under the tagOwner key named tag:container:

"tagOwners": {
  "tag:container": ["autogroup:admin"],
},

Save that with the button beneath and now we have a group we can assign the OAuth token to. So head over to your Tailscale Settings page, the “OAuth clients” option along the sidebar and click the “Generate OAuth Client” button. Type a description like “FreshRSS Test Token” and make sure you give it the “Devices:Core” and “Keys:Auth Keys” write permissions. Make sure you have the tag:container that we just created chosen under the “Add tag” dropdown. It should automatically assign to the second permission if you’ve set it for the first one.

Remember, “Devices:Core” and “Keys:Auth Keys” write permissions, assigned to your new tag, tag:container!

Click “Generate Client” and copy the key it gives you into your .ts.env file. You’ll never see this key again, but if you lose it just delete this one and regenerate a new one. The TS_AUTHKEY line should now have your auth token in place of our previous placeholder, giving you a file that looks like:

TS_AUTHKEY=tskey-client-yournew-tailscaleoauthtoken
TS_EXTRA_ARGS=--advertise-tags=tag:container
TS_STATE_DIR=/var/lib/tailscale
TS_USERSPACE=false
TS_SERVE_CONFIG=/config/ts.json

Running our Container

Now we have everything we need to run our container and access it over http. We need two containers, so we’ll write a short script to help coordinate it. I call it run, to mimic the podman run command. It will create 3 volumes, use the images we pulled in the first step, and our .env and .ts.env files to load everything up.

#!/usr/bin/env bash

# load our environment variables
source .env

# run our sidecar container
podman run --detach \
  --hostname "$HOSTNAME" \
  --name "$SIDECAR_NAME"\
  --cap-add net_admin \
  --device /dev/net/tun:/dev/net/tun \
  --volume freshrss_tailscale_state:/var/lib/tailscale \
  --env-file .ts.env \
  ghcr.io/tailscale/tailscale:latest

# run our FreshRSS service
podman run --detach \
  --restart unless-stopped \
  --log-opt max-size=10m \
  --network container:"$SIDECAR_NAME" \
  -e TZ=America/New_York \
  -e 'CRON_MIN=1,31' \
  -v freshrss_data:/var/www/FreshRSS/data \
  -v freshrss_extensions:/var/www/FreshRSS/extensions \
  --name "$CONTAINER_NAME" \
  docker.io/freshrss/freshrss

Make the file executable by running chmod +x run, the run the script with ./run. Or you can call bash run and not need to run the chmod command.

Shortly, we should see a “machine” with the tag:container under our Tailscale Dashboard’s machine tab called “rss”.

Navigate a web browser to http://rss.${tailnet-name}.ts.net (eg, http://rss.yak-bebop.ts.net) from your tailnet connected computer and you should see the FreshRSS welcome page! Because we don’t have https certificates working anyway, you could use http://rss to go to the same site, but the short names don’t work with HTTPS certs so stick with the longer version.

You could be done at this point, but Tailscale Serve ensures you can work with many different container ports and HTTPS Certs will make your experience much better in a modern browser.

Tailscale Serve Config

This is the last configuration we need, Tailscale Serve. Tailscale won’t provision a TLS cert for us without this. And the config will tell tailscale what port to proxy to, so we don’t need to type that into our browser either. We didn’t need a port number with FreshRSS because it exposes itself on the correct port, port 80. But for most services we’ll want this set up. And 80 is the wrong port for https (that’s port 443). Make a config directory with ts.json in it, to mount to our tailscale sidecar container. Under the Proxy key is where you would set the port for your service; FreshRSS is on port 80.

{
  "TCP": {
    "443": {
      "HTTPS": true
    }
  },
  "Web": {
    "${TS_CERT_DOMAIN}:443": {
      "Handlers": {
        "/": {
          "Proxy": "http://127.0.0.1:80"
        }
      }
    }
  },
  "AllowFunnel": {
    "${TS_CERT_DOMAIN}:443": false
  }
}

Since this serve config tells Tailscale to generate a https certificate for us, let’s turn that on in Tailscale before we use it.

Tailscale HTTPS Certificates

In your Tailscale Dashboard, go to the DNS tab and scroll to the bottom. Click Enable HTTPS and go through any of its prompts. Done! Now let’s return to the terminal and restart these containers. The official docs are easy to follow for this step.

Restart with https

Our service container, freshrss, depends on the tailscale sidecar to function. That means we can’t stop tailscale until our freshrss container has stopped. Let’s write a small script to help us always stop the containers in the correct order. Call it stop to again mimic the podman cli.

#!/usr/bin/env bash

# reuse our envvars to get the container names
source .env

podman stop "$CONTAINER_NAME"
podman stop "$SIDECAR_NAME"

Then, we can also make a rm script to fully remove the containers. This could be one script, but I like breaking it into two for a few reasons. One, I might not always want to fully remove the containers. Two, it mimics the podman cli which has pretty good ergonomics for all its power.

#!/usr/bin/env bash

source .env

podman rm "$CONTAINER_NAME"
podman rm "$SIDECAR_NAME"

We need to add a line to our run script to mount the tailscale serve config, making the full file:

#!/usr/bin/env bash

source .env

podman run --detach \
  --hostname "$HOSTNAME" \
  --name "$SIDECAR_NAME"\
  --cap-add net_admin \
  --device /dev/net/tun:/dev/net/tun \
  --volume freshrss_tailscale_state:/var/lib/tailscale \
  --volume "$(pwd)/config":/config \
  --env-file .ts.env \
  ghcr.io/tailscale/tailscale:latest

podman run --detach \
  --restart unless-stopped \
  --log-opt max-size=10m \
  --network container:"$SIDECAR_NAME" \
  -e TZ=America/New_York \
  -e 'CRON_MIN=1,31' \
  -v freshrss_data:/var/www/FreshRSS/data \
  -v freshrss_extensions:/var/www/FreshRSS/extensions \
  --name "$CONTAINER_NAME" \
  docker.io/freshrss/freshrss

After all that, run the stop script, then the rm script, then the run script again. You should see everything come back up and work, with any state/login/rss feeds you already setup in FreshRSS!

Troubleshooting

Deleting the Tailscale state volume

If we delete the “freshrss_tailscale_state” volume, tailscale won’t be able to reuse the same hostname. Instead you’ll end up seeing “rss-1”, “rss-2”, etc. in your list of tailscale machines. To rememdy this, just use the tailscale dashboard to remove those machines, delete the “freshrss_tailscale_state” volume again and rerun the run script to reset the tailscale hostname

Tailscale isn’t showing my machine or it isn’t connected

If you can’t see the rss machine, or it isn’t connected, run podman ps -a to get a list of all the containers, including stopped ones. If you can see it has exited, you can run podman logs freshrss_tailscale to see what error messages it might have thrown.

Taking it further

Now that we have the container up and running, there’s more we can do to automate! Podman has a great Quadlets feature that will run this all via systemd for us, adding a lot of resiliency. We can add more envvars to our .env file to make our scripts more configurable. We could write an up script to mimic a docker compose up style command. We could automate backing up FreshRSS by copying out the volume data somewhere. We could automate upgrades using podman rename before spinning up a new instance and inserting it where the old one was.

We could enjoy our FreshRSS instance and download a reader like Capy Reader to use it on our phone.

There are so many possibilities! I hope you enjoyed this tutorial.