WireGuard and Tailscale for CI/CD and Hybrid Infrastructure Security

#Start here: the basics in plain words

A VPN (virtual private network) makes two machines on different networks behave as if they were plugged into the same switch, by wrapping each packet in encryption and sending it across the internet. WireGuard is a small, modern VPN protocol built into the Linux kernel. Tailscale is a product that uses WireGuard for the actual traffic and adds the part WireGuard deliberately leaves out: finding the other machines, handing out keys, getting through firewalls, and deciding who may talk to whom.

Why does this land in security interviews about CI? Because the easiest way to let a build job reach an internal artefact store or a bare-metal rack is to put the runner on the company's overlay network. At that moment, code from a pull request is standing on your internal network, and the question becomes how little of it the job can see.

   control plane (Tailscale coordination server)            holds: public keys, addresses, policy
          ▲   ▲                                              never holds: private keys, your packets
   keys,  │   │  keys,
   policy │   │  policy
   ┌──────┴┐ ┌┴──────┐      WireGuard, directly (UDP) ──────────────┐
   │ CI    │ │ artif-│ ◄──────────────────────────────────────────► │  data plane
   │ runner│ │ acts  │      or via DERP relay (ciphertext only) ───┘
   └───────┘ └───────┘

Memory hook

"the drop box and the mesh." The coordination server is a drop box for public keys and rules; it can lie about who exists, never read what they say. The traffic is a mesh of WireGuard tunnels between machines that each enforce the rules locally. Every Tailscale security question is about one of the two.

#WireGuard: The Cryptographic Data Plane

What is this?A VPN protocol with one fixed set of modern primitives, a one-round-trip handshake, and a routing rule tied to public keys. It has no cipher negotiation, no certificates and no concept of users. The paper (WireGuard, NDSS 2017) is short enough to read before an interview; the Linux implementation shipped in kernel 5.6 (2020).

Why it mattersInterviewers want to hear that you can separate what the cryptography guarantees from what the surrounding product does, and that you know why a 4,000-line kernel module is considered safer than an IPsec stack ten times its size.

How it works

The pieces you need to knowEach peer has a static Curve25519 key pair; the public key is 32 bytes, 44 characters in Base64, and is the peer's identity. The handshake is the Noise protocol framework's IK pattern with an optional pre-shared key mixed in afterwards, written Noise_IKpsk2_25519_ChaChaPoly_BLAKE2s. "IK" means the initiator already Knows the responder's static key and sends its own Identity encrypted in the first message. Symmetric traffic is ChaCha20-Poly1305 (an authenticated cipher), hashing is BLAKE2s, key derivation is HKDF.

Cryptokey routing is the part most people miss. The interface keeps a table from each peer's public key to the IP ranges that peer is allowed to use. On send, the destination address picks the key to encrypt with. On receive, after decryption, the inner packet's source address must be inside the allowed set for the key that decrypted it, or the packet is dropped. So a packet that arrives on wg0 from 100.101.5.7 is cryptographically guaranteed to have come from the peer that owns that address. Firewall rules written against WireGuard addresses are therefore rules against identities, which is the foundation Tailscale's ACLs stand on.

In practiceWhat the table looks like on a Linux node (keys and endpoints abridged), with the lines that matter for security:

$ sudo wg show wg0
interface: wg0
  public key: HIgo…8ykw
  listening port: 41414

peer: TrMv…WXX0
  endpoint: 203.0.113.9:51820                 ← where it last spoke from; roams freely, identity is the key, not the address
  allowed ips: 100.101.5.7/32                 ← this key may only ever be this address; anything else is dropped after decryption
  latest handshake: 47 seconds ago            ← a new handshake at most every 120 s is normal; "never" means mac1 or the key is wrong
  transfer: 1.21 GiB received, 88.4 MiB sent
  persistent keepalive: every 25 seconds      ← keeps the NAT mapping alive for a peer behind a home or cloud NAT

peer: gN65…z6EA
  allowed ips: 10.10.0.0/16                   ← a subnet router: this peer may *source* packets from a whole range

Kernel versus user spaceThe Linux module is the fast path, with no copies to user space and multi-queue parallelism across the cores of an EPYC or AmpereOne node. wireguard-go is the same protocol in Go, which is what runs on macOS (including inside Apple Virtualization guests), on Windows, and inside every Tailscale client; it is slower per packet but Tailscale's segmentation-offload work (sending many packets per system call) narrowed the gap to within reach of 10 Gbit/s on modern hardware. gVisor's netstack can carry WireGuard as just another UDP flow; the sandbox sees a UDP socket, the host sees ciphertext.

🎯

On the job. A WireGuard review has four questions. Who generated the keys and where do the private halves live (a file readable only by root on the node, never in a CI secret store)? Are the allowed IPs /32 per peer, or did someone set 0.0.0.0/0 on a peer that should only be a client? Is there a pre-shared key for post-quantum hedging on long-lived site-to-site links? And what distributes the keys, because WireGuard itself refuses to answer that question.

#Security angle

#Tailscale: The Control Plane

What is this?A coordination server that holds every machine's public key, assigns each a stable address in 100.64.0.0/10, computes a per-machine "netmap" (the peers it may see and the packet filter it must apply), and hands out a map of relay servers. The data plane stays WireGuard between the machines.

Why it mattersThe cleanest interview answer separates what a compromised Tailscale could do (lie about peers, push a bad policy) from what it could never do (read traffic). Tailnet lock exists to shrink the first list.

How it works

DERP relaysEach client measures latency to the relays in its DERP map, picks a home relay and reports it; peers that cannot punch through NAT send WireGuard packets through that relay over HTTPS. Because the relay only ever sees ciphertext, the trust question about DERP is availability and metadata (who talks to whom, how much), not confidentiality. In May 2025 Tailscale rotated DERP mesh secrets after finding they were compared in non-constant time (TS-2025-003): a timing side channel against the relays' mutual authentication, not against user traffic, which is a precise example of the control-plane/data-plane split.

PolicyThe tailnet policy file is HuJSON (JSON with comments). If an acls or grants section exists, everything not listed is denied, and rules are directional: allowing A to reach B does not allow B to reach A. Rules are enforced locally on the destination node from its netmap, so the coordination server being briefly unreachable does not open anything. ACLs do not touch a machine's local LAN access, which matters for runners that sit on a rack network as well as on the tailnet.

In practiceA policy fragment for a CI fleet, with the lines interviewers probe:

{
  "tagOwners": {
    "tag:ci-runner":  ["group:platform"],          // ← who may *mint* this identity; owners can act as the tag
    "tag:artifacts":  ["group:platform"],
    "tag:prod-db":    ["group:dba"]
  },
  "acls": [
    { "action": "accept", "src": ["tag:ci-runner"], "dst": ["tag:artifacts:443"] },   // ← runners reach one service, one port
    { "action": "accept", "src": ["group:platform"], "dst": ["tag:ci-runner:22"] }    // ← humans reach runners, never the reverse
    // nothing lets tag:ci-runner reach tag:prod-db, or any user's laptop: default deny does the rest
  ],
  "ssh": [
    { "action": "check", "src": ["group:platform"], "dst": ["tag:ci-runner"], "users": ["ci"] }  // ← re-auth before SSH, non-root user
  ]
}

And what a runner actually sees, which is the test of the policy, not the file:

$ tailscale status
100.101.5.7   ci-runner-812   tag:ci-runner   linux   -
100.98.3.20   artifacts-1     tag:artifacts   linux   active; direct 203.0.113.40:41641   ← the only peer in the netmap
$ tailscale ping prod-db-1
no matching peer                                                                           ← not "blocked": the runner never learned it exists

📰

Real incident — Tailscale Windows client, DNS rebinding to the local API (2022). The pieces: tailscaled on Windows exposed a local control API on a TCP port so the GUI could talk to it, and it trusted that anything arriving on that port came from the local user; a browser resolves a hostname once, then trusts later connections to the same name even if the attacker's DNS now points it at 127.0.0.1. What went wrong: the local API checked neither the HTTP Host header nor any token, so a malicious web page could rebind its own domain to the victim's machine and issue API calls, including changing the coordination server to one the attacker ran (TS-2022-004, CVE-2022-41924, critical). Why it was bad: a malicious coordination server can push configuration and, on Windows, executables and shares, so this was remote code execution from a web page. A sibling bug (TS-2022-005, CVE-2022-41925) let a page reach the peer API on all platforms and read environment variables, which on CI machines often contain auth keys. Both fixed in 1.32.3. Lessons: "it's only listening on localhost" is not authentication; and a CI runner's environment should not contain a reusable auth key in the first place, which is where the next section goes.

🎯

On the job. Open the admin console's machine list filtered to tag:ci-runner. Every entry should be ephemeral, recently created and tagged, with key expiry showing the tag default. A runner that is not ephemeral, or that has a user's name instead of a tag, was joined by hand with someone's login and is now that person on your network.

#Security angle

#Tags, Ephemeral Nodes and Identity Without Static Secrets

What is this?The three mechanisms that turn "a runner on the VPN" into "a disposable machine identity that exists for one job": tags, ephemeral registration, and OIDC workload identity federation instead of auth keys.

Why it mattersMost real damage in this area comes from a long-lived auth key sitting in CI secrets. Whoever steals it can add their own machine to the tailnet, with the runner's tag and the runner's access, from anywhere, for as long as the key lives.

How it works

TagsA device has either a user identity or a tag identity, never both: applying a tag removes the user. Tags are declared in tagOwners, and only the listed owners (users, groups or other tags) may apply them, except that Owners, Admins and Network admins may apply any tag. Two defaults matter for CI: tagged devices have key expiry disabled when first authenticated (a tagged machine never gets kicked off for a stale key), and tagged devices may only SSH to other tagged devices. The first default is why ephemeral removal, not expiry, is the lifetime control for runners.

Ephemeral nodesCreated with an ephemeral auth key, or by running tailscaled with --state=mem: (v1.22+), which keeps all state in memory and marks the node ephemeral with no key needed for that flag alone. The control plane removes an ephemeral node 30 to 60 minutes after it goes quiet, or immediately on tailscale logout, which --state=mem: performs automatically on exit (v1.30+). Each new ephemeral node gets a new address, so an address is never recycled onto a different job within that window. A node that stays four hours stops being treated as ephemeral for billing, a hint that a "runner" alive that long is a pet, not cattle.

Workload identity federationInstead of an auth key, the runner presents the signed OIDC token its CI platform already gives it (GitHub Actions, Buildkite, GitLab, or a cloud provider's instance identity). An admin creates a federated identity in Tailscale naming the issuer to trust, a sub pattern (for GitHub, repository and workflow; for AWS, an IAM principal ARN), optional extra claim rules, the tags the resulting node may carry, and the API scopes it gets. Tailscale fetches the issuer's public keys, verifies the token's signature, issuer, audience (api.tailscale.com/<client ID>) and expiry, applies the matching rules, and hands back a short-lived credential. Since v1.90.1 tailscale up accepts the client ID and token directly to register a node; since v1.94 --audience lets the client fetch the platform token itself on GitHub Actions, AWS and Google Cloud. Nodes registered this way default to ephemeral.

In practiceThe workflow step and the resulting node, as a reviewer sees them:

permissions:
  id-token: write          # ← lets the job mint an OIDC token; without it the action falls back to failing, not to a stored key
  contents: read
steps:
  - uses: tailscale/github-action@v4
    with:
      oauth-client-id: ${{ secrets.TS_FEDERATED_CLIENT_ID }}   # ← an identifier, not a secret that grants access by itself
      audience:        ${{ secrets.TS_AUDIENCE }}
      tags: tag:ci-runner

$ tailscale status --json | jq '.Self | {HostName, Tags, KeyExpiry, Online}'
{
  "HostName": "fv-az1234-812",
  "Tags": ["tag:ci-runner"],        ← tag identity, no user
  "KeyExpiry": "0001-01-01T00:00:00Z",   ← expiry disabled for tagged nodes; ephemeral removal is the lifetime
  "Online": true
}

📰

Real incident — one-off auth keys that weren't (2025). Tailscale's November 2025 bulletin (TS-2025-007) disclosed a race in the control plane: a single-use auth key presented by several registrations at the same moment could register several nodes instead of one. The pieces: "one-off" was enforced by checking a flag and then marking it used, two steps that could interleave under concurrency. Why it matters for CI specifically: fleets of runners register in bursts, exactly the condition that triggers the race, and a key that leaked from one job could ride along with a legitimate burst. It was fixed server-side with no user action, and an earlier bulletin that year (TS-2025-005) covered MDM-supplied auth keys being written to client logs. Lesson: an auth key is a bearer credential with all the usual bearer-credential problems; the durable fix is not a better key, it is no key, which is what the OIDC exchange gives you.

🎯

On the job. List federated identities and auth keys in the admin console. For each auth key still in use by CI, write down who can read it and when it was last rotated; for each federated identity, read the sub pattern aloud and ask whether a fork, a different workflow, or a deleted-and-recreated repository with the same name would match it.

#Security angle

#Threat Model: What Goes Wrong When CI Joins Your Tailnet

What is this?The attack paths that open the moment code from a pull request runs on a machine that is a peer on your private network.

Why it mattersConvenience is the whole pitch (no bastion, no firewall tickets), and convenience is symmetric: the runner can reach the artefact store, and whatever runs on the runner can try to reach everything else the runner's identity is allowed.

How it works

📰

Real incident — exit nodes and subnet routers accepting LAN traffic (2024). Tailscale's May 2024 bulletin (TS-2024-005) described a gap in inbound filtering: on a node acting as an exit node or subnet router, an ACL with "src": "*" (any source) could allow hosts on that node's local LAN, which are not tailnet members at all, to send traffic into the tailnet through it. The pieces: a router forwards between its LAN and the overlay in both directions; * was interpreted as "anyone", and the LAN counts as anyone. What went wrong: the policy author meant "any tailnet device" and the filter heard "any address". Why it matters for CI: a subnet router on a rack network, next to bare-metal runners, is exactly where untrusted LAN neighbours exist. Fixed in 1.66.0. Lesson: never use * as a source in a policy that includes routers; name tags and groups, and treat a subnet router as a firewall whose rules are the ACLs.

🎯

On the job. From a throwaway job, run tailscale status and tailscale netcheck, then try nc -zv -w2 <each peer address> 22 443 5432. Every peer in the status output is a peer the runner may talk to; every open port is a path a malicious dependency gets for free. The target state is one peer, one port.

#Security angle

#Isolation Patterns: tsnet, Tailnet Lock and Egress

What is this?The design choices that keep the job from inheriting the runner's network identity, and that keep the control plane from being a single point of compromise.

Why it matters"We use Tailscale" says nothing about security; "the runner process has no TUN device, the job's identity is a tsnet listener inside the agent, nodes are tailnet-locked, and internet egress goes through a proxy" does.

How it works

tsnet versus a system TUNtailscaled normally creates a TUN interface (tailscale0) that every process on the machine can use; the machine is the node. tsnet is a Go library that embeds the whole client, with a user-space network stack (gVisor's netstack), inside one binary: that process is the node, it needs no root and no TUN, and nothing else on the machine can send through it. For CI this inverts the trust: the platform's agent is on the tailnet to fetch work and upload artefacts; the job's shell is not on the tailnet at all. The threat profile of a TUN is "any process on the host gets the node's access"; the threat profile of tsnet is "a bug in this one Go process, and the node key in its memory".

Tailnet lockEvery node generates a tailnet lock key (TLK) alongside its node key. When you enable the feature (v1.46.1+), you pick between two and twenty signing nodes; a node's public node key must carry a signature from a trusted TLK before other nodes will exchange traffic with it, and the list of trusted TLKs is kept in a signed chain (the tailnet key authority) that every node verifies. The coordination server still distributes keys and signatures, but it can no longer get an unsigned node accepted: a compromised or coerced control plane cannot add a peer. It also cannot turn the lock off, because that requires one of ten disablement secrets generated at setup, which you hold. For CI fleets you pre-sign auth keys (tailscale lock sign $AUTH_KEY) or sign with the federated flow, so ephemeral runners join without a human signing each one. Caveats worth saying: nodes trust the first state they receive (check tailscale lock status on several nodes and compare), TLKs live on devices so a compromised signing laptop is a compromised authority, and removing a compromised signing key is a multi-signer ceremony (tailscale lock revoke-keys).

Egress is a separate problemThe tailnet decides what the runner can reach inside. Internet-bound traffic from the runner does not touch WireGuard unless an exit node is forced, and forcing one only moves the question to the exit node's firewall. Treat the two as different systems: tag-scoped ACLs for private destinations, a hostname-enforcing egress proxy with dedicated egress IPs for public ones, and a host firewall on the runner that allows only the tailnet peers, the proxy and the resolver. Subnet routers into cloud VPCs (AWS, GCP) are where the two meet: the router's SNAT setting decides whether the VPC flow logs see the job or the router, and the VPC's own security groups should admit the router's address only on the ports the ACLs already allow, so neither layer is trusted alone.

In practiceThe agent-side pattern in Go, and the lock state a reviewer checks:

srv := &tsnet.Server{
    Hostname:  "ci-agent-" + jobID,
    Ephemeral: true,                                   // ← gone when the process exits
    AuthKey:   os.Getenv("TS_AUTHKEY"),                // ← or, better, a token from the OIDC exchange
    Store:     &mem.Store{},                            // ← node key in memory only
}
ln, err := srv.Listen("tcp", ":443")                    // only this listener is on the tailnet; the job's processes are not

$ tailscale lock status
Tailnet lock is ENABLED.
This node is accessible under tailnet lock. Node signature:   ← if a node is "locked out", peers drop its traffic until signed
Trusted signing keys:
  tlpub:4f2e…a91c   (2 nodes)   ← two or more signers; compare this list on another node (trust on first use)
  tlpub:9b70…0d3e   (2 nodes)

🎯

On the job. ls -l /var/lib/tailscale/ on a runner image. A tailscaled.state file baked into the image means every runner from that image is the same node identity, and whoever has the image has the key. The correct finding is no state file at all: --state=mem: or tsnet with a memory store, and identity minted per job.

#Security angle