Egress Control for Untrusted CI Workloads: eBPF/cgroup Filtering, DNS vs IP Allow-lists, SNI Limits and DNS Tunnelling

#Start here: the basics in plain words

A CI job is a program you did not write, running on a machine you own, with network access. Egress is everything that program sends out: package downloads, test traffic, and, when the job is malicious, your secrets. Sandboxing (see the sandbox page) decides whether the job can break out of its box. Egress control decides what the job can do from inside the box, and in practice most real CI compromises never leave the box: they simply phone home with the credentials the job was handed.

Think of a job as a hotel guest. The sandbox is the room lock. Egress control is the switchboard: the guest can call room service and the front desk, but every outside call goes through an operator who checks the number, logs it, and refuses calls to the competitor across the street.

There are five places you can stop an outbound packet, from closest to the program to furthest away. The closer the control, the more context it has (which process, which job); the further away, the harder it is for the job to tamper with it:

   job process                                                    internet
      │                                                               ▲
      │ connect()                                                     │
   ┌──▼──────────────┐  ①  cgroup eBPF: per-job yes/no at connect()   │
   │ kernel socket   │     (knows the job, no packet exists yet)      │
   └──┬──────────────┘                                                │
   ┌──▼──────────────┐  ②  netns / nftables / tc on the job's veth    │
   │ job network ns  │     or tap device (knows the VM, sees packets) │
   └──┬──────────────┘                                                │
   ┌──▼──────────────┐  ③  host firewall: drop metadata, RFC 1918,    │
   │ host kernel     │     other tenants; allow only the proxy        │
   └──┬──────────────┘                                                │
   ┌──▼──────────────┐  ④  egress proxy: hostname allow-list, SNI,    │
   │ proxy (Go)      │     DNS pinning, per-job logs                  │
   └──┬──────────────┘                                                │
   ┌──▼──────────────┐  ⑤  NAT with a dedicated IP pool per tenant ───┘
   │ egress gateway  │
   └─────────────────┘

Memory hook

"name, not number; outside, not inside." Policy should be written in hostnames (that's what dependencies are) but enforced on IPs the firewall resolved, and the enforcement point must sit where the job can't reach it: outside the VM, or in a cgroup hook the job lacks the capability to detach.

#Threat Model: What Leaves a CI Job and Why

What is this?A list of the outbound flows an attacker wants from a CI job, so you can design controls that break each one instead of "blocking the internet" and then opening it again when the first build fails.

Why it mattersInterviewers use egress to test whether you can secure a pipeline without stopping it from building. Every honest build needs the network: it downloads packages, pulls base images and pushes artefacts. A control that cannot tell a dependency download from an exfiltration is a control someone will turn off.

How it worksThree flows cover nearly every real incident:

📰

Real incident — Codecov Bash Uploader (2021). Codecov's uploader is a shell script thousands of CI pipelines fetched and ran on every build. From 31 January to 1 April 2021 an attacker, who had obtained a credential from a flaw in how Codecov built its Docker images, periodically altered the hosted script by adding a single line: a curl that sent the job's environment variables to a server the attacker controlled. The pieces: a script fetched over the network, executed with every secret the job had in its environment, on runners that could reach any host on the internet. What went wrong: nothing in the pipeline distinguished "upload coverage to codecov.io" from "upload the environment to an unknown IP". Why it was bad: the environment of a CI job is a bundle of API keys, cloud credentials and signing tokens, and hundreds of customer networks were affected; a customer only noticed because the script's checksum no longer matched the one on the download page. Lesson: a job with a hostname allow-list of codecov.io plus its registry would have had its exfiltration blocked and logged on the first run; and verify the integrity of anything you curl | bash.

📰

Real incident — Shai-Hulud npm worm (2025). In September 2025 a self-replicating payload spread through hundreds of npm packages. When installed inside a CI job or developer machine it harvested npm tokens, GitHub tokens and cloud credentials (it also queried the cloud metadata endpoints), then exfiltrated them two ways: by pushing a GitHub Actions workflow into every repository the stolen GitHub token could reach, which sent that repository's secrets to a single webhook.site URL, and by creating a public repository containing the dump. It used the stolen npm token to republish itself into the victim's own packages. The pieces: npm install runs package lifecycle scripts, so any dependency can run code; CI jobs hold tokens for other systems. What went wrong: the job could reach webhook.site, the metadata service and the npm and GitHub APIs with write scope. Why it was bad: each infected maintainer became a new publisher of the worm. Lesson: deny-by-default egress blocks the webhook.site channel and the metadata lookup outright; the GitHub and npm API channels are legitimate destinations, which is why egress control is paired with short-lived, narrowly scoped tokens (see OIDC in CI) rather than used alone.

🎯

On the job. When a team says "we allow-list egress", ask three questions. Where is it enforced, and can root inside the job turn it off? Is the list written in hostnames, and who resolved them? What happens to DNS: does a query for evil.example leave the building? If the answers are "iptables inside the container", "IP ranges we copied from a doc" and "we never thought about DNS", the control stops a careless script, not an attacker.

#Security angle

#Kernel-Level Enforcement: cgroups v2 and eBPF

What is this?Attaching a small eBPF program to the cgroup that contains a job, so the kernel asks that program before letting any process in the job open a connection or send a packet.

Why it mattersIt is the only enforcement point that knows which job is talking, at the moment of connect(), before a single packet exists, and it is identical on an AMD EPYC and an AmpereOne node because it is the same kernel code. iptables inside the container can be flushed by root inside the container; a cgroup program attached by the host cannot be removed by a process that lacks the capability to touch BPF.

How it works

The pieces you need to knowcgroups v2 is one tree under /sys/fs/cgroup. Each container, systemd scope or Kubernetes pod is a directory in that tree, and every process belongs to exactly one directory. eBPF programs can be attached to a directory, and they apply to every process in it and in its sub-directories. The kernel offers several hook points for network programs on a cgroup:

The decision in the program is deliberately tiny: a map lookup keyed by destination IP and port, plus hard-coded denies for link-local and private ranges that no DNS answer may override. Written in C and loaded from Go with the cilium/ebpf library, the core of a connect4 program looks like this (defensive code, it only ever says no):

SEC("cgroup/connect4")
int job_connect4(struct bpf_sock_addr *ctx) {
    __u32 dst = bpf_ntohl(ctx->user_ip4);
    if ((dst & 0xffff0000) == 0xa9fe0000) return 0;   // 169.254.0.0/16 ← metadata, always denied
    if ((dst & 0xff000000) == 0x0a000000) return 0;   // 10.0.0.0/8  ← internal, always denied
    struct allow_key k = { .ip = ctx->user_ip4, .port = ctx->user_port };
    if (bpf_map_lookup_elem(&allowed, &k)) return 1;  // pinned by the resolver ← the allow-list
    log_deny(ctx);                                     // ring buffer entry, read by the host agent
    return 0;                                          // EPERM to the caller
}

In practiceChecking what is attached where, from the host:

$ bpftool cgroup tree /sys/fs/cgroup/jobs
CgroupPath
ID       AttachType      AttachFlags     Name
/sys/fs/cgroup/jobs
812      cgroup_inet4_connect  multi     platform_egress        ← platform-wide floor, applies to every job
/sys/fs/cgroup/jobs/job-812
901      cgroup_inet4_connect  multi     job_connect4           ← this job's allow-list
902      cgroup_inet6_connect  multi     job_connect6           ← don't forget IPv6
903      cgroup_udp4_sendmsg   multi     job_sendmsg4           ← DNS goes only to the pinned resolver

$ cat /sys/fs/cgroup/jobs/job-812/cgroup.procs | head -3
48213   ← runner
48260   ← bash step
48301   ← buildkitd

And what the job sees when its dependency probes the metadata service:

$ curl -m 2 http://169.254.169.254/latest/meta-data/
curl: (7) Failed to connect to 169.254.169.254 port 80: Operation not permitted   ← EPERM from connect(), no packet sent

🎯

On the job. Review the attach step, not the program. Is the program attached before the job's first command? Is it on the job's own cgroup (not on a shared parent only, or every tenant shares one list)? Does the job's user lack CAP_BPF, CAP_SYS_ADMIN and CAP_NET_ADMIN, so it cannot detach or replace it? Is there a connect6 twin, and does the host have IPv6 at all? Most "the eBPF firewall was bypassed" stories are one of these four.

#Security angle

#Sandboxed Runner Networking: gVisor Netstack vs Apple Virtualization

What is this?Where the network stack lives changes where your enforcement point can live. Three runner shapes are common on a mixed fleet: ordinary runc containers, gVisor sandboxes on Linux (EPYC, AmpereOne), and macOS virtual machines made with Apple's Virtualization framework (vz) on Apple Silicon.

Why it mattersA control that works for runc (a cgroup hook on the job's sockets) silently does nothing for a gVisor sandbox, and does not exist at all on macOS. Interviewers who run heterogeneous fleets ask this precisely because candidates assume Linux everywhere.

How it works

• runc — The job's processes create sockets in the host kernel. Cgroup hooks see every connect(). The pod's veth carries its packets, so nftables and tc on the host side of the veth see them too. Both layers ① and ② are available.
• gVisor — The application's TCP/IP runs inside the Sentry, gVisor's user-space kernel written in Go (its stack is called netstack). The Sentry cannot open host sockets; it is given a single AF_PACKET socket on a veth and writes raw Ethernet frames to it. Consequences: a connect4 hook on the Sentry's cgroup sees no connections, because the job's connect() never reaches the host kernel. Enforcement moves to the packets: nftables or tc on the host end of the veth, or a cgroup_skb program, which does see the frames the Sentry emits. Host TCP/IP bugs are unreachable from the job, which is a security win, but --network=host (gVisor's "hostinet" mode) hands sockets back to the host kernel and gives that up.
• Apple vz on macOS — The guest's network device is attached one of three ways. VZNATNetworkDeviceAttachment shares the host's network through Apple's NAT, convenient but the host sees only translated traffic and filtering it with pf is awkward. VZBridgedNetworkDeviceAttachment puts the guest on the LAN, the worst option for untrusted code. VZFileHandleNetworkDeviceAttachment hands the virtual NIC to a socket your own host process owns, so every Ethernet frame the guest sends passes through your code first: you can run a user-space stack (netstack again, in Go) that applies the same hostname allow-list and proxying as the Linux fleet. The Apple Silicon licence limits a Mac to two macOS guests, so density is low, but the enforcement story is the cleanest of the three because the "firewall" is a process the guest cannot see.

In practiceThe gVisor runtime flags that decide this, in containerd's runsc config:

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runsc.options]
  TypeUrl = "io.containerd.runsc.v1.options"
  ConfigPath = "/etc/containerd/runsc.toml"

# /etc/containerd/runsc.toml
[runsc_config]
  network = "sandbox"        ← netstack in the Sentry (default); "host" would bypass it
  platform = "systrap"

And a host-side rule set that treats the veth as the enforcement point, because the cgroup hook cannot see gVisor's connections:

table inet jobs {
  chain fwd {
    type filter hook forward priority 0; policy drop;
    iifname "veth-job812" ip daddr 169.254.0.0/16 counter drop          ← metadata
    iifname "veth-job812" ip daddr { 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 } counter drop
    iifname "veth-job812" ip daddr 10.255.0.10 udp dport 53 accept       ← pinned resolver only
    iifname "veth-job812" ip daddr 10.255.0.11 tcp dport 3128 accept     ← egress proxy only
  }
}

🎯

On the job. On a gVisor node, run bpftool cgroup tree and look for a connect4 program on the sandbox's cgroup with a non-zero run count. A zero count on a busy job is not "the job made no connections"; it means the hook is in the wrong layer. Move the policy to the veth or the proxy.

#Security angle

#DNS vs IP Allow-lists

What is this?Whether your policy says "allow 140.82.112.0/20" or "allow api.github.com", and, if the latter, who turns the name into an address.

Why it mattersDependencies are named by hostname and served by content delivery networks (CDNs) whose addresses change hourly. GitHub release assets come from Azure blob storage, npm from Cloudflare, PyPI from Fastly; the IP sets are large, shared with thousands of other customers, and published only as whole provider ranges. An IP allow-list is either stale (builds break) or so wide that it allows the attacker's site on the same CDN.

How it works

This is how Cilium's toFQDNs policy works in Kubernetes (its DNS proxy records answers for names that match and programs the allowed IPs into the datapath), and how GitHub's hosted-runner egress firewall expands its default wildcard entries from the /meta API rather than letting users write patterns.

In practiceA Cilium policy for a job pod: the toFQDNs rule needs a matching DNS rule, or the proxy never sees the answers it must learn from.

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata: { name: job-812-egress, namespace: ci-jobs }
spec:
  endpointSelector: { matchLabels: { job: "812" } }
  egress:
  - toEndpoints:
    - matchLabels: { k8s:io.kubernetes.pod.namespace: kube-system, k8s-app: kube-dns }
    toPorts:
    - ports: [{ port: "53", protocol: UDP }]
      rules: { dns: [{ matchPattern: "*" }] }        ← DNS is inspected here; answers feed the allow-list
  - toFQDNs:
    - matchName: "registry.npmjs.org"
    - matchName: "api.github.com"
    - matchPattern: "*.githubusercontent.com"        ← one label, so evil.githubusercontent.com would match: keep patterns tight
    toPorts: [{ ports: [{ port: "443", protocol: TCP }] }]

What the datapath holds after the job's first npm ci, from the Cilium agent:

$ cilium fqdn cache list | grep npmjs
ci-jobs/job-812   registry.npmjs.org.   104.16.1.35,104.16.0.35   TTL 300   ← pinned from the answer the proxy saw

📰

Real incident — DNS rebinding used against local services (2022). Tailscale's Windows client exposed a local control API on a TCP port with no check of the HTTP Host header (TS-2022-004, CVE-2022-41924). A web page could point a domain it controlled at the victim's own machine after the browser had resolved it once, then issue requests to that local API, including changing the coordination server to an attacker's. The pieces: a service that trusted "requests arrive from localhost" as proof of who sent them; DNS answers that can change between the check and the use. What went wrong: identity was inferred from an address rather than checked. Why it matters here: an IP allow-list is the same mistake in reverse. If an allowed name resolves to an internal address, a pinning firewall will happily allow the internal address, which is why the private and link-local floors are enforced before any DNS-derived entry. Lesson: never wildcard a name that strangers can register under (*.github.io, *.cloudfront.net, *.ngrok.app), and never let a DNS answer open an internal range.

🎯

On the job. Pull the allow-list and the resolver config. Can a job reach 8.8.8.8:53, or any host on 853 (DNS over TLS), or a public DNS-over-HTTPS resolver on 443? If so the job can resolve names the firewall never saw, connect to the IP, and the pinning layer is bypassed; only the proxy layer can save you then. The fix is sendmsg4 to force DNS to one address, plus a proxy that is the only route out.

#Security angle

#L7 Inspection and the SNI Boundary

What is this?Running the job's TLS connections through a proxy that reads the destination hostname and enforces the allow-list on it, and understanding the three ways that hostname can lie.

Why it mattersThe hostname is the policy language. IP pinning gets you "an address that some allowed name once resolved to"; the proxy gets you "this specific connection is to api.github.com". But the two common ways of learning the hostname, the HTTP CONNECT line and the TLS SNI, are both claims the client makes, and the newest TLS encrypts one of them.

How it works

Two proxy shapes exist. An explicit proxy is configured in the job (HTTPS_PROXY=http://10.255.0.11:3128); clients send CONNECT api.github.com:443 and the proxy opens that connection. A transparent proxy receives the job's TCP connections via nftables redirection (tproxy) without the job knowing, peeks at the first bytes, reads the SNI from the TLS ClientHello, and decides. Jobs ignore proxy variables (Go binaries compiled without HTTP_PROXY support, hard-coded clients), so production systems do both: explicit for clients that cooperate, transparent as the catch-all, and the host firewall allows nothing else.

In practiceThe heart of a transparent proxy in Go: peek the ClientHello, decide on the SNI, dial the name, splice. The standard library parses the hello for you if you hand it a connection that replays the bytes you peeked.

func handle(c net.Conn) {
    peek := &replayConn{Conn: c}                       // records bytes so the real server still sees a full ClientHello
    var sni string
    tls.Server(peek, &tls.Config{GetConfigForClient: func(h *tls.ClientHelloInfo) (*tls.Config, error) {
        sni = h.ServerName; return nil, errStop          // we only wanted the name
    }}).Handshake()
    if !policy.Allows(jobOf(c), sni, 443) {              // job identity from the source address → cgroup/VM map
        audit(jobOf(c), sni, "deny"); c.Close(); return
    }
    up, err := dialPinned(sni, 443)                      // ← resolve the NAME ourselves; never trust the client's dst IP
    if err != nil { c.Close(); return }
    audit(jobOf(c), sni, "allow")
    splice(peek.Replay(), up)                            // hand the original bytes on, then copy both ways
}

The deny line in the proxy log, which is what an investigator will search for:

2026-10-24T09:14:02Z job=812 tenant=acme src=10.42.7.3 sni=webhook.site port=443 decision=deny rule=default-deny comm=node   ← one line per connection, with the job

🎯

On the job. Three checks for any "SNI firewall". Does it dial the SNI name or the client's IP? Does its allow-list contain any bare CDN wildcard (*.cloudfront.net, *.azureedge.net, *.githubusercontent.com is borderline)? What does it do with a ClientHello that carries the encrypted_client_hello extension: pass, drop, or terminate? "Pass" means the policy is advisory for any site behind an ECH-enabled CDN.

#Security angle

#DNS Tunnelling: Mechanics and Defences

What is this?Encoding data in DNS queries so that an attacker's name server receives it, and receiving instructions back in DNS answers. It works through a default-deny egress firewall because the firewall allows DNS to the internal resolver, and the resolver talks to the internet.

Why it mattersThis is the channel that survives everything above if you forget the resolver. A job whose only open port is 53 to 10.255.0.10 can still move a stolen OIDC token off the machine in a few hundred queries.

How it works

The pieces you need to knowA recursive resolver that does not know a name asks the authoritative server for that zone, wherever on the internet it is. So any name under a domain the attacker owns is, in effect, a message delivered to the attacker's server by your resolver, and the answer is a message back. The firewall allowed "DNS to the internal resolver"; it never saw the second hop. Tools such as iodine and dnscat2 package this into a general-purpose channel; from a defender's seat the details of the encoding matter far less than the shape of the traffic, which is distinctive: many queries, to one parent domain, with long labels that look like random text, often for record types a build never uses (TXT, NULL, CNAME) and with a high rate of "no such name" answers.

In practiceAn Unbound configuration for the per-job resolver that only knows the allow-list, and the query log line an investigator would look for:

server:
  interface: 10.255.0.10
  access-control: 10.42.0.0/16 allow          ← job networks only
  local-zone: "." always_nxdomain             ← default: every name does not exist
  local-zone: "npmjs.org." transparent        ← allow-listed zones are recursed upstream
  local-zone: "github.com." transparent
  local-zone: "githubusercontent.com." transparent
  log-queries: yes
  ratelimit: 200                              ← per zone, queries per second

[1761297812] unbound[412:0] info: job=812 10.42.7.3 x7k2m9q4p1z8v3c6b5n0.c2.example. TXT IN   ← long random label, TXT, unknown zone: NXDOMAIN locally, alert
[1761297812] unbound[412:0] info: job=812 10.42.7.3 registry.npmjs.org. A IN                     ← normal

📰

Real incident — Shai-Hulud, the channel that failed (2025). The worm's primary exfiltration endpoint was a free webhook.site URL. Under the free plan that endpoint could receive only 100 callbacks before the platform deactivated it for excessive activity, so most victims' secrets never arrived there, although they were still exposed in workflow logs and in the public repository the worm created. The lesson is not that attackers are sloppy; it is that exfiltration channels have capacity and shape, and that a defender who meters the channels a job is allowed (how many connections, how many queries, to how many distinct names) catches even a channel they did not think to block.

🎯

On the job. Point dig +short TXT test.$(date +%s).example at the job resolver from inside a job. If it returns an answer or a SERVFAIL after a delay, the resolver recursed to the internet for an unknown zone and the tunnel is open. The correct result is an immediate NXDOMAIN and a log line with the job ID.

#Security angle

#BuildKit and Multi-Stage Pipeline Confinement

What is this?Making the container-image build itself, which runs arbitrary RUN instructions from a tenant's Dockerfile, obey the same egress rules as the rest of the job, without losing remote cache.

Why it mattersRUN is a shell with the network; a Dockerfile is a program. If the build daemon is more privileged than the job (it often is), a RUN step is the easiest place to reach things the job shouldn't.

How it works

In practiceA Dockerfile that fetches online and builds offline, and the daemon flags that keep it honest:

# syntax=docker/dockerfile:1.7
FROM golang:1.23 AS deps
ENV GOPROXY=https://goproxy.internal.example       ← allow-listed mirror; direct module fetches are denied by the proxy
COPY go.mod go.sum ./
RUN go mod download                                  ← the only step that needs the network

FROM deps AS build
COPY . .
RUN --network=none go build -trimpath -o /out/app ./cmd/app   ← compile and test with no network at all
RUN --network=none go test ./...

$ buildkitd --oci-worker-net=cni --rootless \
    --oci-worker-no-process-sandbox=false          ← keep the process sandbox
# note what is absent: no --allow-insecure-entitlement network.host, so `--allow network.host` is refused
$ buildctl build --frontend dockerfile.v0 --local context=. --local dockerfile=. \
    --import-cache type=registry,ref=registry.internal.example/acme/app:cache   ← cache import is done by the daemon, RUN stays offline
    --allow network.host
error: entitlement network.host is not allowed   ← the daemon refuses

🎯

On the job. grep -rn "network.host\|security.insecure" .buildkit/ buildkitd.toml on the runner image, and look at how buildkitd is started. If a tenant can reach a daemon socket that belongs to another process, the daemon's privileges, not the job's, define the blast radius.

#Security angle

#Production Architecture and In-Practice Telemetry

What is this?How the layers combine on a real fleet: Linux pods under gVisor on EPYC and AmpereOne racks, macOS guests on Apple Silicon, GitHub Actions and Buildkite jobs arriving with single-use tokens.

Why it mattersThe system-design question is rarely "name a control"; it is "put them in order, say where each runs, and say what you'd see when one fails".

How it works

Who owns the policy matters as much as where it runs. GitHub's hosted-runner firewall, in technical preview during 2026, reads .github/egress-firewall.yaml from the branch being built, runs outside the runner VM, offers a log mode and an enforce mode, and terminates TLS with a per-run certificate. That is the shape to copy for self-hosted platforms: policy as code in the tenant's repository, enforcement in infrastructure the job cannot reach, log mode first so the allow-list is built from observed traffic rather than guesses.

In practiceA tenant policy file and the run summary the job's author sees when a dependency misbehaves:

# .github/egress-firewall.yaml  (job needs permissions: contents: read, or the file is silently not loaded)
mode: enforce
allow:
  - api.github.com
  - registry.npmjs.org
  - goproxy.internal.example

Egress firewall summary — job 812 (enforce)
  allowed  registry.npmjs.org:443        142 connections
  allowed  api.github.com:443              6 connections
  DENIED   webhook.site:443                3 attempts   binary=node   rule=default-deny   ← this is your incident
  DENIED   169.254.169.254:80              1 attempt    binary=node   rule=platform-floor
  DNS      NXDOMAIN (not allow-listed)    2,114 queries  parent=c2.example               ← and this

🎯

On the job. Before switching a tenant from log to enforce, diff a week of log-mode destinations against the proposed allow-list and look for three things: cloud SDK endpoints that only appear on some runs, anything on UDP 443, and names with more than a handful of distinct subdomains. Those are, respectively, the builds you are about to break, the QUIC fallback you need to force, and the jobs to investigate before you give them an allow-list at all.

#Security angle