Nix, Melange/Apko and Hermetic Reproducible Builds

#Start here: the basics in plain words

A build turns source code into an artifact: a binary, a container image, a package. Most builds are not a pure function of the source. They also depend on whatever happens to be on the machine (the compiler version, a cached library, the clock, the locale) and on whatever the network returns that day. That hidden input is where supply-chain attackers live: change the machine or the network, and the "same" source silently produces a different artifact.

Two properties close that gap, and people constantly mix them up:

The relationship between hermetic and reproducible, in one picture:

                 inputs declared?        same output every time?
hermetic only         yes                       no   (e.g. embeds a build timestamp)
reproducible only     no                        yes  (happens to be stable, but trusts the host)
both                  yes                       yes  ← the goal: anyone can rebuild and verify
neither               no                        no   ← a typical Dockerfile with `apt-get install` and `curl | bash`

Memory hook

"hermetic is about inputs, reproducible is about outputs." Hermetic: nothing gets in that you didn't list. Reproducible: what comes out never changes. You need both before "rebuild it and compare" becomes a security control.

#What is this?

This page covers the tooling that makes builds a verifiable function of their inputs: Nix (a package manager and build system built around a content-addressed store and sandboxed builds), melange and apko (Chainguard's tools that build packages and container images declaratively, with no Dockerfile and no Docker daemon), and the attestation layer on top (in-toto statements, Sigstore/cosign keyless signing, SLSA levels). It also covers where the guarantees stop: fixed-output derivations that are allowed to reach the network, poisoned binary caches, and the question of who compiled your compiler.

#Why it matters

An interviewer who asks about reproducible builds is not asking whether you can spell SBOM. They want to know whether you understand that the build system is part of the trusted computing base, that a compromised build host can produce a backdoored artifact from clean source (SolarWinds, 2020), and that the only general defence is to make builds independent of the host so a second, independent builder can catch the difference. For a platform that runs other people's CI jobs, hermeticity is also what makes a build cacheable and sandboxable: if the build declares its inputs, you can pre-fetch them through a controlled proxy and run the actual compilation with no network at all.

#Hermeticity vs Reproducibility

What is this?Hermeticity is a property of the environment: the build is sealed. Reproducibility is a property of the result: it is deterministic. You can have one without the other in both directions.

Why it mattersEach property defeats a different attacker. A hermetic build defeats the attacker who changed something on the build host or on the network path. A reproducible build defeats the attacker who changed the artifact after the fact, or who compromised one builder but not another, because an independent rebuild exposes the mismatch. Only together do they give you "verify by rebuilding".

How it worksThe verification loop that both properties enable:

In practiceNix has the comparison built in. --rebuild builds a path that already exists and diffs the result:

$ nix build nixpkgs#hello --rebuild
$ nix build .#my-service --rebuild
error: derivation '/nix/store/9k2…-my-service-1.4.2.drv' may not be deterministic:
  output '/nix/store/4q7…-my-service-1.4.2' differs from '/nix/store/4q7…-my-service-1.4.2.check'   ← same recipe, different bytes

$ diffoscope /nix/store/4q7…-my-service-1.4.2 /nix/store/4q7…-my-service-1.4.2.check
├── bin/my-service
│ ├── readelf --notes
│ │ -  Build ID: 3f1c…a9                                                                           ← linker embedded a different build ID
│ │ +  Build ID: 77e0…c2
│ ├── strings
│ │ -  built 2026-10-24T09:12:41Z                                                                  ← a timestamp baked in at compile time
│ │ +  built 2026-10-24T09:13:05Z

Two rules fall out of this. A hermetic build that is not reproducible still protects you against host contamination, but you cannot use "rebuild and compare" as evidence. A reproducible build that is not hermetic (it just happens to be stable on your fleet) is fragile: the first time a host has a different /usr/bin/cc, the output changes and you will not know whether that was the host or an attacker.

📰

Real incident — SolarWinds Orion / SUNSPOT (2020). The pieces you need to know. Orion was built with MSBuild on SolarWinds' own build servers. MSBuild reads .cs source files from disk and compiles them; it trusts that what is on disk is what is in version control. What went wrong. A program the attackers named SUNSPOT ran on the build server. It polled the process list every second, and when it saw MsBuild.exe building the Orion solution it swapped one source file, InventoryManager.cs, for a backdoored copy, waited for the compile to finish, and put the original back. It checked the MD5 of the original before swapping so it only ever targeted the exact file it expected. Version control was clean the whole time. Why it's bad. The signed, released product contained the SUNBURST backdoor, shipped to roughly 18,000 organisations, and every code review, scanner and signature check passed because the source was fine and the signature was genuine. The compromise lived in the ten seconds between "read source" and "emit binary". The lesson. The build host is in the trusted computing base, so treat it like one: ephemeral, hermetic, and never the only builder. A second independent build of the same commit would have produced a different binary, and the mismatch is the detection. That is the whole argument for reproducibility as a security control, not a tidiness goal.

🎯

On the job. When you review a build pipeline, ask one question of every step: "what does this read that is not in the lockfile?" The answers (a base image by tag, curl | bash, apt-get update, $PATH on the runner, the system clock) are your list of unpinned inputs. Each is a place where the artifact can change without a source change.

#Nix Architecture: Derivations, the Store and the Sandbox

What is this?Nix models a build as a pure function. The recipe (a derivation) lists its inputs by store path, and the output lands at a store path whose name is a hash. Because inputs are named by hash, the whole dependency graph is a Merkle tree: change one byte in any input and every path that depends on it changes.

Why it mattersThis is what makes hermeticity enforceable rather than aspirational. The sandbox only mounts the store paths the derivation declares. If a build tries to read /usr/lib/libssl.so, the file does not exist inside the sandbox, so the build fails loudly instead of silently linking against the host.

#The derivation

A .drv file is the complete, serialised recipe. The fields that matter:

In practicenix derivation show prints the recipe. Abridged for a small Go service:

$ nix derivation show .#my-service
{
  "/nix/store/9k2…-my-service-1.4.2.drv": {
    "outputs": { "out": { "path": "/nix/store/4q7…-my-service-1.4.2" } },   ← output path known BEFORE building
    "inputDrvs": {
      "/nix/store/b1x…-go-1.23.4.drv": { "outputs": ["out"] },             ← the compiler is an input, by hash
      "/nix/store/c8m…-my-service-go-modules.drv": { "outputs": ["out"] }  ← vendored modules, a fixed-output derivation
    },
    "inputSrcs": [ "/nix/store/0s1…-source" ],                              ← the source tree, copied into the store
    "system": "aarch64-linux",
    "builder": "/nix/store/d4f…-bash-5.2p37/bin/bash",
    "env": {
      "CGO_ENABLED": "0",
      "GOFLAGS": "-trimpath -mod=vendor",
      "PATH": "/nix/store/b1x…-go-1.23.4/bin:/nix/store/…",                ← PATH lists store paths only; no /usr/bin
      "SOURCE_DATE_EPOCH": "315532800"                                      ← 1980-01-01, so nothing embeds "now"
    }
  }
}

Notice that the output path is already in the file. For an input-addressed derivation, Nix computes the output path from a hash of the recipe before the build runs. That is only sound if the build is deterministic, which is why non-determinism is treated as a bug in Nixpkgs, not a shrug.

#How a store path is computed

Every store path is /nix/store/<digest>-<name>. The digest is derived from a fingerprint string:

fingerprint  = "<type>:sha256:<inner-digest>:/nix/store:<name>"
digest       = nix32( compress_to_20_bytes( sha256(fingerprint) ) )      ← 32 characters of Nix's base-32 alphabet

The <type> says what the inner digest covers:

A worked example, conceptually:

1. Your service's recipe changes (new Go flag)   → the .drv text changes
2. sha256(.drv modulo FODs)                        → new inner digest
3. "output:out:sha256:<inner>:/nix/store:my-service-1.4.2"  → new fingerprint
4. sha256 → compress → nix32                       → /nix/store/j3r…-my-service-1.4.2   ← a brand new path
5. Every derivation that lists this one as an input sees a new input path, so ITS fingerprint changes too

The security consequence: a store path is a commitment to the complete recipe graph. Two machines that agree on a path agree on every byte of every input that produced it. That is the property a binary cache signs.

#Input-addressed vs content-addressed derivations

What content addressing solves: with an input-addressed cache, a rebuild of glibc with a cosmetic change means a new path and a new signed entry for every package above it; the trust question "is this the glibc I expect?" is answered only by the recipe hash. With content addressing, a path is its bytes, so two independent builders that agree on the output automatically produce the same path, which is exactly the "rebuild and compare" check expressed as a store property.

What it introduces: the mapping from derivation to output is no longer computable; it is a claim. A cache now has to sign realisations ("drv X produced output Y"), and the client must decide whose claims to trust. Non-determinism also becomes visible rather than hidden: two builders that disagree produce two different valid paths, and you need a policy for which wins. The feature is still marked experimental in the Nix manual, and treating it as production-ready in an interview is a mistake; treat it as "the direction, with open trust questions".

#The sandbox

On Linux, sandbox = true is the default. Each build runs in private PID, mount, network, IPC and UTS namespaces (the kernel's mechanisms for giving a process its own view of processes, filesystems, network, shared memory and hostname). The filesystem view is only the declared input store paths, a temporary build directory, a private /proc and /dev, and whatever sandbox-paths adds. There is no network device, so curl fails with "network unreachable", not with a surprise download.

In practiceChecking the configuration that matters on a builder:

$ nix config show | grep -E '^(sandbox|require-sigs|trusted-users|trusted-public-keys|substituters) '
require-sigs = true                                                            ← unsigned paths are refused
sandbox = true                                                                 ← Linux default; false on macOS
substituters = https://cache.nixos.org/
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
trusted-users = root                                                           ← anyone here can turn all of this off

On macOS the sandbox is off by default and, when enabled, is built on Apple's sandbox-exec profiles rather than namespaces. It is weaker (no private network namespace, so builds can still open sockets), which matters when your Apple Silicon builders under Apple's Virtualization framework (vz) are producing release artifacts: the VM is the hermetic boundary there, not Nix.

🎯

On the job. trusted-users is the setting to grep for in any Nix builder audit. The manual's own words are that adding a user there is "essentially equivalent to giving that user root access". A trusted user can pass --option sandbox false, add arbitrary substituters and trusted-public-keys on the command line, and import unsigned paths. If your CI job runs as a trusted user, every guarantee on this page is advisory.

#Declarative Distroless Packaging: melange, apko and dockerTools

What is this?A Dockerfile is an imperative script: RUN apt-get install …, RUN curl … | sh. Each RUN can do anything, read anything and fetch anything. apko and melange replace that with declarations. melange builds an .apk package (the Alpine/Wolfi package format) from a YAML pipeline, signs it, and emits per-architecture outputs. apko assembles an OCI image from a list of .apk packages and a keyring, with no RUN steps at all, and writes an SBOM at the same time because it already knows exactly what it put in.

Why it mattersThe image build is the last step before deployment and the one most often done on a shared Docker daemon with root. Removing the daemon removes a privileged process from the runner; removing RUN removes arbitrary code execution and arbitrary network access from the image-assembly step; generating the SBOM from the inputs removes the guesswork of scanning a finished filesystem.

How it works

In practiceAn apko.yaml for a Go service, and what the build prints:

contents:
  repositories:
    - https://packages.wolfi.dev/os
    - '@local /work/packages'                       # ← melange output from the previous step
  keyring:
    - https://packages.wolfi.dev/os/wolfi-signing.rsa.pub
    - /work/melange.rsa.pub                          # ← only packages signed by these keys are accepted
  packages:
    - wolfi-baselayout
    - ca-certificates-bundle
    - my-service@local                               # ← pinned to the local repo, no accidental upstream package of the same name
accounts:
  users: [{ username: app, uid: 65532 }]
  run-as: app
entrypoint:
  command: /usr/bin/my-service
archs: [x86_64, aarch64]

$ apko build apko.yaml my-service:1.4.2 my-service.tar --sbom-path sbom/
INFO building image my-service:1.4.2 for x86_64, aarch64
INFO verifying signature of my-service-1.4.2-r0.apk with /work/melange.rsa.pub        ← signature checked per package
INFO installed 6 packages                                                             ← six, not six hundred
INFO generating SBOM sbom/sbom-x86_64.spdx.json                                       ← written from the install list, not by scanning
INFO final image digest sha256:7a1f…e3 (x86_64), sha256:0c9d…b8 (aarch64)
$ apko build apko.yaml my-service:1.4.2 again.tar && sha256sum my-service.tar again.tar
7a1f…e3  my-service.tar
7a1f…e3  again.tar                                                                    ← identical: reproducible by default

Why the apko SBOM is better than a scanner's SBOM: a scanner looks at a filesystem and infers packages from file paths and metadata databases, missing statically linked code and vendored copies, and sometimes inventing matches. apko knows the package list, versions and the signing key that vouched for each one, because that list is the only way anything got into the image. The SBOM is a by-product of the build, not an audit of it.

Nixpkgs dockerTools gets to a similar place from the other direction: dockerTools.buildLayeredImage takes a Nix closure and writes an OCI image where each layer is a set of store paths. Reproducibility comes from the store; the SBOM can be derived from the closure. The difference is cultural more than technical: apko/melange speak the apk and OCI vocabulary a container team already knows, while dockerTools asks the team to adopt Nix for the whole build.

🎯

On the job. The smell in a Dockerfile review is any RUN that touches the network: apt-get, pip install, curl, go mod download. Each one is an unpinned input and a network egress during the build. The migration path is: pin digests → move fetches into a dedicated stage with --network allowed and hash verification → build with --network=none → replace the whole thing with apko or dockerTools once packages are the unit of trust.

#Cryptographic Integrity: in-toto, cosign and SLSA

What is this?A reproducible build proves what was built. Attestation proves who built it and how, in a form someone else can verify without rebuilding. in-toto defines the statement format. Sigstore's cosign signs it, usually "keyless": a short-lived certificate is issued to a workload's OIDC identity by the Fulcio certificate authority and recorded in the Rekor transparency log, so there is no long-lived signing key to steal. SLSA v1.0 grades the build platform that produced the statement.

How it worksThe shape of a provenance attestation:

In practiceVerifying an image's provenance against a specific builder identity:

$ cosign verify-attestation \
    --type slsaprovenance1 \
    --certificate-oidc-issuer https://token.actions.githubusercontent.com \
    --certificate-identity-regexp '^https://github.com/acme/release-workflows/.github/workflows/build.yml@refs/tags/v[0-9.]+$' \   ← the exact reusable workflow, not "anything in the org"
    ghcr.io/acme/my-service@sha256:7a1f…e3 | jq -r '.payload' | base64 -d | jq '.predicate.runDetails.builder.id, .predicate.buildDefinition.resolvedDependencies[0]'
Verification for ghcr.io/acme/my-service@sha256:7a1f…e3 --
  - The cosign claims were validated
  - Existence of the claims in the transparency log was verified offline        ← Rekor inclusion proof
  - The code-signing certificate was verified using trusted certificate authority certificates
"https://github.com/acme/release-workflows/.github/workflows/build.yml@refs/tags/v2.3.0"
{ "uri": "git+https://github.com/acme/my-service@refs/tags/v1.4.2", "digest": { "gitCommit": "e1b2…9f" } }   ← which commit, exactly

The line that does the security work is --certificate-identity-regexp. Without it, verification only proves "someone with a GitHub OIDC token signed this", which includes every fork on the planet.

#SLSA v1.0 Build track, and why self-hosted is the hard case

GitHub's artifact attestations give L2 by default and L3 when the build runs in a reusable workflow, because the reusable workflow separates the trusted build definition from the calling repository's steps. On self-hosted Buildkite or GitHub runners the platform is you, so L3 means you must build the separation yourself:

The failure that drops you from L3 to L1 is letting a build step call cosign attest with a credential available to the job. A malicious step can then attest to anything, and the signature is genuine.

📰

Real incident — tj-actions/changed-files (2025, CVE-2025-30066). The pieces you need to know. A GitHub Action is code pulled into your job at run time by a reference like uses: tj-actions/changed-files@v45. A tag is a movable pointer, and the runner process holds every secret the job was given, in memory. What went wrong. In March 2025 an attacker with access to the action's repository rewrote almost every version tag to point at one malicious commit. The new code dumped the runner worker's memory, found the secrets in it and printed them, encoded, into the public build log. Any pipeline that referenced the action by tag, in tens of thousands of repositories, ran the attacker's code on its next trigger. Why it's bad. No pipeline file changed and no source changed. The artifact of the pipeline (its logs, and anything built with the stolen credentials) was compromised by an unpinned build input. The lesson. An action by tag is a dependency without a lockfile. Pin to a commit SHA, prefer a vetted internal mirror, and make the provenance record the resolved digest of every action so the attestation shows what actually ran.

#Cross-Architecture Determinism

What is this?Reproducibility is defined per target: the same source and toolchain, for the same target platform, must give the same bytes. An x86-64 build on an AMD EPYC host and an arm64 build on an AmpereOne host are supposed to differ. The question is whether the x86-64 build is identical whether it was produced on EPYC, on an AmpereOne cross-compiling, or on an Apple Silicon machine inside a Linux VM.

Why it mattersA platform with mixed hardware wants to build on whatever is free and verify anywhere. Every host-dependent input breaks that, and the breaks are silent: the binary works, it just has a different hash, and now "rebuild and compare" produces false alarms until nobody looks at it.

How it worksThe usual culprits:

In practiceGo is the easy case once you know the two switches:

$ CGO_ENABLED=0 go build -trimpath -ldflags='-buildid=' -o out/my-service ./cmd/my-service
$ sha256sum out/my-service
c41e…07  out/my-service
$ go version -m out/my-service | grep -E 'vcs|CGO|trimpath'
	build	-trimpath=true                                   ← no /home/runner paths inside
	build	CGO_ENABLED=0                                    ← no host libc or linker involved
	build	vcs.revision=e1b2…9f
	build	vcs.modified=false                               ← built from a clean checkout; "true" means the hash is unrepeatable by design
# same command on an AmpereOne builder targeting GOARCH=amd64:
$ GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -trimpath -ldflags='-buildid=' -o out/my-service ./cmd/my-service && sha256sum out/my-service
c41e…07  out/my-service                                  ← identical bytes from a different host architecture

Go 1.21 was the first toolchain release whose own binaries were perfectly reproducible, and the list of things the Go team had to remove to get there (map ordering, host linker paths, cgo, timestamps, user IDs in archives, an ARM floating-point default that depended on the build host) is a good checklist for any language. For TypeScript the equivalent discipline is npm ci from a committed lockfile, a bundler with deterministic chunk naming, and no postinstall scripts that fetch binaries; for C it is a pinned toolchain plus the flags above.

🎯

On the job. Put a determinism check in the pipeline, not in a wiki: build the release artifact twice on two different runner pools (EPYC and AmpereOne, or Linux and the vz Linux guest on Apple Silicon) and fail the release if the digests differ. The first week finds your timestamps and paths; after that, a failure means something changed that you did not change.

#Failure Modes and Attack Vectors

#Fixed-output derivations: the hole in the sandbox, by design

The pieces you need to knowA hermetic sandbox has no network, but source code has to come from somewhere. Nix's answer is the fixed-output derivation (FOD): a derivation that declares outputHash, outputHashAlgo and outputHashMode up front. Because the output's hash is already known, Nix lets this one kind of derivation keep the host's network namespace. Whatever it downloads is checked against the declared hash and rejected on mismatch. This is how fetchurl, fetchFromGitHub and the Go/npm/cargo vendoring helpers work.

What can go wrong

In practiceAn egress proxy makes FOD traffic visible and bounded. The sandbox keeps the host network namespace for FODs, so routing them through a proxy is a matter of setting the proxy variables (which is what impureEnvVars exists for) and denying everything else at the host firewall:

# builder host: only the proxy may leave; the build users' uids are nixbld1..32
$ nft list chain inet filter output | grep -E 'nixbld|drop'
    meta skuid 30001-30032 ip daddr 10.9.0.5 tcp dport 3128 accept    ← build users may only talk to the proxy
    meta skuid 30001-30032 drop                                        ← anything else from a build is dropped, and logged

# proxy log, one line per FOD fetch, correlated with the derivation name by the controller
2026-10-24T09:12:03Z ALLOW  drv=go-modules-my-service-1.4.2   CONNECT proxy.golang.org:443   bytes=18492011
2026-10-24T09:12:09Z ALLOW  drv=source-my-service-1.4.2       CONNECT github.com:443         bytes=2210441
2026-10-24T09:14:51Z DENY   drv=libfoo-fetch-1.0.3            CONNECT 203.0.113.9:443        reason=host-not-allowlisted   ← a FOD talking to something that is not a package source

A FOD that talks to an address that is not a known package host is either a packaging mistake or a beacon. Either way you want the DENY line, not the download.

#Binary cache poisoning

The pieces you need to knowA binary cache serves .narinfo files (metadata) and .nar.xz files (the bytes). The narinfo carries a Sig: line: a signature by the cache's Ed25519 key over the store path, the NAR hash, the NAR size and the list of referenced paths. Your Nix checks that signature against trusted-public-keys before adding the path, because require-sigs is true by default.

What can go wrongAn attacker who controls the storage (an S3 bucket, an Attic or Cachix account, an internal HTTP server) but not the key can delete entries or serve stale ones, but cannot forge a path: the signature fails and Nix falls back to building. The real risks are therefore about the key:

In practiceWhat a healthy narinfo looks like, and the error you want to see for a bad one:

$ curl -s https://cache.example.com/4q7…-my-service-1.4.2.narinfo
StorePath: /nix/store/4q7…-my-service-1.4.2
URL: nar/1b9c…f0.nar.xz
Compression: xz
FileHash: sha256:1b9c…f0
FileSize: 4120933
NarHash: sha256:0a3e…77                                                  ← hash of the unpacked bytes, what the signature covers
NarSize: 11829248
References: 4q7…-my-service-1.4.2 d4f…-bash-5.2p37 …
Sig: cache.example.com-1:MEUCIQ…==                                      ← key NAME must match an entry in trusted-public-keys

$ nix build .#my-service --substituters https://evil.example.net
warning: ignoring substitute for '/nix/store/4q7…-my-service-1.4.2' from 'https://evil.example.net', as it's not signed by any of the keys in 'trusted-public-keys'   ← the control working
building '/nix/store/9k2…-my-service-1.4.2.drv'...

📰

Real incident — xz-utils backdoor (2024, CVE-2024-3094). The pieces you need to know. xz is a compression library (liblzma) that, on several Linux distributions, is linked into the SSH server through systemd's notification library. Open-source projects publish a release tarball that is supposed to match a tagged commit, plus generated files (the configure script and its helper macros) that are not kept in version control. Downstream distributions build from the tarball. What went wrong. Over about two years a contributor gained maintainer trust, then shipped versions 5.6.0 and 5.6.1 whose release tarball contained a modified build macro, build-to-host.m4, that was not in the git repository. During the build it extracted code hidden inside two binary test files and patched it into liblzma. The resulting library hooked the SSH server's key verification so a holder of a specific private key could run commands as root. It was found in March 2024 by Andres Freund, a database developer, who noticed SSH logins using too much CPU. Why it's bad. The source in git was clean, the tarball was signed by the legitimate maintainer, and every build was reproducible: the same tarball gave the same backdoored library everywhere. Reproducibility proves you built what you were given. It says nothing about whether what you were given is what you think it is. The lesson. Reproducibility must reach back to the source you can read. Build from the tagged commit and regenerate configure yourself, or reproduce the tarball from the tag and compare; a tarball that cannot be regenerated from version control is an unverifiable input. Bootstrappable, hermetic builds that start from reviewed source are the structural fix, and they would have made the extra macro visible as a diff instead of a mystery.

🎯

On the job. Three greps on any Nix-based pipeline: __noChroot, sandbox = relaxed|false, and the identity that runs nix copy with a signing key. Then one question: "if a PR from a fork lands on this runner, what can it sign?" If the answer is "anything", the cache is a privilege escalation path from "open a PR" to "run code on every developer laptop".

#The Bootstrap Trust Problem

What is this?Every compiler was compiled by an earlier compiler. Reproducible builds show that source S gives binary B with toolchain T. They do not show that T is honest. Ken Thompson's 1984 Turing Award lecture, Reflections on Trusting Trust, described a C compiler that inserts a backdoor when it compiles the login program, and inserts the backdoor-inserting code when it compiles itself, so the malice survives in the binary even after the source is cleaned.

Why it mattersIt sets the floor of what "verified from source" can mean. If your only compiler is a binary someone handed you, the whole reproducible tower stands on an unverifiable seed.

How it worksTwo practical answers exist:

In practiceYou will not run stage0 on a release pipeline. What you will do is name the seed in the provenance, so trust is explicit rather than assumed:

# resolvedDependencies entry for the toolchain, from a provenance statement
{ "uri": "pkg:nix/nixpkgs@e9f6…?attr=stdenv", "digest": { "sha256": "b7c0…51" },
  "annotations": { "bootstrap": "nixpkgs-bootstrap-tools-2025-11 (reproduced from nixpkgs 7d8a…)" } }   ← says where the compiler came from, and that it was rebuilt, not downloaded once in 2019

🎯

On the job. Interviewers rarely expect you to have run a bootstrap. They expect you to know the shape of the argument: reproducibility verifies a build given a toolchain; diverse double-compiling or a bootstrappable seed verifies the toolchain; and in practice you pick a toolchain whose own build is reproducible (Go since 1.21, Guix's full-source chain, Nixpkgs' rebuilt bootstrap tools) and record its digest in provenance.

#In Practice: an Isolated Build Pipeline on Buildkite or GitHub Actions

What is this?Putting the pieces together on a platform you operate: bare-metal AMD EPYC and AmpereOne Linux hosts, Apple Silicon hosts running Linux and macOS guests under vz, with self-hosted Buildkite agents or GitHub Actions runners.

                  ┌──────────────── controller (Go) ───────────────┐
  pipeline event ─►  resolve source digest, mint job OIDC token,    │
                  │  start sandbox, record resolved inputs           │
                  └───┬──────────────────────────────┬─────────────┘
                      │                              │ after job: sign provenance
                      ▼                              ▼
   ┌──── sandbox (gVisor pod / Firecracker / vz VM) ────┐     ┌── signer ──┐      ┌── binary cache ──┐
   │ phase 1  FETCH   network → egress proxy only        │     │ keyless    │ ───► │ server-side sig  │
   │          FODs, go mod download, npm ci              │     │ cosign     │      │ require-sigs     │
   │ phase 2  BUILD   --network=none, declared inputs    │     └────────────┘      └──────────────────┘
   │ phase 3  OUTPUT  artifact + SBOM to controller      │
   └─────────────────────────────────────────────────────┘
                      │
                      ▼
              egress proxy (Go) ── allowlist per job: proxy.golang.org, registry.npmjs.org, github.com
                                   log every CONNECT with job id and derivation name

The rules the controller enforces:

# policy.star, evaluated by the controller for each job
def egress_allow(job):
    base = ["proxy.golang.org", "sum.golang.org", "registry.npmjs.org", "github.com"]
    if job.tenant == "platform":
        base += ["packages.wolfi.dev"]
    return base                                    # ← nothing else resolves; a FOD to anywhere else is DENY

def accept_artifact(att):
    return (att.issuer == "https://token.actions.githubusercontent.com" and
            att.identity.startswith("https://github.com/acme/release-workflows/") and   # ← only the vetted reusable workflow
            att.builder_id == "https://ci.acme.internal/controller/v3")

Deploy-time verificationThe last step is the consumer refusing anything without the right provenance: cosign verify-attestation (or slsa-verifier) against the identity above, plus a digest match against the SBOM and, for release artifacts, a second build on the other architecture's pool with a digest comparison.

🎯

On the job. If you inherit a pipeline and can change one thing, split fetch from build and put the proxy between them. It immediately gives you a list of every network input, a place to pin them, and a log that turns "a build downloaded something odd" from invisible into a DENY line with a derivation name on it.