A Result Monad for Nix Derivations

nix-unit and its like handle eval-time logic well: a plain expression, compared against what you expected, no derivation involved. But that’s all they can express. There’s no way to write a test for whether a linter passes or a generated file contains what it should.

You can just have packages, though. Ordinary derivations, and a broken build already tells you something failed. That’s nix flake check in a nutshell, but that’s exactly where the data problem shows up. --keep-going may build past a failure, but what you get is a pile of raw build logs and stack traces, one per failure, in whatever order they finished. Nothing about that is structured enough to ask “which of these fifty checks passed” without reading every log by hand. That gap — no structured way to test something that needs an actual build — is what got this started.

The Workaround§

Don’t let failure unwind the call stack. Return it as an ordinary value instead. Let the caller decide what to do with it, independent of whatever else is running. Rust calls this Result<T, E>. Haskell calls it Either. It’s all the same idea: the Result monad. Some might call it the king of error handling.

Nix has no return value to repurpose that way: A derivation either builds or it doesn’t, full stop. The trick has to move one level down. If we make the derivation itself impossible to fail, we can let its outputs carry the result.

The check still runs. It still might fail. But the wrapper always exits zero, and whatever actually happened gets written into the wrapper’s outputs. Once failure lives in an output instead of a build result, the Nix daemon has nothing stopping it from building every check in parallel, whether they’re all passing or half of them are broken.

Building the Shape§

The first version gave every check three outputs: out for any artifacts, log for stdout and stderr captured together, and exitCode for the actual pass-or-fail verdict. mkResult was the constructor — a name, an environment, a command.¹ The command’s real exit code was captured as exitCode while the wrapping derivation’s own exit code stayed zero.

Once that worked, comparing results needed the same treatment. mkResultSnapshot took a result derivation and built a result-shaped derivation which compared its exit code and log against expected values. Thus a snapshot check is a check like any other, falling right out of the architecture.

That uniformity turned out to matter more than I expected. Because every check reduces to the same shape regardless of what it’s testing, nothing downstream needs to implement special treatment. mkSkip elegantly leverages that uniformity: because every check is overridable with the same shape, mkSkip can supplant a check with a derivation which produces empty outputs and has no dependencies. One simple override, applicable to any check is all it took to implement skipping.²

Failure as Data§

Every check became a normal derivation, built by the ordinary Nix daemon in parallel, and a report at the end that told you which was which without a single early exit. That’s the whole idea. Failure stopped being an event the daemon reacted to and became a value the store could just carry, same as any other output.

It only covered half the problem, though. Everything above is about checks that build something real — a derivation, actually realized in the store. The other kind of thing worth testing, pure Nix logic that never touches a builder, didn’t fit this shape at all yet. That gap is where the next mistake was waiting.