Don't reverse proxy hydra.nixos.org

[mweinelt]

Observations

in ccc6d2d you started telling netlify to reverse proxy the nix manuals directly from hydra.nixos.org

Problem

This became a problem when we started using Anubis for dealing with automated abusive requests. The reverse proxied site then served a broken validation page, that would never complete the validation steps.

Approaches

Fetch the documentation packages and host them yourself.

Willing to help?

The infra team has limited resources available, but you are free to ask, if you need anything.

Timeline
NixOS/infra#663
NixOS/nix#13099 (comment)
NixOS/infra#665

Priorities

Add 👍 to issues you find important.

[Mic92]

Assuming the build is cheap, this could be moved to github actions:

$ nix-build
$ netlify deploy --dir=./result --prod --site=<YOUR_NETLIFY_SITE_ID> --auth=$NETLIFY_AUTH_TOKEN

We should avoid hitting hydra with more load than necessary.

[vcunat]

It should be cheap: if it used to pull from hydra.nixos.org, a local build would substitute it from cache.nixos.org.

[Mic92]

Well likely it won't be cached at this point yet. But if building a static website would be expensive slow, I think there would be something else to worry about.

[fricklerhandwerk]

Building the Nix manual is atrociously expensive, because it requires building all of Nix: NixOS/nix#8518

This is why we have this contraption to fetch only cached builds for each release. Running that update script is also rather expensive, this is also why the development version points directly to Hydra.

Ideally we'd catch a webhook and grab the built tarball of the manual directly, since the development version should be available ASAP and is ephemeral (there's no point in re-building nix.dev for it). But we'd need control over the web server for that.