hackage2nix: add excluded-packages config option

[wolfgangwalther]

This adds a config option excluded-packages, which can be used in Nixpkgs to remove a big chunk of old, broken packages.

WIP PR in Nixpkgs at NixOS/nixpkgs#441204.

[wolfgangwalther]

I'm currently experimenting with this in Nixpkgs and I'm confused about something: When I remove a package that is used by others, then in some cases hackage2nix / cabal2nix behave differently than in others:

• Sometimes the respective dependency is explicitly set to null in the callPackage args,
• and sometimes it's not, which then fails eval.

Two random examples. I excluded accelerate-llvm and haskell98:

  "caldims" = callPackage
    ({ mkDerivation, base, containers, directory, haskell98, mtl
     , parsec, readline
     }:
     mkDerivation {
       pname = "caldims";
       version = "0.1.0";
       sha256 = "0mlgxghah8mw0v17rywfj190kmc4jajpmjpgkpgfxdqzw8djyph0";
       isLibrary = true;
       isExecutable = true;
       libraryHaskellDepends = [
         base containers directory haskell98 mtl parsec readline
       ];
       executableHaskellDepends = [
         base containers directory haskell98 mtl parsec readline
       ];
       description = "Calculation tool and library supporting units";
       license = "GPL";
       hydraPlatforms = lib.platforms.none;
       mainProgram = "caldims";
       broken = true;
     }) {haskell98 = null;};

haskell98 is passed as null (good)

  "accelerate-fft" = callPackage
    ({ mkDerivation, accelerate, accelerate-llvm
     , accelerate-llvm-native, accelerate-llvm-ptx, base, bytestring
     , carray, containers, cuda, cufft, fft, file-embed, hashable
     , hedgehog, lens-accelerate, mtl, tasty, tasty-hedgehog
     , unordered-containers
     }:
     mkDerivation {
       pname = "accelerate-fft";
       version = "1.3.0.0";
       sha256 = "1a7cwzbs8r3rvaymrq2kfx83lqb3i7wz0gmz3ppz59f40rxn974x";
       libraryHaskellDepends = [
         accelerate accelerate-llvm accelerate-llvm-native
         accelerate-llvm-ptx base bytestring carray containers cuda cufft
         fft file-embed hashable lens-accelerate mtl unordered-containers
       ];
       testHaskellDepends = [
         accelerate accelerate-llvm-native accelerate-llvm-ptx base hedgehog
         tasty tasty-hedgehog
       ];
       description = "FFT using the Accelerate library";
       license = lib.licenses.bsd3;
       hydraPlatforms = lib.platforms.none;
       broken = true;
     }) {lens-accelerate = null;};

accelerate-llvm is not passed as null (bad).

Why is that? @sternenseemann any insight?

[sternenseemann]

I would need to investigate, too. Possible that this is stateful somehow. I've never really touched hackaeg2nix, but do know that it uses STM, so may be tricky behavior involved.

[wolfgangwalther]

So it seems like accelerate-fft depends on both lens-accelerate unconditionally and accelerate-llvm behind a flag. When I remove both dependencies, Cabal's finalizePD will return early with only a missing dependency of lens-accelerate- it doesn't even need to check flags, because it already knows it won't be able to succeed. This means we only set lens-accelerate = null, but not accelerate-llvm, because we do that based on the missing dependencies returned by that function.

[wolfgangwalther]

The first approach wasn't working well: Initially, I removed the excluded package from the hackage db at the very beginning. But this would lead to unexpected results later: Cabal would miss some dependencies, and thus toggle flags in a different way. This would also affect packages that were not broken, right now. For example prettyprinter has a text flag, which suddenly got toggled because Cabal couldn't find one of the benchmark dependencies anymore - this dependency was removed, because it was long broken. But since we never enabled the benchmarks anyway, we didn't care about this being broken. This then caused the whole set to rebuild.

The new approach is different: Now, I'm just writing foo = null; for each dependency that should be excluded. That already reduces the number of lines by a lot - and seems to work well, because the flag selection is not affected: Cabal still has access to the full package db. We could potentially just remove this null binding as well, and run the same thing in Nixpkgs, directly from the excluded file. That would save us another few thousand lines. But given the scale of what we're operating on, this isn't important right now: The diff in Nixpkgs has a few 100k lines less with this.

[maralorn]

LGTM

[maralorn]

I find it a bit confusing to define def as https://hackage-content.haskell.org/package/data-default-0.8.0.1/docs/Data-Default.html#v:def is a very widespread binding.

[wolfgangwalther]

I had binding at first, but this would shadow something else already in scope.

I went with def, because the list of these is called defs (see a few lines down).

I could maybe just do definition? (I assumed that's what defs means...)