chore: Use Dune package management #3132

sabine · 2025-05-26T14:04:58Z

This patch changes the build configuration to use Dune Package Management.

Done / To do:

To test this

$ make start

$ make playground

$ make scrape

Current status

I believe this is ready to merge.

gridbugs · 2025-06-05T01:23:24Z

I've been using dune package management to build this project whenever I've needed to contribute to it over the past year and it's worked fine until recently. Here are some PRs/issues relating to fixing the problems I've encountered recently:

gridbugs · 2025-06-05T01:26:24Z

There's also currently an issue with the ocamlnet package source being missing. I've added it to the opam-source-archives repo and I have a PR to point the ocamlnet package at the archived source. Once the latter PR is merged, updating the opam-repository revision in this project's dune-workspace to the current tip of opam-repository will fix the problem with ocamlnet.

Sudha247 · 2025-06-05T13:20:46Z

This is great! Thanks for the work, @sabine and @gridbugs. Upon offline discussion with Sabine, we wanted to test this on MacOS and Windows to ensure the workflow remains unaffected for contributors on those platforms. I tested with WSL2 (which is the recommended way to develop ocaml.org on Windows), and the ocaml.org project (with make start), builds and runs locally on WSL2, in this branch with dune pkg. Moreover, the testsuite passes too.

The ocaml.org playground doesn't build, as expected - as it seems to have some dependency issues. I'll try to investigate what's happening there!

sabine · 2025-06-06T07:35:46Z

Sounds great! Looks like we'll be migrating fully to dune pkg as soon as the playground builds.

sabine · 2025-06-16T07:41:19Z

Thanks @Sudha247 for sabine#53. This makes the playground dune build succeed.

One thing I observe on my machine:

Both on the main branch, and on this branch, running the playground build just deletes the .cmi files in the asset/ folder of the playground. It should be copying the .cmi files from somewhere. The script gen_stdlib.sh performs this part by looking up via opam the location of these .cmi files, so we need to either find where they reside with a dune build, or find another way to regenerate these files. 🤔

cuihtlauac · 2025-06-20T13:27:56Z

HACKING.md

Replace all ````` by ````bash` in this file

cuihtlauac · 2025-06-20T13:32:03Z

dune-project

Could the removal of the dependency on Anton's river fork be merged ahead in a separate PR?

Leonidas-from-XIV · 2025-06-20T14:45:03Z

Dockerfile

-RUN opam install . --deps-only
+# Install Dune Developer Preview
+RUN curl -fsSL https://get.dune.build/install | sh
+RUN /bin/bash -c 'source "/root/.local/share/dune/env/env.bash"'


Does this have any effect? The environment disappears after this call.

We must not download a dev preview on each build, that's brittle and wasteful.

Leonidas-from-XIV · 2025-06-20T14:46:50Z

Dockerfile

-RUN sudo apk -U upgrade --no-cache && sudo apk add --no-cache \
+RUN apk -U upgrade --no-cache && apk add --no-cache \
+    # to download and install Dune Developer Preview with alpine:3.21
+    build-base patch tar ca-certificates git rsync curl sudo bash \


Is there a need for sudo, rsync, bash or nano?

You also probably don't need curl. Docker can itself download remote files. See https://docs.docker.com/reference/dockerfile/#adding-files-from-a-url

Indeed, but dune uses curl internally for (url ..) fields, so we still need it.

Leonidas-from-XIV · 2025-06-20T14:49:14Z

playground/dune-project

-  rpclib-lwt))
+  (astring (= 0.8.5))
+  (base-bigarray (= base))
+  (base-bytes (= base))


I don't think it is customary to depend on all these base-* packages. After all, they're just transitive dependencies of the compiler selected.

v-gb · 2025-06-20T23:08:19Z

Makefile

 watch: ## Watch for the filesystem and rebuild on every change
-	opam exec -- dune build @run -w --force --no-buffer
+	dune build @run -w --force --no-buffer


I read the HACKING doc a bit quickly, and ran make watch instead of make start first:

$ make watch dune build @run -w --force --no-buffer File "src/ocamlorg_web/lib/dune", line 13, characters 2-15: 13 | mirage-kv-mem)) ^^^^^^^^^^^^^ Error: Library "mirage-kv-mem" not found. -> required by library "ocamlorg_web" in _build/default/src/ocamlorg_web/lib -> required by executable main in src/ocamlorg_web/bin/dune:2 -> required by _build/default/src/ocamlorg_web/bin/main.exe -> required by alias src/ocamlorg_web/bin/run in src/ocamlorg_web/bin/dune:8 ######################################################################## 100.0% Done: 95% (95/100, 5 left, 1 failed) (jobs: 1)Browserslist: caniuse-lite is outdated. Please run: npx update-browserslist-db@latest Why you should do it regularly: https://github.com/browserslist/update-db#readme Rebuilding... Done in 1911ms.

Shouldn't every rule that does a build depend on dune.lock, instead of just a couple of them? (or taking a step back, I thought the usual thing to do is commit the dune.lock, so I'm a bit confused about why it's being generated on the fly).

thanks, you're right. need to update the Makefile

Updates

gridbugs · 2025-07-30T07:40:15Z

dev-tools.locks/ocamlformat/ocamlformat.pkg

@@ -0,0 +1,14 @@
+(version
+ 0.26.2+binary-ocaml-5.2.1-built-2024-12-04.0-x86_64-unknown-linux-musl)


Be careful checking this file in as it's not portable. This version of ocamlformat will only work on linux. I'd recommend adding the dev-tools.locks directory to .gitignore so it doesn't get checked in and each user can install the version of the dev tools appropriate to their platform.

shonfeder · 2025-07-31T21:45:02Z

The install script is ready here, thanks to @gridbugs' work: https://github.com/ocaml-dune/dune-bin-install

@Sudha247 is planning to add documentation here about how to use it.

It also sounds like we should take Steve's advice about ignoring the lock dir, and IIUC @cuihtlauac's review has been addressed and just needs to be re-requested to clear the change request?

cuihtlauac

Small remarks:

File .github/workflows/scrape_changelog.yml should be updated.
There are a few remaining references to olinkcheck that should be removed, it is no longer used

Big question

Is this PR proposing to build ocaml.org with a version of Dune that changes daily? If that is the case, I must object. This is unwise. I support building ocaml.org as a regression test in Dune developer preview CI. However, using an unstable build tool in a production delivery chain is reckless.

cuihtlauac · 2025-08-01T06:14:15Z

dune-workspace

+(pin
+(name olinkcheck)
+ (url "git+https://github.com/tarides/olinkcheck")
+ (package
+  (name olinkcheck)))


PR #3144 Removed olinkcheck. I dont think this is needed any longer

cuihtlauac · 2025-08-01T06:14:51Z

dune-workspace

+  (pins olinkcheck)
+   (version_preference newest)
+  )


Same remark w.r.t. PR#3144

cuihtlauac · 2025-08-01T06:31:00Z

.github/workflows/ci.yml

+      - name: Use Dune Developer Preview
+        uses: ocaml-dune/setup-dune@v0


I'm not sure I fully understand this.

Are we attempting to build ocaml.org with a build tool that changes daily?

Sudha247 · 2025-08-01T08:47:14Z

Is this PR proposing to build ocaml.org with a version of Dune that changes daily? If that is the case, I must object.

The PR in its current state proposes this approach. However, as @shonfeder noted above, we can use @gridbugs' installer to install the released Dune binary (currently version 3.19.1). I'll update the PR and documentation accordingly. The idea is to use dune pkg for building and deploying the website. Since dune pkg is available in the released version of Dune, we don’t need the developer preview as the experimental features it offers aren’t essential for this PR.

sabine mentioned this pull request May 26, 2025

Add GitHub Actions workflow for dune pkg #3127

Closed

sabine force-pushed the dune_pkg branch from 8786902 to 0b27bb5 Compare June 3, 2025 10:54

sabine force-pushed the dune_pkg branch from 0b27bb5 to 35c9778 Compare June 5, 2025 10:12

sabine force-pushed the dune_pkg branch 3 times, most recently from a279318 to d48543c Compare June 16, 2025 07:37

sabine mentioned this pull request Jun 17, 2025

Dune pkg sabine/ocaml.org#53

Closed

sabine marked this pull request as ready for review June 17, 2025 09:49

sabine changed the title ~~chore: migrate to using dune package management~~ chore: Use Dune package management Jun 20, 2025

cuihtlauac requested changes Jun 20, 2025

View reviewed changes

Leonidas-from-XIV reviewed Jun 20, 2025

View reviewed changes

This was referenced Jun 20, 2025

Do not use Anton's river fork #3178

Merged

No code blocks w/o language #3181

Merged

Only use Alpine 3.21 #3180

Merged

Call make instead of dune build #3179

Merged

v-gb reviewed Jun 20, 2025

View reviewed changes

sabine force-pushed the dune_pkg branch from 936ba3f to 7d4f405 Compare June 26, 2025 12:58

Sudha247 mentioned this pull request Jul 17, 2025

Updates sabine/ocaml.org#58

Merged

4 tasks

sabine and others added 7 commits July 23, 2025 12:01

chore: migrate to using dune package management

0f4db7a

make Dockerfile use Dune Developer Preview to build

b21247b

update Makefile to use Dune Developer Preview

182a78f

remove .opam files

e1f7500

update HACKING.md for dune pkg, CONTRIBUTING.md doesn't need updating

e1aebfc

attempt to make Playground build with Dune Developer Preview (fails)

2b6f402

Build playground with dune pkg

a6a466c

sabine added 7 commits July 23, 2025 12:01

update CI to use Dune Developer Preview

e43025a

update scraper workflow to use dev preview

57c57c9

copy playground cmi files

fa47f31

update playground gitignore

36da910

remove opam.locked file

fdbd3db

rely on dune.lock

74d3104

remove obsolete config

5674aad

sabine force-pushed the dune_pkg branch from 7d4f405 to 5674aad Compare July 23, 2025 10:03

sabine and others added 5 commits July 23, 2025 12:09

build scraper in makefile

aab39ea

rely on dune.lock

3c38088

Pin dune pkg to a specific version in the Dockerfile

19b7967

update Dockerfile and HACKING.md

ed0b182

Merge pull request #58 from Sudha247/dune_pkg_new

2365158

Updates

sabine added the stalled Blocked waiting. Will not progress w/o external progress label Jul 23, 2025

revert accidental playground changes

1f79aea

shonfeder mentioned this pull request Jul 29, 2025

changelog: add a post about OCaml.org migration to Dune Developer Pre… #3152

Open

gridbugs reviewed Jul 30, 2025

View reviewed changes

cuihtlauac requested changes Aug 1, 2025

View reviewed changes

		@@ -0,0 +1,14 @@
		(version
		0.26.2+binary-ocaml-5.2.1-built-2024-12-04.0-x86_64-unknown-linux-musl)

		- name: Use Dune Developer Preview
		uses: ocaml-dune/setup-dune@v0

chore: Use Dune package management #3132

Are you sure you want to change the base?

chore: Use Dune package management #3132

Uh oh!

Conversation

sabine commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

To test this

Current status

Uh oh!

gridbugs commented Jun 5, 2025

Uh oh!

gridbugs commented Jun 5, 2025

Uh oh!

Sudha247 commented Jun 5, 2025

Uh oh!

sabine commented Jun 6, 2025

Uh oh!

sabine commented Jun 16, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

v-gb Jun 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gridbugs Jul 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shonfeder commented Jul 31, 2025

Uh oh!

cuihtlauac left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cuihtlauac Aug 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Sudha247 commented Aug 1, 2025

Uh oh!

Uh oh!

sabine commented May 26, 2025 •

edited

Loading

v-gb Jun 20, 2025 •

edited

Loading

gridbugs Jul 30, 2025 •

edited

Loading

cuihtlauac left a comment •

edited

Loading

cuihtlauac Aug 1, 2025 •

edited

Loading