Skip to content

chore: Use Dune package management #3132

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 20 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
!tool/
!dune
!dune-project
!ocamlorg.opam
!dune-workspace
!tailwind.config.js
!init-cache
!.git
18 changes: 2 additions & 16 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,21 +22,13 @@ jobs:
os:
- macos-latest
- ubuntu-latest
ocaml-compiler:
- "5.2.0"

steps:
- name: Checkout Repo
uses: actions/checkout@v4

- name: Use OCaml ${{ matrix.ocaml-compiler }}
uses: ocaml/setup-ocaml@v3
with:
ocaml-compiler: ${{ matrix.ocaml-compiler }}
dune-cache: ${{ matrix.os != 'macos-latest' }}
opam-repositories: |
pin: git+https://github.com/ocaml/opam-repository#584630e7a7e27e3cf56158696a3fe94623a0cf4f
opam-disable-sandboxing: true
- name: Use Dune Developer Preview
uses: ocaml-dune/setup-dune@v0
Comment on lines +30 to +31
Copy link
Collaborator

@cuihtlauac cuihtlauac Aug 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I fully understand this.

Are we attempting to build ocaml.org with a build tool that changes daily?


- name: Install system dependencies (Linux)
run: sudo apt update && sudo apt-get install libev-dev libonig-dev libcurl4-openssl-dev
Expand All @@ -46,12 +38,6 @@ jobs:
run: brew update && brew reinstall libev openssl@3 # Openssl is a workaround for https://github.com/ocaml/opam-repository/issues/19676
if: runner.os == 'macOS'

- name: Install opam dependencies
run: opam install --deps-only --with-test .

- name: Install ocamlformat
run: opam install ocamlformat.0.26.2

- name: Build project
run: make

Expand Down
17 changes: 2 additions & 15 deletions .github/workflows/scrape.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,25 +24,12 @@ jobs:
- name: Checkout code
uses: actions/checkout@v4

- name: Use OCaml ${{ matrix.ocaml-compiler }}
uses: ocaml/setup-ocaml@v3
with:
ocaml-compiler: ${{ matrix.ocaml-compiler }}
dune-cache: ${{ matrix.os != 'macos-latest' }}
opam-repositories: |
pin: git+https://github.com/ocaml/opam-repository#584630e7a7e27e3cf56158696a3fe94623a0cf4f
opam-disable-sandboxing: true
- name: Use Dune Developer Preview
uses: ocaml-dune/setup-dune@v0

- name: Install system dependencies
run: sudo apt update && sudo apt-get install libev-dev libonig-dev libcurl4-openssl-dev

- name: Install opam dependencies
run: opam install --deps-only --with-test .

- name: Build scraper
run: |
opam exec -- dune build tool/ood-gen/bin/scrape.exe

- name: Run scrapers
run: |
make scrape_ocaml_planet
Expand Down
40 changes: 20 additions & 20 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
FROM ocaml/opam:alpine-3.21-ocaml-5.2 AS build
FROM alpine:3.21 AS build

# Install system dependencies
RUN sudo apk -U upgrade --no-cache && sudo apk add --no-cache \
RUN apk -U upgrade --no-cache && apk add --no-cache \
# to download and install Dune Developer Preview with alpine:3.21
build-base patch tar ca-certificates git \
libx11-dev coreutils xz curl bash \
autoconf \
curl-dev \
gmp-dev \
Expand All @@ -10,25 +13,22 @@ RUN sudo apk -U upgrade --no-cache && sudo apk add --no-cache \
oniguruma-dev \
openssl-dev

# Use Opam 2.2 and enable the backup mirror if primary sources of packages are unavailable
RUN sudo mv /usr/bin/opam-2.2 /usr/bin/opam && opam update
RUN opam option --global 'archive-mirrors+="https://opam.ocaml.org/cache"'

# Branch freeze was opam-repo HEAD at the time of commit
RUN cd ~/opam-repository && git reset --hard 584630e7a7e27e3cf56158696a3fe94623a0cf4f && opam update

WORKDIR /home/opam

# Install opam dependencies
COPY --chown=opam ocamlorg.opam .
RUN opam install . --deps-only
ADD https://get.dune.build/2025-07-17/x86_64-unknown-linux-musl/dune-2025-07-17-x86_64-unknown-linux-musl.tar.gz .
RUN tar xzf dune-2025-07-17-x86_64-unknown-linux-musl.tar.gz
RUN mkdir -p /root/.local/bin \
&& mv dune-2025-07-17-x86_64-unknown-linux-musl/dune /root/.local/bin/
ENV PATH="/root/.local/bin:$PATH"
RUN dune --version

# Build project
COPY --chown=opam . .
RUN opam exec -- dune build @install --profile=release
WORKDIR "/root/ocaml.org"
COPY --chown=root . .
RUN ls

RUN dune pkg lock
RUN dune build @install --profile=release

# Launch project in order to generate the package state cache
RUN cd ~/opam-repository && git checkout master && git pull origin master && opam update
ENV OCAMLORG_PKG_STATE_PATH=package.state \
OCAMLORG_REPO_PATH=opam-repository
RUN touch package.state && ./init-cache package.state
Expand All @@ -40,9 +40,9 @@ RUN apk -U upgrade --no-cache && apk add --no-cache \
gmp \
libev

COPY --from=build /home/opam/package.state /var/package.state
COPY --from=build /home/opam/opam-repository /var/opam-repository
COPY --from=build /home/opam/_build/default/src/ocamlorg_web/bin/main.exe /bin/server
COPY --from=build "/root/ocaml.org/package.state" /var/package.state
COPY --from=build "/root/ocaml.org/opam-repository" /var/opam-repository
COPY --from=build "/root/ocaml.org/_build/default/src/ocamlorg_web/bin/main.exe" /bin/server

COPY playground/asset playground/asset

Expand Down
62 changes: 13 additions & 49 deletions HACKING.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,32 +24,20 @@ git clone https://github.com/<username>/OCaml.org.git
cd OCaml.org
```

Ensure you have `opam` installed. Opam will manage the OCaml compiler along with all of the OCaml packages needed to build and run the project. By this point, we should all be using some Unix-like system (Linux, macOS, WSL2), so you should [run the opam install script](https://opam.OCaml.org/doc/Install.html#Binary-distribution). There are also manual instructions for people that don't want to run a script from the internet. We assume you are using `opam.2.1.0` or later, which provides a cleaner, friendlier experience when installing system dependencies.
Ensure you have [Dune Developer Preview](https://preview.dune.build) installed. Dune will manage the OCaml compiler along with all of the OCaml packages needed to build and run the project. By this point, we should all be using some Unix-like system (Linux, macOS, WSL2). We assume you are using the most recent version of Dune Developer Preview.

With opam installed, you can now initialise opam with `opam init`. Note that in containers or WSL2, you will have to run `opam init --disable-sandboxing`. Opam might complain about some missing system dependencies like `unzip`, `cc` (a C compiler like `gcc`), etc. Make sure to install these before `opam init`.

Finally from the root of your project, you can setup a [local opam switch](https://opam.OCaml.org/doc/Manual.html#Switches) and install the dependencies. There is a single `make` target to do just that.

```bash
make switch
```

If you don't want a local opam switch and are happy to install everything globally (in the opam sense), then you can just install the dependencies directly.

```bash
make deps
```

Opam will likely ask questions about installing system dependencies. Ror the project to work, you will have to answer yes to installing these.
If you would like to build using Dune installed via `opam`, make sure to remove the `dune.lock/` directories and use the same build commands present in the Makefile, skipping `dune pkg lock`.

### Running the Server

After building the project, you can run the server with:
From the root of your project, you can just build and run the project with

```bash
make start
```

Dune will install the OCaml compiler, as well as all the dependencies needed by the project.

To start the server in watch mode, you can run:

```bash
Expand Down Expand Up @@ -80,13 +68,7 @@ The OCaml Playground is compiled separately from the rest of the server. The gen

You can build the playground from the root of the project. There is no need to move to the `./playground/` directory for the following commands.

To regenerate the playground, you need to install the playground's dependencies first:

```bash
make deps -C playground
```

After the dependencies have been installed, simply build the project to regenerate the JavaScript assets:
Simply build the project to regenerate the JavaScript assets:

```bash
make playground
Expand Down Expand Up @@ -128,30 +110,13 @@ before they get merged.

### Managing Dependencies

OCaml.org is using an opam switch that is local and bound to a pinned commit in `opam-repository`. This is intended to protect the build from upstream regressions. The opam repository is specified in three (3) places:
OCaml.org is using an pinned version of `opam-repository`. This is intended to protect the build from upstream regressions. The opam repository is specified in one place:

```bash
Dockerfile
Makefile
.github/workflows/*.yml
```

When bringing up OCaml.org to a newer pin, the commit hash found it those files must be changed all at once.

Once the opam repo pin is updated, the local switch must be updated using the following command:

```sh
opam repo set-url pin git+https://github.com/ocaml/opam-repository#<commit-hash>
dune-workspace
```

Where `<commit-hash>` is the pinned hash specified in the files mentioned above.

Once this is done, you can run `opam update` and `opam upgrade`. If OCamlFormat
was upgraded in the process, the files `.ocamlformat` and
`.github/workflows/ci.yml` must be modified with the currently installed version
of OCamlFormat.

### Handling the Tailwind CSS
### Handling the Tailwind CSS CLI

The Tailwind CSS framework. The tailwind binary pulled from its GitHub [repo](https://github.com/tailwindlabs/tailwindcss). Download is performed by Dune during the build. When working on a local switch for hacking, you don't want `dune clean` to delete this binary. Just do `dune install tailwind` to have it installed in the local switch.

Expand Down Expand Up @@ -192,11 +157,10 @@ The following snippet describes the repository structure:
├── dune
├── dune-project
│ Dune file used to mark the root of the project and define project-wide parameters.
│ For the documentation of the syntax, see https://dune.readthedocs.io/en/stable/dune-files.html#dune-project.
├── ocamlorg.opam
├── ocamlorg.opam.template
│ opam package definitions.
│ For the documentation of the syntax, see https://dune.readthedocs.io/en/latest/reference/dune-project/index.html.
├── dune-workspace
│ Dune file used to define the repositories used for dependencies by Dune package management
│ For the documentation of the syntax, see https://dune.readthedocs.io/en/latest/reference/dune-workspace/index.html.
├── CONTRIBUTING.md
Expand Down
52 changes: 20 additions & 32 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,69 +1,57 @@
.DEFAULT_GOAL := all

.PHONY: all
all:
opam exec -- dune build --root .
all: dune.lock
dune build --root .

.PHONY: deps
deps: create_switch ## Install development dependencies
opam install -y ocamlformat=0.26.2 ocaml-lsp-server
opam install -y --deps-only --with-test --with-doc .

.PHONY: create_switch
create_switch: ## Create switch and pinned opam repo
opam switch create . 5.2.0 --no-install --repos pin=git+https://github.com/ocaml/opam-repository#584630e7a7e27e3cf56158696a3fe94623a0cf4f

.PHONY: switch
switch: deps ## Create an opam switch and install development dependencies

.PHONY: lock
lock: ## Generate a lock file
opam lock -y .
dune.lock: ## Generate a lock file
dune pkg lock

.PHONY: build
build: ## Build the project, including non installable libraries and executables
opam exec -- dune build --root .
build: dune.lock ## Build the project, including non installable libraries and executables
dune build --root .

.PHONY: playground
playground:
make build -C playground

.PHONY: install
install: all ## Install the packages on the system
opam exec -- dune install --root .
dune install --root .

.PHONY: start
start: all ## Run the produced executable
opam exec -- dune exec src/ocamlorg_web/bin/main.exe
dune exec src/ocamlorg_web/bin/main.exe

.PHONY: test
test: ## Run the unit tests
opam exec -- dune build --root . @runtest
dune build --root . @runtest

.PHONY: clean
clean: ## Clean build artifacts and other generated files
opam exec -- dune clean --root .
dune clean --root .

.PHONY: doc
doc: ## Generate odoc documentation
opam exec -- dune build --root . @doc
dune build --root . @doc

.PHONY: fmt
fmt: ## Format the codebase with ocamlformat
opam exec -- dune build --root . --auto-promote @fmt
dune build --root . --auto-promote @fmt

.PHONY: watch
watch: ## Watch for the filesystem and rebuild on every change
opam exec -- dune build @run -w --force --no-buffer
watch: dune.lock ## Watch for the filesystem and rebuild on every change
dune build @run -w --force --no-buffer

.PHONY: utop
utop: ## Run a REPL and link with the project's libraries
opam exec -- dune utop --root . . -- -implicit-bindings
dune utop --root . . -- -implicit-bindings

.PHONY: scrape_ocaml_planet
scrape_ocaml_planet:
opam exec -- dune exec --root . tool/ood-gen/bin/scrape.exe planet
opam exec -- dune exec --root . tool/ood-gen/bin/scrape.exe video
scrape_ocaml_planet: dune.lock ## Generate the po files
dune build --root . tool/ood-gen/bin/scrape.exe
dune exec --root . tool/ood-gen/bin/scrape.exe planet
dune exec --root . tool/ood-gen/bin/scrape.exe video

.PHONY: scrape_changelog
scrape_changelog:
Expand All @@ -75,4 +63,4 @@ docker: ## Generate docker container

.PHONY: linkcheck
linkcheck:
opam exec -- dune exec --root . olinkcheck md data
dune exec --root . olinkcheck md data
34 changes: 34 additions & 0 deletions dev-tools.locks/ocamlformat/lock.dune
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
(lang package 0.1)

(dependency_hash 9ab9b4cd76bc6260d907c3de3771c4bc)

(repositories
(complete true)
(used
((source
https://github.com/ocaml-dune/ocaml-binary-packages.git#fb7b39863a4d229e2cd3ac212459e15650c75d4d))
((source
https://github.com/ocaml-dune/opam-overlays.git#2a9543286ff0e0656058fee5c0da7abc16b8717d))
((source
https://github.com/ocaml/opam-repository.git#584630e7a7e27e3cf56158696a3fe94623a0cf4f))
((source
https://github.com/ocaml/opam-repository#584630e7a7e27e3cf56158696a3fe94623a0cf4f))))

(expanded_solver_variable_bindings
(variable_values
(with-doc false)
(with-dev-setup false)
(sys-ocaml-version 5.2.1)
(post true)
(os-distribution pop)
(os linux)
(opam-version 2.2.0~alpha-vendored)
(arch x86_64))
(unset_variables
with-test
sys-ocaml-libc
sys-ocaml-cc
sys-ocaml-arch
enable-ocaml-beta-repository
dev
build))
14 changes: 14 additions & 0 deletions dev-tools.locks/ocamlformat/ocamlformat.pkg
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
(version
0.26.2+binary-ocaml-5.2.1-built-2024-12-04.0-x86_64-unknown-linux-musl)
Copy link
Contributor

@gridbugs gridbugs Jul 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Be careful checking this file in as it's not portable. This version of ocamlformat will only work on linux. I'd recommend adding the dev-tools.locks directory to .gitignore so it doesn't get checked in and each user can install the version of the dev tools appropriate to their platform.


(install
(progn
(run find . -type d -exec mkdir -p %{prefix}/{} ";")
(run find . -type f -exec cp {} %{prefix}/{} ";")))

(source
(fetch
(url
https://github.com/ocaml-dune/ocaml-binary-packages/releases/download/2024-12-04.0/ocamlformat.0.26.2+binary-ocaml-5.2.1-built-2024-12-04.0-x86_64-unknown-linux-musl.tar.gz)
(checksum
sha256=f466234d0f725203f480ce38dddb14006e5d9af0041bba85bf2276e28593ca12)))
Loading
Loading