Skip to content

Commit 1319758

Browse files
per1234per1234
committed
Manage versioning of Poetry tool dependency
The project Python package dependencies are managed by the "Poetry" tool. Previously, the version of Poetry was not managed in any way. The GitHub Actions workflows used whichever version of Poetry happened to be installed on the runner machine. This meant that the GitHub Actions workflows could break at any time through the Poetry installation on the runner machine being updated to an incompatible version. The contributors used whichever version of Poetry happened to be installed on their machine. This meant that they might get different results from that produced by the environment of the GitHub Actions workflows. The better solution is to take the same approach for managing the Poetry dependency as is done for the project's other dependencies: * Install a specific version of Poetry according to a single source of versioning data. * Use the Dependabot service to get automated update pull requests. The logical place to define the Poetry package dependency version is in pyproject.toml, as is done for all other Python package dependencies. Dependabot has support for two different forms of dependency data in the pyproject.toml file: * Original Poetry data format, under the `tool.poetry` table * PEP 621 format data, under the `project` table Since Poetry can't be used to manage itself (it is instead installed using the "pipx" tool), the obvious approach would be to define the Poetry dependency via the `project` table in the file. However, this is not possible because if a `tool.poetry` table is present in pyproject.toml, Dependabot ignores the dependencies data from the `project` table. So it is necessary to place the data for the Poetry dependency under the `tool.poetry` table of the file. A special dependencies group is created for this purpose. That group is configured as "optional" so that it won't be installed redundantly by `poetry install` commands. Unfortunately pipx doesn't support using pyproject.toml as a dependency configuration file so it is necessary to get the Poetry version constraint for use in the dependency argument of the `pipx install` command by parsing the project.toml file.
1 parent 4b49cdf commit 1319758

25 files changed

+1555
-79
lines changed

‎.github/workflows/check-python-task.yml

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -71,9 +71,6 @@ jobs:
7171
with:
7272
python-version-file: pyproject.toml
7373

74-
- name: Install Poetry
75-
run: pip install poetry
76-
7774
- name: Install Task
7875
uses: arduino/setup-task@v2
7976
with:
@@ -102,9 +99,6 @@ jobs:
10299
with:
103100
python-version-file: pyproject.toml
104101

105-
- name: Install Poetry
106-
run: pip install poetry
107-
108102
- name: Install Task
109103
uses: arduino/setup-task@v2
110104
with:

‎.github/workflows/check-yaml-task.yml

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -99,9 +99,6 @@ jobs:
9999
with:
100100
python-version-file: pyproject.toml
101101

102-
- name: Install Poetry
103-
run: pip install poetry
104-
105102
- name: Install Task
106103
uses: arduino/setup-task@v2
107104
with:

‎.github/workflows/spell-check-task.yml

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -53,9 +53,6 @@ jobs:
5353
with:
5454
python-version-file: pyproject.toml
5555

56-
- name: Install Poetry
57-
run: pip install poetry
58-
5956
- name: Install Task
6057
uses: arduino/setup-task@v2
6158
with:

‎.github/workflows/test-python-poetry-task.yml

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -67,9 +67,6 @@ jobs:
6767
with:
6868
python-version-file: pyproject.toml
6969

70-
- name: Install Poetry
71-
run: pip install poetry
72-
7370
- name: Install Task
7471
uses: arduino/setup-task@v2
7572
with:

‎Taskfile.yml

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -827,15 +827,66 @@ tasks:
827827
-r "{{.STYLELINTRC_SCHEMA_PATH}}" \
828828
-d "{{.INSTANCE_PATH}}"
829829
830+
# Print the version constraint for the project's Poetry tool dependency.
831+
# Source: https://github.com/arduino/tooling-project-assets/blob/main/workflow-templates/assets/poetry-task/Taskfile.yml
832+
poetry:get-version:
833+
cmds:
834+
- |
835+
if ! which yq &>/dev/null; then
836+
echo "yq not found or not in PATH."
837+
echo "Please install: https://github.com/mikefarah/yq/#install"
838+
exit 1
839+
fi
840+
- |
841+
yq \
842+
--input-format toml \
843+
--output-format yaml \
844+
'.tool.poetry.group.pipx.dependencies.poetry' \
845+
< pyproject.toml
846+
847+
# Source: https://github.com/arduino/tooling-project-assets/blob/main/workflow-templates/assets/poetry-task/Taskfile.yml
848+
poetry:install:
849+
desc: Install Poetry
850+
run: once
851+
vars:
852+
POETRY_VERSION:
853+
sh: task poetry:get-version
854+
cmds:
855+
- |
856+
if ! which python &>/dev/null; then
857+
echo "Python not found or not in PATH."
858+
echo "Please install a version of Python meeting the constraint {{.POETRY_VERSION}}:"
859+
echo "https://wiki.python.org/moin/BeginnersGuide/Download"
860+
exit 1
861+
fi
862+
- |
863+
if ! which pipx &>/dev/null; then
864+
echo "pipx not found or not in PATH."
865+
echo "Please install: https://pipx.pypa.io/stable/installation/#installing-pipx"
866+
exit 1
867+
fi
868+
- |
869+
export PIPX_DEFAULT_PYTHON="$( \
870+
task utility:normalize-path \
871+
RAW_PATH="$(which python)" \
872+
)"
873+
pipx install \
874+
--force \
875+
"poetry=={{.POETRY_VERSION}}"
876+
830877
# Source: https://github.com/arduino/tooling-project-assets/blob/main/workflow-templates/assets/poetry-task/Taskfile.yml
831878
poetry:install-deps:
832879
desc: Install dependencies managed by Poetry
880+
deps:
881+
- task: poetry:install
833882
cmds:
834883
- poetry install --no-root
835884

836885
# Source: https://github.com/arduino/tooling-project-assets/blob/main/workflow-templates/assets/poetry-task/Taskfile.yml
837886
poetry:update-deps:
838887
desc: Update all dependencies managed by Poetry to their newest versions
888+
deps:
889+
- task: poetry:install
839890
cmds:
840891
- poetry update
841892

‎docs/development.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,6 @@ The following development tools must be available in your local environment:
1010
- The **Node.js** version in use is defined by the `engines.node` key of [`package.json`](../package.json).
1111
- [**Python**](https://wiki.python.org/moin/BeginnersGuide/Download)
1212
- The **Python** version in use is defined by the `tool.poetry.dependencies.python` key of [`pyproject.toml`](../pyproject.toml).
13-
- [**Poetry**](https://python-poetry.org/docs/#installation) - Python dependencies management tool
1413
- [**Task**](https://taskfile.dev/installation/) - task runner tool
1514

1615
## Building the Project

‎poetry.lock

Lines changed: 1431 additions & 23 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎pyproject.toml

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,14 @@ pep8-naming = "^0.15.1"
1616
pytest = "^8.4.1"
1717
GitPython = "^3.1.44"
1818

19+
# The dependencies in this group are installed using pipx; NOT Poetry. The use of the `tool.poetry.group` super-table
20+
# is a hack required in order to be able to manage updates of these dependencies via Dependabot.
21+
[tool.poetry.group.pipx]
22+
optional = true
23+
24+
[tool.poetry.group.pipx.dependencies]
25+
poetry = "2.1.3"
26+
1927
[build-system]
2028
requires = ["poetry-core>=1.0.0"]
2129
build-backend = "poetry.core.masonry.api"

‎workflow-templates/assets/poetry-task/Taskfile.yml

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,65 @@
22
version: "3"
33

44
tasks:
5+
# Print the version constraint for the project's Poetry tool dependency.
6+
# Source: https://github.com/arduino/tooling-project-assets/blob/main/workflow-templates/assets/poetry-task/Taskfile.yml
7+
poetry:get-version:
8+
cmds:
9+
- |
10+
if ! which yq &>/dev/null; then
11+
echo "yq not found or not in PATH."
12+
echo "Please install: https://github.com/mikefarah/yq/#install"
13+
exit 1
14+
fi
15+
- |
16+
yq \
17+
--input-format toml \
18+
--output-format yaml \
19+
'.tool.poetry.group.pipx.dependencies.poetry' \
20+
< pyproject.toml
21+
22+
# Source: https://github.com/arduino/tooling-project-assets/blob/main/workflow-templates/assets/poetry-task/Taskfile.yml
23+
poetry:install:
24+
desc: Install Poetry
25+
run: once
26+
vars:
27+
POETRY_VERSION:
28+
sh: task poetry:get-version
29+
cmds:
30+
- |
31+
if ! which python &>/dev/null; then
32+
echo "Python not found or not in PATH."
33+
echo "Please install a version of Python meeting the constraint {{.POETRY_VERSION}}:"
34+
echo "https://wiki.python.org/moin/BeginnersGuide/Download"
35+
exit 1
36+
fi
37+
- |
38+
if ! which pipx &>/dev/null; then
39+
echo "pipx not found or not in PATH."
40+
echo "Please install: https://pipx.pypa.io/stable/installation/#installing-pipx"
41+
exit 1
42+
fi
43+
- |
44+
export PIPX_DEFAULT_PYTHON="$( \
45+
task utility:normalize-path \
46+
RAW_PATH="$(which python)" \
47+
)"
48+
pipx install \
49+
--force \
50+
"poetry=={{.POETRY_VERSION}}"
51+
552
# Source: https://github.com/arduino/tooling-project-assets/blob/main/workflow-templates/assets/poetry-task/Taskfile.yml
653
poetry:install-deps:
754
desc: Install dependencies managed by Poetry
55+
deps:
56+
- task: poetry:install
857
cmds:
958
- poetry install --no-root
1059

1160
# Source: https://github.com/arduino/tooling-project-assets/blob/main/workflow-templates/assets/poetry-task/Taskfile.yml
1261
poetry:update-deps:
1362
desc: Update all dependencies managed by Poetry to their newest versions
63+
deps:
64+
- task: poetry:install
1465
cmds:
1566
- poetry update

‎workflow-templates/assets/poetry/pyproject.toml

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,5 +7,13 @@ build-backend = "poetry.core.masonry.api"
77
[tool.poetry]
88
package-mode = false
99

10+
# The dependencies in this group are installed using pipx; NOT Poetry. The use of the `tool.poetry.group` super-table
11+
# is a hack required in order to be able to manage updates of these dependencies via Dependabot.
12+
[tool.poetry.group.pipx]
13+
optional = true
14+
15+
[tool.poetry.group.pipx.dependencies]
16+
poetry = "2.1.3"
17+
1018
[tool.poetry.dependencies]
1119
python = "~3.9"

‎workflow-templates/check-mkdocs-task.yml

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -67,9 +67,6 @@ jobs:
6767
with:
6868
python-version-file: pyproject.toml
6969

70-
- name: Install Poetry
71-
run: pip install poetry
72-
7370
- name: Install Task
7471
uses: arduino/setup-task@v2
7572
with:

‎workflow-templates/check-python-task.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ The code style defined in `pyproject.toml` and `.flake8` is the official standar
2828
Add the tool dependencies using this command:
2929

3030
```
31-
poetry add --dev "black@^25.1.0" "flake8@^7.2.0" "pep8-naming@^0.15.1"
31+
task poetry:install && poetry add --dev "black@^25.1.0" "flake8@^7.2.0" "pep8-naming@^0.15.1"
3232
```
3333

3434
Commit the resulting changes to the `pyproject.toml` and `poetry.lock` files.

‎workflow-templates/check-python-task.yml

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -71,9 +71,6 @@ jobs:
7171
with:
7272
python-version-file: pyproject.toml
7373

74-
- name: Install Poetry
75-
run: pip install poetry
76-
7774
- name: Install Task
7875
uses: arduino/setup-task@v2
7976
with:
@@ -102,9 +99,6 @@ jobs:
10299
with:
103100
python-version-file: pyproject.toml
104101

105-
- name: Install Poetry
106-
run: pip install poetry
107-
108102
- name: Install Task
109103
uses: arduino/setup-task@v2
110104
with:

‎workflow-templates/check-yaml-task.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ The code style defined in this file is the official standardized style to be use
3030
Add the tool dependency using this command:
3131

3232
```
33-
poetry add --dev "yamllint@^1.37.1"
33+
task poetry:install && poetry add --dev "yamllint@^1.37.1"
3434
```
3535

3636
Commit the resulting changes to the `pyproject.toml` and `poetry.lock` files.

‎workflow-templates/check-yaml-task.yml

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -99,9 +99,6 @@ jobs:
9999
with:
100100
python-version-file: pyproject.toml
101101

102-
- name: Install Poetry
103-
run: pip install poetry
104-
105102
- name: Install Task
106103
uses: arduino/setup-task@v2
107104
with:

‎workflow-templates/deploy-cobra-mkdocs-versioned-poetry.yml

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -68,11 +68,6 @@ jobs:
6868
with:
6969
python-version-file: pyproject.toml
7070

71-
- name: Install Poetry
72-
run: |
73-
python -m pip install --upgrade pip
74-
python -m pip install poetry
75-
7671
- name: Install Task
7772
uses: arduino/setup-task@v2
7873
with:

‎workflow-templates/deploy-mkdocs-poetry.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ Install the [`deploy-mkdocs-poetry.yml`](deploy-mkdocs-poetry.yml) GitHub Action
2626
Add the tool dependencies using this command:
2727

2828
```
29-
poetry add --dev "mkdocs@^1.3.0" "mkdocs-material@^8.2.11" "mdx_truly_sane_lists@^1.2"
29+
task poetry:install && poetry add --dev "mkdocs@^1.3.0" "mkdocs-material@^8.2.11" "mdx_truly_sane_lists@^1.2"
3030
```
3131

3232
Commit the resulting changes to the `pyproject.toml` and `poetry.lock` files.

‎workflow-templates/deploy-mkdocs-poetry.yml

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -38,9 +38,6 @@ jobs:
3838
repo-token: ${{ secrets.GITHUB_TOKEN }}
3939
version: 3.x
4040

41-
- name: Install Poetry
42-
run: pip install poetry
43-
4441
- name: Install Dependencies
4542
run: |
4643
task \

‎workflow-templates/deploy-mkdocs-versioned-poetry.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ See the ["Deploy Website" workflow (MkDocs, Poetry) documentation](deploy-mkdocs
3333

3434
1. Run this command:
3535
```
36-
poetry add --dev "gitpython@^3.1.44" "mike@^1.1.2"
36+
task poetry:install && poetry add --dev "gitpython@^3.1.44" "mike@^1.1.2"
3737
```
3838
1. Commit the resulting changes to the `pyproject.toml` and `poetry.lock` files.
3939

‎workflow-templates/deploy-mkdocs-versioned-poetry.yml

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -65,11 +65,6 @@ jobs:
6565
repo-token: ${{ secrets.GITHUB_TOKEN }}
6666
version: 3.x
6767

68-
- name: Install Poetry
69-
run: |
70-
python -m pip install --upgrade pip
71-
python -m pip install poetry
72-
7368
- name: Install Dependencies
7469
run: |
7570
task \

‎workflow-templates/spell-check-task.yml

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -53,9 +53,6 @@ jobs:
5353
with:
5454
python-version-file: pyproject.toml
5555

56-
- name: Install Poetry
57-
run: pip install poetry
58-
5956
- name: Install Task
6057
uses: arduino/setup-task@v2
6158
with:

‎workflow-templates/test-go-integration-task.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ Install the [`test-go-integration-task.yml`](test-go-integration-task.yml) GitHu
3232
Add the tool dependencies using this command:
3333

3434
```
35-
poetry add --dev "pytest@^8.4.1" "invoke@^1.7.0"
35+
task poetry:install && poetry add --dev "pytest@^8.4.1" "invoke@^1.7.0"
3636
```
3737

3838
Commit the resulting changes to the `pyproject.toml` and `poetry.lock` files.

‎workflow-templates/test-go-integration-task.yml

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -84,9 +84,6 @@ jobs:
8484
with:
8585
python-version-file: pyproject.toml
8686

87-
- name: Install Poetry
88-
run: pip install poetry
89-
9087
- name: Install Task
9188
uses: arduino/setup-task@v2
9289
with:

‎workflow-templates/test-python-poetry-task.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ Install the [`test-python-poetry-task.yml`](test-python-poetry-task.yml) GitHub
3030
Add the tool dependency using this command:
3131

3232
```
33-
poetry add --dev "pytest@^8.4.1"
33+
task poetry:install && poetry add --dev "pytest@^8.4.1"
3434
```
3535

3636
Commit the resulting changes to the `pyproject.toml` and `poetry.lock` files.

‎workflow-templates/test-python-poetry-task.yml

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -67,9 +67,6 @@ jobs:
6767
with:
6868
python-version-file: pyproject.toml
6969

70-
- name: Install Poetry
71-
run: pip install poetry
72-
7370
- name: Install Task
7471
uses: arduino/setup-task@v2
7572
with:

0 commit comments

Comments
 (0)