Skip to content

🧪 Pre-build armv7l images for building wheels #719

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 7 commits into
base: devel
Choose a base branch
from

Conversation

webknjaz
Copy link
Member

@webknjaz webknjaz commented May 9, 2025

These are relatively recent upstream and only exist for glibc 2.31+.

SUMMARY

$sbj.

ISSUE TYPE
  • Maintenance Pull Request
  • Packaging Pull Request
ADDITIONAL INFORMATION

Blocks #648.

@webknjaz webknjaz self-assigned this May 9, 2025
Copy link

Congratulations! One of the builds has completed. 🍾

You can install the built RPMs by following these steps:

  • sudo yum install -y dnf-plugins-core on RHEL 8
  • sudo dnf install -y dnf-plugins-core on Fedora
  • dnf copr enable packit/ansible-pylibssh-719
  • And now you can install the packages.

Please note that the RPMs should be used only in a testing environment.

@webknjaz webknjaz force-pushed the maintenance/gha-qemu-armv7l branch 2 times, most recently from 22b3e7e to 976251d Compare May 9, 2025 17:23
@@ -12,6 +12,7 @@
RUN ./install_libffi.sh "${RELEASE}"
ADD install_openssl.sh /root/install_openssl.sh
ADD openssl-version.sh /root/openssl-version.sh
ADD list-util-pairs-25367.patch /root/list-util-pairs-25367.patch

Check notice

Code scanning / SonarCloud

Prefer COPY over ADD for copying local resources

<!--SONAR_ISSUE_KEY:AZa2EmCFH_fKJUCx6ZIR-->Replace this ADD instruction with a COPY instruction. <p>See more on <a href="https://sonarcloud.io/project/issues?id=ansible_pylibssh&issues=AZa2EmCFH_fKJUCx6ZIR&open=AZa2EmCFH_fKJUCx6ZIR&pullRequest=719">SonarQube Cloud</a></p>
@webknjaz
Copy link
Member Author

@Jakuje any idea if it's possible to stop linking against libdl? https://github.com/ansible/pylibssh/actions/runs/14943974923/job/41986747379?pr=719#step:4:4032. Ubuntu Focal doesn't ship libdl.so.2 and the 2_31 containers (the first ones to ship armv7l) are based on it.

@Jakuje
Copy link
Contributor

Jakuje commented May 13, 2025

The libdl (dlopen, ...) is not needed directly by the libssh, but whole openssl is now based on dlopened modules (providers) so I assume this is transitive dependency.

That sounds weird that this would not be shipped in the recent ubuntu. Isn't it just somewhere else or accessible somehow differently? What were the errors you got when you tried to link with dl?

@webknjaz webknjaz force-pushed the maintenance/gha-qemu-armv7l branch from bd0b7c3 to 7ce622c Compare June 6, 2025 17:13
@webknjaz
Copy link
Member Author

webknjaz commented Jun 6, 2025

I don't remember exactly, I've rebased this PR and this job will hopefully reveal the problem: https://github.com/ansible/pylibssh/actions/runs/15496056920/job/43632975971?pr=719.

If not, it might be a good idea to start bumping OpenSSL separately.

@webknjaz webknjaz force-pushed the maintenance/gha-qemu-armv7l branch from 7ce622c to d509fed Compare June 6, 2025 17:17
@webknjaz
Copy link
Member Author

webknjaz commented Jun 6, 2025

@Jakuje here it is: https://github.com/ansible/pylibssh/actions/runs/15496056920/job/43632975971?pr=719#step:4:2990.

[ 62%] Building C object examples/CMakeFiles/libssh_scp.dir/connect_ssh.c.o
[ 63%] Linking C executable libssh_scp
/usr/bin/ld: ../lib/libssh.so.4.10.1: undefined reference to `dlopen'
/usr/bin/ld: ../lib/libssh.so.4.10.1: undefined reference to `dlclose'
/usr/bin/ld: ../lib/libssh.so.4.10.1: undefined reference to `dlerror'
/usr/bin/ld: ../lib/libssh.so.4.10.1: undefined reference to `dlsym'
/usr/bin/ld: ../lib/libssh.so.4.10.1: undefined reference to `dladdr'
collect2: error: ld returned 1 exit status
make[2]: *** [examples/CMakeFiles/libssh_scp.dir/build.make:149: examples/libssh_scp] Error 1
make[1]: *** [CMakeFiles/Makefile2:289: examples/CMakeFiles/libssh_scp.dir/all] Error 2
make: *** [Makefile:156: all] Error 2
Error: building at STEP "RUN ./install_libssh.sh "${LIBSSH_VERSION}"": while running runtime: exit status 2
Error: Error: buildah exited with code 2

@webknjaz
Copy link
Member Author

webknjaz commented Jun 6, 2025

I've only found ancient mentions of libdl in relation to OpenSSL @ https://patchwork.ozlabs.org/project/buildroot/patch/[email protected]/#819587

Comment on lines 1 to 2
export OPENSSL_VERSION="openssl-3.4.1"
export OPENSSL_SHA256="002a2d6b30b58bf4bea46c43bdd96365aaf8daa6c428782aa4feee06da197df3"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3.5 version was released quite recently and will be needed for support for PQC kex (in 0.12) so unless there is something blocking the update, I would jump to that one.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I was just attempting to sync what pyca/infra had in hopes that'd fix stuff. Bumping OpenSSL isn't the main objective here, but it'd be good to have a standalone PR dedicated to it.

@webknjaz
Copy link
Member Author

webknjaz commented Jun 6, 2025

Apparently, there's no image for manylinux_2_34_armv7l.

@webknjaz webknjaz force-pushed the maintenance/gha-qemu-armv7l branch from 7754035 to 6205bbd Compare June 9, 2025 20:57
Jakuje added a commit to Jakuje/pylibssh that referenced this pull request Jun 10, 2025
This also changes the way how the libssh finds the OpenSSL installation
to use the `OPENSSL_ROOT_DIR` variable.

Based partially on the changes in ansible#719.

Signed-off-by: Jakub Jelen <[email protected]>
Jakuje added a commit to Jakuje/pylibssh that referenced this pull request Jun 10, 2025
This also changes the way how the libssh finds the OpenSSL installation
to use the `OPENSSL_ROOT_DIR` variable.

Based partially on the changes in ansible#719.

Signed-off-by: Jakub Jelen <[email protected]>
Jakuje added a commit to Jakuje/pylibssh that referenced this pull request Jun 10, 2025
This also changes the way how the libssh finds the OpenSSL installation
to use the `OPENSSL_ROOT_DIR` variable.

Based partially on the changes in ansible#719.

Signed-off-by: Jakub Jelen <[email protected]>
Jakuje added a commit to Jakuje/pylibssh that referenced this pull request Jun 10, 2025
This also changes the way how the libssh finds the OpenSSL installation
to use the `OPENSSL_ROOT_DIR` variable.

Based partially on the changes in ansible#719.

Signed-off-by: Jakub Jelen <[email protected]>
Jakuje added a commit to Jakuje/pylibssh that referenced this pull request Jun 10, 2025
This also changes the way how the libssh finds the OpenSSL installation
to use the `OPENSSL_ROOT_DIR` variable.

Based partially on the changes in ansible#719.

Signed-off-by: Jakub Jelen <[email protected]>
Jakuje added a commit to Jakuje/pylibssh that referenced this pull request Jun 10, 2025
This also changes the way how the libssh finds the OpenSSL installation
to use the `OPENSSL_ROOT_DIR` variable.

Based partially on the changes in ansible#719.

Signed-off-by: Jakub Jelen <[email protected]>
Jakuje added a commit to Jakuje/pylibssh that referenced this pull request Jun 10, 2025
This also changes the way how the libssh finds the OpenSSL installation
to use the `OPENSSL_ROOT_DIR` variable.

Based partially on the changes in ansible#719, partially on current version of
https://github.com/pyca/infra/tree/main/cryptography-linux

The s390x issue was fixed with the suggestion from openssl/openssl#27323

The new openssl build takes longer (especially on ppc64le) so bumping
the timeout to 60minutes.

Signed-off-by: Jakub Jelen <[email protected]>
Jakuje added a commit to Jakuje/pylibssh that referenced this pull request Jun 10, 2025
This also changes the way how the libssh finds the OpenSSL installation
to use the `OPENSSL_ROOT_DIR` variable.

Based partially on the changes in ansible#719, partially on current version of
https://github.com/pyca/infra/tree/main/cryptography-linux

The s390x issue was fixed with the suggestion from openssl/openssl#27323

The new openssl build takes longer (especially on ppc64le) so bumping
the timeout to 50 minutes.

Signed-off-by: Jakub Jelen <[email protected]>
Jakuje added a commit to Jakuje/pylibssh that referenced this pull request Jun 10, 2025
This also changes the way how the libssh finds the OpenSSL installation
to use the `OPENSSL_ROOT_DIR` variable.

Based partially on the changes in ansible#719, partially on current version of
https://github.com/pyca/infra/tree/main/cryptography-linux

The s390x issue was fixed with the suggestion from openssl/openssl#27323

The new openssl build takes longer (especially on ppc64le) so bumping
the timeout to 50 minutes.

Signed-off-by: Jakub Jelen <[email protected]>
@webknjaz webknjaz force-pushed the maintenance/gha-qemu-armv7l branch 3 times, most recently from 0c1d5fd to 59c16a6 Compare June 16, 2025 14:01
@webknjaz webknjaz force-pushed the maintenance/gha-qemu-armv7l branch from 59c16a6 to 7583b14 Compare June 17, 2025 15:05
alex and others added 7 commits June 17, 2025 17:56
* build armv7l images

* Update build-docker-images.yml

* Don't use `enable-ec_nistp_64_gcc_128` on armv7l

* hack?

* We are an armv4 build

(cherry picked from commit 519b16bd0803ea22dbd0a4525c641060581c0215)
These are relatively recent upstream and only exist for glibc 2.31.
@webknjaz webknjaz force-pushed the maintenance/gha-qemu-armv7l branch from 7583b14 to 1585685 Compare June 17, 2025 15:57
Copy link

@webknjaz
Copy link
Member Author

This is solved by #740, but I'll keep the PR open for a bit and extract some commits separately before closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants