make check is blocking on several tests

[rokdun]

Hello, I'm building xmlsec 1.3.7 on Rocky Linux 8.9, using gcc 12, with openssl 3.0.15, libxml2 2.12.10, libxslt 1.1.43 and gcrypt 1.11.2
Build is OK and most tests ("make check") are OK but some fail and get stuck. I can kill the stuck process and the checking continues, but then get stuck again; after killing a few of them I gave up and interrupted the tests.

I tried to run one blocking test manually:

LD_LIBRARY_PATH=/usr/local/openssl/openssl-3.0.15/lib:/usr/local/libxml2/libxml2-2.12.10/lib:/build/xmlsec1-1.3.7/src/openssl/.libs:/build/xmlsec1-1.3.7/src/.libs \
/build/xmlsec1-1.3.7/apps/.libs/lt-xmlsec1 verify --X509-skip-strict-checks --verbose \
--crypto openssl --crypto-config /tmp/xmlsec-testDSig.sh-openssl-20250808_095841/crypto-config-115 \
--trusted-der /build/xmlsec1-1.3.7/tests/keys/cacert.der --enabled-key-data x509 \
--verification-gmt-time 2022-12-20+00:00:00 \
/build/xmlsec1-1.3.7/tests/aleksey-xmldsig-01/enveloping-expired-cert.xml

I get this output:

func=xmlSecOpenSSLX509StoreVerifyCert:file=x509vfy.c:line=745:obj=unknown:subj=unknown:error=76:certificate has expired:subject=/C=US/ST=California/O=XML Security Library (http:\/\/www.aleksey.com\/xmlsec)/OU=Test Expired RSA Certificate/CN=Aleksey Sanin/[email protected]; issuer=/C=US/ST=California/O=XML Security Library (http:\/\/www.aleksey.com\/xmlsec)/CN=Aleksey Sanin/[email protected]; err=10; msg=certificate has expired
func=xmlSecKeysMngrGetKey:file=keys.c:line=1344:obj=unknown:subj=unknown:error=45:key is not found:details=NULL
func=xmlSecDSigCtxProcessKeyInfoNode:file=xmldsig.c:line=822:obj=unknown:subj=unknown:error=45:key is not found:details=NULL
func=xmlSecDSigCtxProcessSignatureNode:file=xmldsig.c:line=537:obj=unknown:subj=xmlSecDSigCtxProcessKeyInfoNode:error=1:xmlsec library function failed:
func=xmlSecDSigCtxVerify:file=xmldsig.c:line=357:obj=unknown:subj=xmlSecDSigCtxProcessSignatureNode:error=1:xmlsec library function failed:
Verification status: FAILED
Failure reason: KEY-NOT-FOUND
SignedInfo References (ok/all): 0/0
Manifests References (ok/all): 0/0
Error: failed to verify file "/DT/DEV/dev/netloc/xmlsec1-1.3.7/tests/aleksey-xmldsig-01/enveloping-expired-cert.xml"

but then, the process is stuck, it doesn't exit. I must hit Ctrl-C to abort.

Is this an already known issue?
Thanks in advance for any help...

[lsh123]

Something is not right with openssl. Not sure if RockyLinux does any openssl patches but this test can't fail with "certificate has expired" error because the specific (valid) verification time is passed to the test in the command line: "--verification-gmt-time 2022-12-20+00:00:00"

I would recommend to try and build xmlsec against custom built openssl downloaded from openssl.org website.

Re hangs, I would recommend to attach debugger to the hanging process and then see what is going on. Again, it smells like something is hanging during shutdown / cleanup and openssl is the most likely option.

[lsh123]

Sorry for delay, I got RockyLinux installed and I can't reproduce the issue using the latest versions of all packages. Please consider upgrading to the latest versions:

### RockyLinux

$ cat /etc/os-release
NAME="Rocky Linux"
VERSION="8.10 (Green Obsidian)"
ID="rocky"
ID_LIKE="rhel centos fedora"
VERSION_ID="8.10"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Rocky Linux 8.10 (Green Obsidian)"
ANSI_COLOR="0;32"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:rocky:rocky:8:GA"
HOME_URL="https://rockylinux.org/"
BUG_REPORT_URL="https://bugs.rockylinux.org/"
SUPPORT_END="2029-05-31"
ROCKY_SUPPORT_PRODUCT="Rocky-Linux-8"
ROCKY_SUPPORT_PRODUCT_VERSION="8.10"
REDHAT_SUPPORT_PRODUCT="Rocky Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.10"


### OpenSSL 3.5.2

$ scl enable gcc-toolset-12 bash
$ ./Configure --prefix=$HOME/local/openssl
$ make -j8
$ make install

### LibXML2 2.14.5

$ scl enable gcc-toolset-12 bash
$ ./autogen.sh --without-python --prefix=$HOME/local/libxml2
$ make -j8
$ make install

### LibXSLT 1.1.43

$ scl enable gcc-toolset-12 bash
$ ./autogen.sh --without-python --prefix=$HOME/local/libxslt --with-libxml-prefix=$HOME/local/libxml2
$ make -j8
$ make install


### XMLSec (master)

$ scl enable gcc-toolset-12 bash
$ ./autogen.sh --prefix=$HOME/local/xmlsec --with-openssl=$HOME/local/openssl --with-libxml2=$HOME/local/libxml2 --with-libxslt=$HOME/local/libxslt
$ make -j8
$ make install


### Running XMLSec tests

$ scl enable gcc-toolset-12 bash
$ export LD_LIBRARY_PATH=$HOME/local/openssl/lib64:$HOME/local/libxml2/lib:$HOME/local/libxslt/lib:$LD_LIBRARY_PATH
$ echo $LD_LIBRARY_PATH
$ make check
...
--- testKeys finished ---
--- detailed log is written to  /tmp/xmlsec-testKeys.sh-openssl-20250812_100246/full.log ---
--- TOTAL OK: 40; OK (percent): 100; TOTAL FAILED: 0; TOTAL SKIPPED: 0
...
--- testDSig finished
--- detailed log is written to  /tmp/xmlsec-testDSig.sh-openssl-20250812_100250/full.log
--- TOTAL OK: 382; OK (percent): 96; TOTAL FAILED: 0; TOTAL SKIPPED: 12
...
--- testEnc finished
--- detailed log is written to  /tmp/xmlsec-testEnc.sh-openssl-20250812_100334/full.log
--- TOTAL OK: 689; OK (percent): 99; TOTAL FAILED: 0; TOTAL SKIPPED: 4

[rokdun]

Thanks a lot for the time you took to perform those tests!
Well, I'm in an uncomfortable situation regarding dependencies versions:

• I must use libxml2 2.12.10, because we use a Perl module, "XML::LibXML", which is not compatible with libxml2 2.13 and later :-(
• we use OpenSSL 3.0.15 but we should upgrade anyway -- in fact we never use OpenSSL provided by the operating system as it's never up-to-date.

I will perform more tests, upgrading OpenSSL and libxml2, and see how it goes. Thanks again!

[lsh123]

I switched to older versions of OpenSSL (3.0.17) and LibXML2 (2.12.10) using same configure/ build commands as above:

$ ls -la $HOME/local
total 4
drwxrwxr-x.  8 aleksey aleksey  175 Aug 13 09:35 .
drwx------. 17 aleksey aleksey 4096 Aug 13 09:31 ..
lrwxrwxrwx.  1 aleksey aleksey   15 Aug 13 09:34 libxml2 -> libxml2-2.12.10
drwxrwxr-x.  6 aleksey aleksey   56 Aug 13 09:33 libxml2-2.12.10
drwxrwxr-x.  6 aleksey aleksey   56 Aug 12 09:52 libxml2-2.14.5
lrwxrwxrwx.  1 aleksey aleksey   15 Aug 13 09:35 libxslt -> libxslt-1.1.43/
drwxrwxr-x.  6 aleksey aleksey   56 Aug 12 09:56 libxslt-1.1.43
lrwxrwxrwx.  1 aleksey aleksey   14 Aug 13 09:34 openssl -> openssl-3.0.17
drwxrwxr-x.  7 aleksey aleksey   69 Aug 13 09:32 openssl-3.0.17
drwxrwxr-x.  7 aleksey aleksey   69 Aug 12 09:45 openssl-3.5.2
drwxr-xr-x.  6 aleksey aleksey   56 Aug 12 10:00 xmlsec

$ echo $LD_LIBRARY_PATH
/home/aleksey/local/openssl/lib64:/home/aleksey/local/libxml2/lib:/home/aleksey/local/libxslt/lib:/opt/rh/gcc-toolset-12/root/usr/lib64:/opt/rh/gcc-toolset-12/root/usr/lib

And results are the same --> all tests pass and there are no hangs.

So I would personally try the following:
1/ Check LD_LIBRARY_PATH and LTDL_LIBRARY_PATH to make sure the right libraries are loaded during test run
2/ Attach debugger to the hanging process to see where is it hanging
3/ Run hanging test with valgrind to see if there is some memory corruption or something

[rokdun]

Thanks again for making more tests. On my side I switched to the "master" version of xmlsec and libxml2 2.14.5 and the tests are currently running with no hangs (the tests take a long time, but it's on a virtual machine and we do have lots of performance issues).

In any case it looks like my problem is not with the libxml2 version. I'll have to rebuild xmlsec 1.3.7 as I'm starting to think something went wrong during the build as you haven't found any issue on your side... And there's no reason this problem would go unnoticed. Especially since we see both unexpected errors reported by the tests, and hangs. That's a lot.

Regarding the stack trace when the process is stuck, I took one using gdb, unfortunately this in not built in debug mode so it's not very informative:

(gdb) bt
#0  0x00007ffff66d185d in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007ffff66caad9 in pthread_mutex_lock () from /lib64/libpthread.so.0
#2  0x00007ffff7fd6e00 in xsltUnregisterAllExtModuleFunction () at extensions.c:1516
#3  xsltCleanupGlobals () at extensions.c:2317
#4  0x0000000000408f0c in xmlSecAppShutdown () at xmlsec.c:3133
#5  0x000000000040950d in xmlSecAppExecute (command=command@entry=xmlSecAppCommandVerify, utf8_argv=utf8_argv@entry=0x7fffffffda38,
    argc=argc@entry=1) at xmlsec.c:1544
#6  0x0000000000409646 in main (argc=15, argv=0x7fffffffd9c8) at xmlsec.c:1303

[lsh123]

Make sure you are building LibXSLT and XMLSec against correct version of LibXML2 and that the same version is loaded at runtime.

[rokdun]

Hello,

I've rebuilt xmlsec 1.3.7 from scratch and now the tests are OK (well they are still running, but they don't get stuck). I must have made a mistake when building the first time, probably a library version mismatch as you suggested...
I've also tried with xmlsec 1.2.42 since we have some code that is incompatible with the new version, tests seem OK too...

I'm very sorry for wasting your time, please accept my apologies and I can't thank you enough for looking into this issue.

[rokdun]

Sorry for the delay. It turns out tests failed on both version 1.2.42 and 1.3.7. They lasted looong (slow virtual machine), so I first thought they were OK.
I found the old versions 1.3.0 and 1.3.1 we built on CentOS 7, two years ago, "make check" runs OK!

The next step for me will be to restart from scratch, rebuilding the dependencies (ssl libxml2...) with the same versions as you. Meanwhile we continue to use the old version, it's no big deal.