Skip to content

Cmake build system implementation #525

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

madjesc
Copy link

@madjesc madjesc commented May 28, 2025

Summary of changes

This pull request includes following changes or fixes.

Description

Added CMakeLists.txt for cmake to work and do some changes in code to not use config.h

Type of change

  • New feature (non-breaking change which adds functionality)

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have confirmed my fix is effective or that my feature works

Progress

  • add CMakeLists.txt to every part of the code
  • make it compile
  • make it installs
  • make it generate the config files
  • make it generate doc files and man pages
  • do compile time tests against autotools

@madjesc madjesc self-assigned this May 28, 2025
@madjesc madjesc force-pushed the build/cmake branch 3 times, most recently from f44df7f to 392aa69 Compare May 28, 2025 23:46
@madjesc
Copy link
Author

madjesc commented May 29, 2025

Also, for some reason ICU cannot find the resource bundles even thought they are linked to the binary. I'm probably missing a compile flag, feature or something. Any idea on that?

[mcardenas@kasane build]$ ./src/ltfs
LTFS11286E Cannot load messages: failed to open resource bundle (2)
LTFS11293E Cannot load messages for libltfs (-1)
LTFS9011E Logging initialization failed
LTFS10000E (could not generate message)
LTFS11286E Cannot load messages: failed to open resource bundle (2)
LTFS10012E (could not generate message)

@piste-jp
Copy link
Member

piste-jp commented May 29, 2025

Also, for some reason ICU cannot find the resource bundles even thought they are linked to the binary. I'm probably missing a compile flag, feature or something. Any idea on that?

[mcardenas@kasane build]$ ./src/ltfs
LTFS11286E Cannot load messages: failed to open resource bundle (2)
LTFS11293E Cannot load messages for libltfs (-1)
LTFS9011E Logging initialization failed
LTFS10000E (could not generate message)
LTFS11286E Cannot load messages: failed to open resource bundle (2)
LTFS10012E (could not generate message)

Did you specify static link against resource bundle created by ICU?

@madjesc
Copy link
Author

madjesc commented May 29, 2025

Also, for some reason ICU cannot find the resource bundles even thought they are linked to the binary. I'm probably missing a compile flag, feature or something. Any idea on that?

[mcardenas@kasane build]$ ./src/ltfs
LTFS11286E Cannot load messages: failed to open resource bundle (2)
LTFS11293E Cannot load messages for libltfs (-1)
LTFS9011E Logging initialization failed
LTFS10000E (could not generate message)
LTFS11286E Cannot load messages: failed to open resource bundle (2)
LTFS10012E (could not generate message)

Did you specify static link against resource bundle created by ICU?

It's dinamically linked, libltfs is compiled as a dynamic lib, the ICU messages bundles are compiled statically and linked into libltfs. libltfs should be a static lib?

@piste-jp
Copy link
Member

Did you specify static link against resource bundle created by ICU?

It's dinamically linked, libltfs is compiled as a dynamic lib, the ICU messages bundles are compiled statically and linked into libltfs. libltfs should be a static lib?

No. The libltfs using in ltfs, mkltfs etc. is dynamic link library. What I'm saying is resource bundle created by ICU, like liblibltfs_dat.a is statically linked to a correct shared library.

For example, below is the build log of libltfs.so on auto tools. You can see ../../messages/liblibltfs_dat.a ../../messages/libinternal_error_dat.a ../../messages/libtape_common_dat.a is specified as object files. This command inject the message bundle into the libltfs.so

libtool: link: gcc -shared  -fPIC -DPIC  .libs/libltfs_la-ltfs.o .libs/libltfs_la-ltfs_internal.o .libs/libltfs_la-ltfs_fsops.o .libs/libltfs_la-ltfs_fsops_raw.o .libs/libltfs_la-ltfssnmp.o .libs/libltfs_la-fs.o .libs/libltfs_la-xml_common.o .libs/libltfs_la-xml_writer.o .libs/libltfs_la-xml_reader.o .libs/libltfs_la-xml_writer_libltfs.o .libs/libltfs_la-xml_reader_libltfs.o .libs/libltfs_la-label.o .libs/libltfs_la-base64.o .libs/libltfs_la-tape.o .libs/libltfs_la-iosched.o .libs/libltfs_la-dcache.o .libs/libltfs_la-kmi.o .libs/libltfs_la-pathname.o .libs/libltfs_la-index_criteria.o .libs/libltfs_la-xattr.o .libs/libltfs_la-ltfslogging.o .libs/libltfs_la-ltfstrace.o .libs/libltfs_la-ltfs_thread.o .libs/libltfs_la-config_file.o .libs/libltfs_la-plugin.o .libs/libltfs_la-periodic_sync.o .libs/libltfs_la-inc_journal.o arch/.libs/libltfs_la-uuid_internal.o arch/.libs/libltfs_la-filename_handling.o arch/.libs/libltfs_la-time_internal.o arch/.libs/libltfs_la-arch_info.o arch/.libs/libltfs_la-errormap.o   -ldl -lfuse -luuid -lxml2 -licui18n -licuuc -licudata -L/usr/lib/x86_64-linux-gnu -lnetsnmpmibs -lsensors -lpci -lnetsnmpagent -lwrap -lnetsnmp -lm -lssl -lcrypto ../../messages/liblibltfs_dat.a ../../messages/libinternal_error_dat.a ../../messages/libtape_common_dat.a -lrt -lpthread  -g -O2 -flto=auto -flto=auto -fstack-protector-strong -g -O2 -flto=auto -flto=auto -fstack-protector-strong -g -O2 -O0 -ggdb -Wl,--no-undefined -Wl,--as-needed -pthread -Wl,-Bsymbolic-functions -flto=auto -flto=auto -Wl,-z -Wl,relro -Wl,-z -Wl,now -Wl,-E -Wl,-Bsymbolic-functions -flto=auto -flto=auto -Wl,-z -Wl,relro -Wl,-z -Wl,now   -pthread -Wl,-soname -Wl,libltfs.so.0 -o .libs/libltfs.so.0.0.0

You can confirm the symbol table of libltfs.so. It must have resource bundle as value like below.

nm ./libltfs.so | grep _dat$
00000000000b20d0 R internal_error_dat
000000000009df90 R libltfs_dat
00000000000b9f00 R tape_common_dat

@madjesc
Copy link
Author

madjesc commented May 30, 2025

No. The libltfs using in ltfs, mkltfs etc. is dynamic link library. What I'm saying is resource bundle created by ICU, like liblibltfs_dat.a is statically linked to a correct shared library.

For example, below is the build log of libltfs.so on auto tools. You can see ../../messages/liblibltfs_dat.a ../../messages/libinternal_error_dat.a ../../messages/libtape_common_dat.a is specified as object files. This command inject the message bundle into the libltfs.so

libtool: link: gcc -shared  -fPIC -DPIC  .libs/libltfs_la-ltfs.o .libs/libltfs_la-ltfs_internal.o .libs/libltfs_la-ltfs_fsops.o .libs/libltfs_la-ltfs_fsops_raw.o .libs/libltfs_la-ltfssnmp.o .libs/libltfs_la-fs.o .libs/libltfs_la-xml_common.o .libs/libltfs_la-xml_writer.o .libs/libltfs_la-xml_reader.o .libs/libltfs_la-xml_writer_libltfs.o .libs/libltfs_la-xml_reader_libltfs.o .libs/libltfs_la-label.o .libs/libltfs_la-base64.o .libs/libltfs_la-tape.o .libs/libltfs_la-iosched.o .libs/libltfs_la-dcache.o .libs/libltfs_la-kmi.o .libs/libltfs_la-pathname.o .libs/libltfs_la-index_criteria.o .libs/libltfs_la-xattr.o .libs/libltfs_la-ltfslogging.o .libs/libltfs_la-ltfstrace.o .libs/libltfs_la-ltfs_thread.o .libs/libltfs_la-config_file.o .libs/libltfs_la-plugin.o .libs/libltfs_la-periodic_sync.o .libs/libltfs_la-inc_journal.o arch/.libs/libltfs_la-uuid_internal.o arch/.libs/libltfs_la-filename_handling.o arch/.libs/libltfs_la-time_internal.o arch/.libs/libltfs_la-arch_info.o arch/.libs/libltfs_la-errormap.o   -ldl -lfuse -luuid -lxml2 -licui18n -licuuc -licudata -L/usr/lib/x86_64-linux-gnu -lnetsnmpmibs -lsensors -lpci -lnetsnmpagent -lwrap -lnetsnmp -lm -lssl -lcrypto ../../messages/liblibltfs_dat.a ../../messages/libinternal_error_dat.a ../../messages/libtape_common_dat.a -lrt -lpthread  -g -O2 -flto=auto -flto=auto -fstack-protector-strong -g -O2 -flto=auto -flto=auto -fstack-protector-strong -g -O2 -O0 -ggdb -Wl,--no-undefined -Wl,--as-needed -pthread -Wl,-Bsymbolic-functions -flto=auto -flto=auto -Wl,-z -Wl,relro -Wl,-z -Wl,now -Wl,-E -Wl,-Bsymbolic-functions -flto=auto -flto=auto -Wl,-z -Wl,relro -Wl,-z -Wl,now   -pthread -Wl,-soname -Wl,libltfs.so.0 -o .libs/libltfs.so.0.0.0

You can confirm the symbol table of libltfs.so. It must have resource bundle as value like below.

nm ./libltfs.so | grep _dat$
00000000000b20d0 R internal_error_dat
000000000009df90 R libltfs_dat
00000000000b9f00 R tape_common_dat

The symbols exists in the library

[mcardenas@kasane libltfs]$ nm ./libltfs.so | grep _dat$
00000000000959a0 R internal_error_dat
0000000000081830 R libltfs_dat
000000000009d800 R tape_common_dat

And cmake compiles it like this

/usr/bin/cc -fPIC -pthread -shared -Wl,-soname,libltfs.so -o libltfs.so CMakeFiles/libltfs.dir/arch/arch_info.c.o CMakeFiles/libltfs.dir/arch/errormap.c.o CMakeFiles/libltfs.dir/arch/filename_handling.c.o CMakeFiles/libltfs.dir/arch/osx/osx_string.c.o CMakeFiles/libltfs.dir/arch/time_internal.c.o CMakeFiles/libltfs.dir/arch/uuid_internal.c.o CMakeFiles/libltfs.dir/base64.c.o CMakeFiles/libltfs.dir/config_file.c.o CMakeFiles/libltfs.dir/dcache.c.o CMakeFiles/libltfs.dir/fs.c.o CMakeFiles/libltfs.dir/inc_journal.c.o CMakeFiles/libltfs.dir/index_criteria.c.o CMakeFiles/libltfs.dir/iosched.c.o CMakeFiles/libltfs.dir/kmi.c.o CMakeFiles/libltfs.dir/label.c.o CMakeFiles/libltfs.dir/ltfs.c.o CMakeFiles/libltfs.dir/ltfs_fsops.c.o CMakeFiles/libltfs.dir/ltfs_fsops_raw.c.o CMakeFiles/libltfs.dir/ltfs_internal.c.o CMakeFiles/libltfs.dir/ltfs_thread.c.o CMakeFiles/libltfs.dir/ltfslogging.c.o CMakeFiles/libltfs.dir/ltfssnmp.c.o CMakeFiles/libltfs.dir/ltfstrace.c.o CMakeFiles/libltfs.dir/pathname.c.o CMakeFiles/libltfs.dir/periodic_sync.c.o CMakeFiles/libltfs.dir/plugin.c.o CMakeFiles/libltfs.dir/tape.c.o CMakeFiles/libltfs.dir/xattr.c.o CMakeFiles/libltfs.dir/xml_common.c.o CMakeFiles/libltfs.dir/xml_reader.c.o CMakeFiles/libltfs.dir/xml_reader_libltfs.c.o CMakeFiles/libltfs.dir/xml_writer.c.o CMakeFiles/libltfs.dir/xml_writer_libltfs.c.o  /usr/lib64/libxml2.so ../../messages/libbin_ltfs.a ../../messages/liblibltfs.a ../../messages/libinternal_error.a ../../messages/libtape_common.a -lpthread /usr/lib64/libfuse.so /usr/lib64/libuuid.so 

@piste-jp
Copy link
Member

piste-jp commented May 31, 2025

I didn't try your code at all yet. But does auto tools build on your tree work fine? If so, you need to investigate step by step and find differences.

Please do not forget that I never ask you to do this. You just want to do this.

@madjesc
Copy link
Author

madjesc commented Jun 1, 2025

I didn't try your code at all yet. But does auto tools build on your tree work fine? If so, you need to investigate step by step and find differences.

Please do not forget that I never ask you to do this. You just want to do this.

Yes, I know that I want to implement this, I'm not blaming anyone, I'm trying to get some advice for this. I'm trying to look at the differences between versions

@piste-jp
Copy link
Member

piste-jp commented Jun 1, 2025

I didn't try your code at all yet. But does auto tools build on your tree work fine? If so, you need to investigate step by step and find differences.
Please do not forget that I never ask you to do this. You just want to do this.

Yes, I know that I want to implement this, I'm not blaming anyone, I'm trying to get some advice for this. I'm trying to look at the differences between versions

The message says error code retuned from ICU is 2. It means U_MISSING_RESOURCE_ERROR like below. ICU might not be able to find resource bundle into libltfs_dat.

LTFS11286E Cannot load messages: failed to open resource bundle (2)
    U_MISSING_RESOURCE_ERROR  =  2,     /**< The requested resource cannot be found */

The root cause is how to create message library file. In auto tools, .res files in the paths.txt does not include res_${NAME}. But it has res_$NAME/ and pkgdata generate resource name as libltfs/res_libltfs/en_US.res. In this resource bundle, the resource name shall be libltfs/res_libltfs, but code specifies it as libltfs. So ICU cannot find a resource bundle correctly. (PLEASE inspect the both .a files, you can see the resource names at the beginning of the files....)

Try the patch below.

diff --git a/.gitignore b/.gitignore
index 0711b11..e36512a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -23,6 +23,12 @@ ltfs.pc
 .libs/
 messages/.lib
 .dirstamp
+# Files renerated by CMake
+CMakeFiles
+*.cmake
+CMakeCache.txt
+install_manifest.txt
+*.so
 # Files created by OS
 .DS_Store
 # Files for development
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 5245b51..10130b2 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -1,8 +1,14 @@
 cmake_minimum_required(VERSION 3.20)
-project(ltfs,
+project(ltfs
   # TODO: Find a better solution to mark this version as different from the official build
   VERSION "2.5.0.9999"
+  LANGUAGES C
 )
+
+set(CMAKE_VERBOSE_MAKEFILE 1)
+set(CMAKE_C_COMPILER gcc)
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -ggdb -O0")
+
 include(CheckSymbolExists REQUIRED)
 include(CheckTypeSize REQUIRED)
 include(CheckLibraryExists REQUIRED)
diff --git a/messages/.gitignore b/messages/.gitignore
new file mode 100644
index 0000000..a445398
--- /dev/null
+++ b/messages/.gitignore
@@ -0,0 +1 @@
+res*
diff --git a/messages/CMakeLists.txt b/messages/CMakeLists.txt
index 46ef6c1..3b853b5 100644
--- a/messages/CMakeLists.txt
+++ b/messages/CMakeLists.txt
@@ -26,12 +26,13 @@ foreach(NAME IN LISTS MESSAGES_LIBS)
     OUTPUT "res_${NAME}/lib${NAME}.a"
     COMMAND mkdir -p res_${NAME}
     COMMAND ${ICU_GENRB_EXECUTABLE} -q ${SRCS} -d res_${NAME} 1> /dev/null
-    COMMAND find res_${NAME} -name *.res > res_${NAME}/paths.txt
-    COMMAND ${ICU_PKGDATA_EXECUTABLE} -d res_${NAME}/ -m static -p ${NAME} -q res_${NAME}/paths.txt 1> /dev/null
+    COMMAND cd res_${NAME} && ls *.res > paths.txt
+    COMMAND cd res_${NAME} && ${ICU_PKGDATA_EXECUTABLE} -m static -p ${NAME} -q paths.txt 1> /dev/null
     BYPRODUCTS "res_${NAME}"
   )
   # Add the command as a target to be build when all is invoked

Message resource problem is gone like below in my env.

[root@rocky9 ltfs-madjesc2]# ./src/ltfs
8c66 LTFS11268E Cannot open configuration file '/ltfs.conf' (-2).
8c66 LTFS10008E Failed to load the configuration file (-2).

Comment on lines 1 to 31
# NOTE: This can be done using an .in file with @VARS@ but somehow this is done using sed
set(PLAT_OPTS)
if(ENABLE_LINTAPE)
list(APPEND PLAT_OPTS "\\nplugin tape lin_tape ${libdir}/ltfs/libtape-lin_tape.so")
endif()

if(LINUX)
list(APPEND PLAT_OPTS "\\nplugin tape sg ${libdir}/ltfs/libtape-sg.so")
elseif(APPLE)
list(APPEND PLAT_OPTS "\\nplugin tape iokit ${libdir}/ltfs/libtape-iokit.so")
elseif(BSD)
if (CMAKE_SYSTEM_NAME STREQUAL "FreeBSD")
list(APPEND PLAT_OPTS "\\nplugin tape cam ${libdir}/ltfs/libtape-cam.so")
elseif(CMAKE_SYSTEM_NAME STREQUAL "NetBSD")
list(APPEND PLAT_OPTS "\\nplugin tape scsipi-ibmtape ${libdir}/ltfs/libtape-scsipi-ibmtape.so")
endif()
endif()

file(REAL_PATH "ltfs.conf.in" CONFIG_FILE)
add_custom_command(OUTPUT "ltfs.conf"
COMMAND sed -e "s!__PLATFORM_DRIVERS__!${PLAT_OPTS}!" ${CONFIG_FILE} > .tmp1
COMMAND sed -e "s!__LIBDIR__!${CMAKE_INSTALL_FULL_LIBDIR}!" .tmp1 > .tmp2
COMMAND sed -e "s!__DEFAULT_TAPE__!${DEFAULT_TAPE}!" .tmp2 > .tmp1
COMMAND sed -e "s!__DEFAULT_IOSCHED__!${DEFAULT_IOSCHED}!" .tmp1 > .tmp2
COMMAND sed -e "s!__DEFAULT_KMI__!${DEFAULT_KMI}!" .tmp2 > .tmp1
COMMAND sed -e "s!__CONFDIR__!${CMAKE_INSTALL_FULL_SYSCONFDIR}!" .tmp1 > ltfs.conf
DEPENDS ${CONFIG_FILE}
BYPRODUCTS .tmp1 .tmp2
)
add_custom_target("conf" ALL DEPENDS "ltfs.conf")
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@piste-jp This works, but cannot this be done in a .in file?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks you are right. We might be able to use .in file instead.

@madjesc madjesc linked an issue Jun 9, 2025 that may be closed by this pull request
Copy link
Member

@piste-jp piste-jp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a description how to build the tree with cmake even if it is experimental support.

fix: CMake definition for cmake specific files

feat: add cmake lists to drivers and kmi

fix: revert unnecesary changes

fix: missing .c file

fix: typos and names

chore: add build specific version number and add comments

feat: add last cmakelists

fix: fix compilation issues

feat: fix names

feat: add manpages

feat: cmake working
@madjesc
Copy link
Author

madjesc commented Jul 12, 2025

Sorry for the delay, I was occupied with other stuff.
Whatever, now this works correctly, I tested it using the file backend and it works as expected

Please add a description how to build the tree with cmake even if it is experimental support.

Should I add it to the README?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add cmake support
2 participants