CMake build system

[jhunkeler]

This PR replaces GNU Autotools with CMake.

I encountered a couple issues that might be a bit out of scope:

1. munit might be incompatible with the latest macOS SDK on 15

[ 17%] Building C object tests/CMakeFiles/munit.dir/munit/munit.c.o
In file included from /Applications/Xcode_16.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework/Headers/Metadata.h:9,
                 from /Applications/Xcode_16.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/CoreServices.framework/Headers/CoreServices.h:43,
                 from /Users/runner/work/libasdf/libasdf/tests/munit/munit.c:507:
/Applications/Xcode_16.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework/Headers/MDItem.h:178:69: error: expected ')' before '^' token
  178 | MD_EXPORT void MDItemGetCacheFileDescriptors(CFArrayRef items, void(^completionHandler)(CFArrayRef array) ) API_AVAILABLE( macos(15.2) ) API_UNAVAILABLE( ios, tvos, watchos );
      |                                                                     ^
      |                                                                     )
make[2]: *** [tests/CMakeFiles/munit.dir/munit/munit.c.o] Error 1
make[1]: *** [tests/CMakeFiles/munit.dir/all] Error 2
make: *** [all] Error 2

2. A memory leak was detected in test-ndarray.c

=================================================================
==5447==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 224 byte(s) in 6 object(s) allocated from:
    #0 0x7f5962cfd9c7 in malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x55fa2b9b71ee in asdf_ndarray_read_tile_ndim (/home/runner/work/libasdf/libasdf/build/tests/test-ndarray+0x191ee) (BuildId: 59c7beed5f48dad8e29ba5356657af984e06a50a)

SUMMARY: AddressSanitizer: 224 byte(s) leaked in 6 allocation(s).
Error: child exited with status 1
3 of 4 (75%) tests successful, 0 (0%) test skipped.
<end of output>
Test time =   0.33 sec
----------------------------------------------------------
Test Failed.

[embray]

FWIW would it be possible to keep the two build systems side-by-side for a little while (not delete the automake stuff)? Longer term obviously better to settle on just one and I'm fine if it's cmake, but while things are under development I'd also prefer to keep the build system I'm more comfortable with.

[embray]

A memory leak was detected in test-ndarray.c

Thanks for the catch, I'll look into that. Usually I run the tests with ASAN enabled but I might have missed one. Maybe I'll add that to the CI as well.

[embray]

Ok, I see you already did that here 👍

[jhunkeler]

FWIW would it be possible to keep the two build systems side-by-side for a little while (not delete the automake stuff)?

This sounds good to me. It should only take a couple reverts to get this back to a working state with both build systems, so I'll do that now.

[embray]

oops, that's weird--I wonder why the build failed on my last commit, when it was passing before.

[embray]

I guess maybe at some point you rebased on main? Anyways I just rebased as well and added a new source file to target_sources, now it should work again.

[embray]

Thing I seem to have learned, unless I'm missing something. ISTM CPack doesn't have some equivalent of make distcheck. But to me this is really important! How can you be confident that your source distribution is correct without a way to test that unpacking and building from it works?

Likewise make package_source seems to just tar up everything in the source directory whether it's needed or not. I see there's a CPACK_SOURCE_IGNORE_FILES but this blacklisting approach seems backwards to me. The advantage of autotools' make dist is it's very conservative about what it includes--only files that the build system knows are actually needed, plus anything else explicitly whitelisted. I don't want my source distribution to include everything and then have to explicitly exclude stuff that shouldn't be there. This is exactly backwards, because it's so easy to miss build garbage that ends up in the tarball (that's how dumb things like .DS_Store files and things like that end up in source releases). I only want it to include what I've explicitly deemed necessary to include.

I'm looking into it because I don't want to give up on this approach just because of that.

[embray]

Nice--I got the "weird" tests working, and I'm actually a bit happier with how they work under cmake.

Now my only quibble is with the source distribution, but I've been doing research an I think I understand the differences in philosophy a little better. CPack is in of itself a standalone program that can just be invoked through cmake, I get that. Its strength seems to be more in its ability to generate different kinds of binary packages. It's not really as worried about "pristine" source distributions the way GNU is.

It still seems to be a shame though; I think all the necessary information is available in cmake's build tree to determine what files are needed for a reproducible build, so to whitelist rather than throw the whole kitchen sink in.

I won't worry about it too much for now as long as the autotools build system is coexisting, but I wonder if I could cobble together some way to emulate make dist and make distcheck with it....

[jhunkeler]

I won't worry about it too much for now as long as the autotools build system is coexisting, but I wonder if I could cobble together some way to emulate make dist and make distcheck with it....

I did emulate it for hstcal before ever realizing the package_source target existed. Here:
https://github.com/spacetelescope/hstcal/blob/0b13716dd50423faa966d0ae727ce3e123608c02/cmake/DistinfoArchive.cmake#L8

There I used git archive [..] --add-file to inject the generated DISTINFO file into the pristine tarball. I'm not opposed to doing something else, I only went with cpack because it looked convenient.

EDIT:

This is the custom target that builds the archive:
https://github.com/spacetelescope/hstcal/blob/0b13716dd50423faa966d0ae727ce3e123608c02/CMakeLists.txt#L29-L36

So implementing distcheck might (hopefully) be as simple as creating a custom target that DEPENDS on dist, and just runs the commands necessary to unpack, build, test, and install to a temporary directory. 🤞

[embray]

Probably could be--it's somewhat useful. Though since #50 there is a more robust asdf_datatype_t type attached to each ndarray which also has on it the byte size of the datatype (including support for strings, tables, etc.)

[embray]

Oops, I think you accidentally committed the compression stuff to the same branch. I'll split it off to its own branch. I'm not sure I want to merge the example code but this is still really useful stuff to have as proof-of-concept.

I think I'm happy enough now to merge the version of this from the other day as-is. I have some ideas for the make dist/distcheck stuff but as long as the autotools support is still there for now I'm not in a rush about it.

[jhunkeler]

That's fair, the compression stuff is way out of scope. @nden had asked me to whip up a working example to pass along to some folks that wanted to read data from a Level 3 Roman ASDF file using libasdf. Everything I was handed to play with was compressed with LZ4 so that's how I ended up going down the rabbit hole.

I'll just drop the compression commits from this branch and open a new PR to implement the cmake side of the compression stuff without the example code. You'll need to link to these libraries eventually anyway.