Allow a possible installation of index-64 library alongside standard index-32 library?

[epsilon-0]

Currently debian has (some) separate headers for blas64 and cblas64 (though not from the reference implementation).

I am not sure if they are correct or not, with regards to the reference index64 API (they are from the blis library).

Would it be possible to add an option to cmake, something like BUILD_INDEX64, which is defaulted to OFF, but if has been turned on, it will create the index-64 libraries?
If I make a PR for such an option, would it be entertained as a possibility?

Some things which I had in mind for allowing this to co-exist with the standard installation - name the libraries to libblas64.so, libcblas64.so, liblapack64.so, liblapacke64.so, this way there is no conflict between the library names (though of course, you can't link with both libblas and libblas64 at the same time).
Also the library would need to be compiled twice, once for index32 and once for index64 (but that is a perfectly normal scenario and not a deal breaker).
The only conflict I am unable to think of a good resolution is the header file names.
If following debians style, it might be wise to call the c headers ending with 64 as well.
(I'm maintaining this for Gentoo and would like to keep the ecosystem very close to debian, so that we have minimal problems for developers switching between systems)

I'm open to any suggestions before I make the PR ❤️

Thanks,
Aisha

[langou]

Hi Aisha, that makes total sense to me, but let us see if we have feedback from others. So let us wait a few days. J

[epsilon-0]

Most definitely, that sounds like a good plan.

[epsilon-0]

OMG @langou
you are so fast ❤️

Just for completeness, I am writing down things that we still have to do:

• Figure out how to name headers so that 32bit API can co-exist with 64bit API
• Fix printf/fprintf statements so that they use correct qualifier for printing.

Any suggestions to solve the first point are welcome, I unfortunately don't have a "clean" solution.

[epsilon-0]

A couple of questions, which will help me handle the naming for the files

• There seems to be a tonne of duplicate definitions between cblas_f77.h and cblas_test.h. Do we really need that?
• Does cblas_test.h need to be installed? Given its name (and the files that it is used in) I presume it to be used only during the testing phase. Maybe we should not install this file on a system-wide level?

[christoph-conrads]

Hi @epsilon-0,

Some things which I had in mind for allowing this to co-exist with the standard installation - name the libraries to libblas64.so, libcblas64.so, liblapack64.so, liblapacke64.so, this way there is no conflict between the library names (though of course, you can't link with both libblas and libblas64 at the same time).

you might be looking for PR #218. The author of this PR is Björn Esser from the Fedora Project.

[weslleyspereira]

Hi @epsilon-0. Did #462 solve this issue?

[epsilon-0]

@weslleyspereira no this isn't complete yet.
There needs to be some more header renaming/handling done.
I am busy for the next few weeks so am not going to be able to do this soon.
Basic outline

• the header files should be called cblas.h and cblas64.h, likewise for other headers
  • this means the *.c files would need some slight adjustment to include the proper header, but this is only during build time, so it can be hacked out.
• the cmake files should be installed under lapack64 or cblas64, etc.

[weslleyspereira]

Ok, I see. Thanks for the quick follow-up!

[drhpc]

I have similar trouble trying to package things with pkgsrc. I would like to have a complete install of the reference, with cblas and lapacke. For differing implementations being installed at the same time, I settled for differing library names and sub-directories for the headers, so for example

/usr/lib/libopenblas.so
/usr/lib/libopenblas64.so
/usr/lib/libblas.so
/usr/lib/libcblas.so
/usr/lib/libblas64.so
/usr/lib/libcblas64.so
/usr/include/openblas/cblas.h
/usr/include/openblas64/cblas.h
/usr/include/netlib/cblas.h
/usr/include/netlib64/cblas.h
/usr/include/cblas.h -> netlib/cblas.h (for compatibility, having the default)

(and so forth)

We do not consider runtime switching like the binary distros, so it is OK if each cblas.h (and lapacke.h) is specific to its matched library, like with extra names for libopenblas. Build-time selection happens via

BLAS_INCLUDES=-I/prefix/include/netlib64
BLAS_LIBS=-lblas64
CBLAS_LIBS=-lcblas64

(etc.) That's what the .pc files are supposed to say, and it a lot easier than to communicate a different header file name. They are not yet consistent in that, but I am fixing it up. Seems so far people just hacked that in their distros, if bothering with all reference libs at all.

I got one question about those headers, though.

I am hacking the cmake build to get each component built separately, and am trying some other fixup (see #556). I get the libblas.so and libblas64.so libraries built fine, I get the header dirs configured … but the installed cblas.h and lapacke.h are identical for the 32 and 64 bit indexing versions. This is at odds with openblas: There, I got a crucial difference that I do not see for the netlib builds:

diff -ruN /data/pkg/include/openblas/openblas_config.h /data/pkg/include/openblas64/openblas_config.h
--- /data/pkg/include/openblas/openblas_config.h	2021-06-03 19:03:53.000000000 +0200
+++ /data/pkg/include/openblas64/openblas_config.h	2021-06-03 19:13:36.000000000 +0200
@@ -44,6 +44,7 @@
 #define OPENBLAS_DLOCAL_BUFFER_SIZE 32768
 #define OPENBLAS_CLOCAL_BUFFER_SIZE 16384
 #define OPENBLAS_ZLOCAL_BUFFER_SIZE 12288
+#define OPENBLAS_USE64BITINT 
 #define OPENBLAS_GEMM_MULTITHREAD_THRESHOLD 4
 #define OPENBLAS_VERSION " OpenBLAS 0.3.15 "
 /*This is only for "make install" target.*/

For the reference libraries, all headers from the 32 and 64 bit index builds are identical and apparently users are expected to put
-DWeirdNEC in their flags (might have been funny 30 years ago) for cblas.h and -DLAPACK_ILP64 -DHAVE_LAPACK_CONFIG_H. Since people use the optimized BLAS libraries in production, the de facto standard is not to expose that to the users. These feed back on the reference, IMHO, and the headers installed from an ILP64 build should not require funky flags to avoid crashing your app when linking to the 64 bit lib.

Do we agree that it is the right solution to modify the headers at build time to define the correct integers?

Btw, the cblas config files that are installed also miss any reference to the necessary defs, so are broken for the 64 bit index builds, as it seems. But actually, I ponder not installing these at all. They are redundant with the .pc files and make it possibly harder to convince cmake-using dependent packages to accept a packager choice via BLAS_LIBS etal.

[drhpc]

PS: With Intel MKL, there is a central switch -DMKL_ILP64 to be set. I imagine setting up trivial
include/intel-mkl64/cblas.h with

#ifndef MKL_ILP64
#define MKL_ILP64
#endif
#include <mkl_cblas.h>

to fit the general scheme. I could also put the define into BLAS_INCLUDES, same for the weird netlib defines. What is better? Do we want to do it like Intel or like OpenBLAS?

[weslleyspereira]

Do we agree that it is the right solution to modify the headers at build time to define the correct integers?

Yes. I agree with that, and prefer the solution that does not replicate the entire header. I think it is cleaner.

Btw, the cblas config files that are installed also miss any reference to the necessary defs, so are broken for the 64 bit index builds, as it seems.

Right. I just installed the 64-bits libraries (BUILD_INDEX64=ON) and couldn't see anything telling me to use WeirdNEC, LAPACK_ILP64 or HAVE_LAPACK_CONFIG_H. Thanks for noticing that!