Arch Linux User Repository

Anyhow, this is the resulting cado_config.h, looking about right I think

Yes, that matches mine. Interestingly, setting _march=haswell on my sandybridge produces the same cado_config.h (except for the C*FLAGS strings of course), but results in incompatible binaries.

Ok, so... x86-64-v4 is clearly too high a target to run on sandybridge CPUs, but on the other hand when I tried to compile it with the lower x86-64-v2 and x86-64-v3 it just wouldn't build... neither would it if I set the various -msse4.2, -mavx etc. manually with the basic -march=x86-64. From what I read v2 is very similar in terms of capabilities to sandybridge, and v3 is to haswell.

About SIMD instructions, -march=x86-64 does have -msse2 but it seems cado wants the higher stuff. In my trials, it always failed trying to compile intrinsics that relate to the AVX instruction set. Of course as I said, manually enabling -msse4.2 and -mavx proved no solution.

It does compile (and work) with -march=sandybridge on my i7 4790K which should be a haswell, being the next major architecture right after sandybridge it would be strange otherwise... but still... it may be that sandybridge is the baseline architecture for this package, being the lowest "phisical" architecture to have AVX and x86-64-v2 not working for some reason.

Anyhow, this is the resulting cado_config.h, looking about right I think

In the patch for issue #30019, if -march is detected during configuration, HAVE_* feature flags are not set at all

They are set somewhere, though the patch superficially suggests otherwise. The configuration produces a file $srcdir/cado-nfs/build/$USER/cado_config.h. If you inspect that, you will find that the HAVE_* macros are set according to your -march spec. Which is why building with -march=x86-64-v4 can be done on my sandybridge platform, except that the produced binaries cannot be run there. However, CADO-NFS seems to require some form of SIMD datatype, and the makepkg default -march=x86-64 apparently provides none, which causes build attempts to fail.

Out of curiosity, I'd like to know whether anyone of you can build with -march=sandybridge, and how this would change your cado_config.h.

Alright, let's set -march to native for now then, after all, binary portability is not of primary importance here on the AUR and at least it'll work in most cases.

Still, I do not like this very much, there must be something wrong with the build system, and at the moment I don't have the time to check it thoroughly: what's the point of checking for availability of certain instruction sets such as AVX, setting flags in the form of HAVE_AVX, and then trying to use those instruction sets anyways even when compiling for platforms that do not have those instructions? In the patch for issue #30019, if -march is detected during configuration, HAVE_* feature flags are not set at all, which depending on how they are used by the build system may mean they are wrongly assumed false. In any case, I would have expected such flags to trigger some kind of conditional compilation to build a portable version of the software when specific platform features are not available, and if that's not the case at least to stop the build early since it clearly can't be completed. The fact x86-64-v4 compiles but crashes at runtime is quite baffling too.

Anyhow that's the way it is, for now at least.

@ccorn I followed the outline of your patch but implemented it a little differently to account for IFS in the word splitting of C*FLAGS.

I also removed i686 and pentium4 from the arch array, since they clearly cannot be supported for the time being.

If I understand the issue correctly, previous builds were effectively -march=native because the auto-configuration examined the build machine and added SIMD support flags according to its capabilities. The recent patch now prevents this if -march is specified.

Thus, previous builds have been tuned to the build system even if that has not been explicit. So far, I have not seen a bug report about that. Therefore, I conclude that -march=native is the major use case. After all, factorization projects tend to be long-running, and not tuning the builds would result in wasting lots of time, energy, and money.

For now, I found out what I think the baseline architecture to have those instructions is, so that binary packages built with this PKGBUILD can still be somewhat redistributable (although owners of older x86_64 CPUs are screwed for now).

For me, x86-64-v4 does not work: The built programs crash in the polynomial selection stage for a simple 90-digit factorization test. I tried x86-64-v3 which did not work either.

However, -march=native and -march=sandybridge do work for me. My build machine has Xeon E5-2670 which are sandybridge plus AES. It has AVX, but not AVX2.

If sandybridge does not work for you even if x86-64-v4 does, I suppose that the specified -march must be close to -march=native. Which would make -march=native the only sensible choice (and the only one consistent with more arch entries than x86_64), but then the built package will not be usable on older-generation platforms.

There is a related issue and a related patchset.

By the way, I have chosen to also set CXXFLAGS and delete other -march specs:

--- a/PKGBUILD
+++ b/PKGBUILD
@@ -22,6 +22,9 @@ provides=('cado-nfs')
 source=("git+https://gitlab.inria.fr/cado-nfs/${_pkg}.git")
 md5sums=('SKIP')

+# Need -march for SIMD support. Makes the resulting package less portable.
+_march=native
+
 pkgver() {
   cd "$_pkg"
   git log -1 --format="%cd.%h" --date=short | sed 's/-//g'
@@ -29,6 +32,13 @@ pkgver() {

 build() {
   cd "$_pkg"
+  shopt -s extglob      # for !()
+  CFLAGS+=" "
+  CFLAGS="${CFLAGS//-march=!(* *) } -march=$_march"
+  CPPFLAGS+=" "
+  CPPFLAGS="${CPPFLAGS//-march=!(* *) } -march=$_march"
+  CXXFLAGS+=" "
+  CXXFLAGS="${CXXFLAGS//-march=!(* *) } -march=$_march"
   cat <<EOF >local.sh
 PREFIX=/usr
 HWLOC=$PREFIX
@@ -37,7 +47,7 @@ CURL=$PREFIX
 # Remove 32-bit barriers to big factorizations
 FLAGS_SIZE="-DSIZEOF_P_R_VALUES=8 -DSIZEOF_INDEX=8"
 EOF
-  make CFLAGS="$CFLAGS -march=x86-64-v4" CPPFLAGS="$CPPFLAGS -march=x86-64-v4"
+  make
 }

 package() {

@Dylan14 thank you for your feedback, it seems the current version of cado-nfs uses Intel intrinsics that depend on very recent versions of the AVX instruction set on x86_64.

For now, I found out what I think the baseline architecture to have those instructions is, so that binary packages built with this PKGBUILD can still be somewhat redistributable (although owners of older x86_64 CPUs are screwed for now).

When I am a little less busy I am gonna check the build system to see if there's some flag to enable more portable code to support those older architectures.

@Dylan14 could you please compile the current PKGBUILD with LC_ALL=C makepkg -L and send me the resulting log file?

@Aquilalrreale - no changes to my /etc/makepkg.conf.

@Dylan14 are you building the package with the standard /etc/makepkg.conf or have you made changes to it? In a regular Arch's makepkg.conf -march should be set by default to x86_64, which I believe matches -march=auto on your CPU platfom. Can you double-check your config file is not setting it wrong it in the first place? Overriding this specific compiler flag inside the build script is usually not good practice.