Arch Linux User Repository

Prebuilt binaries of this package can be found in arch4edu.

I made an updated python-numpy-mkl and python-scipy-mkl that uses intel-oneapi-mkl.

Be sure to install the updated oneapi packages from https://github.com/huyz-git/intel-oneapi-aur since the current intel-oneapi-mkl cannot be installed with packages from AUR.

I have triggered this package to become orphaned, so a new volunteer can adopt it and update it.

Well, I know this package is out-of-date.

This package need icc to build. But all oneapi packages are missing in archlinux. So I can't build it right now.

Luckily, there is an official repository containing all oneapi packages on ubuntu. And we can directly convert the *.deb packages to archlinux packages.

However, I don't have that much of free time right now. If anyone can help, please start migrating the ubuntu packages to AUR.

@RunAndTumble Wait. I mis-understood your previous comment.

Do you mean why I'm using gcc8's headers to build this package?

Maybe they are needed for the old versions. And I haven't tested the current version with the latest headers.

I'll test it and do something about the headers ASAP.

I get the following error when trying to use this package:

/usr/lib/python3.8/site-packages/scipy/integrate/vode.cpython-38-x86_64-linux-gnu.so: invalid ELF header

And tons of errors when trying to run the tests with

python
import scipy
scipy.test()

Am I doing something wrong? Compilation seems to work. I also have installed (and used without any issues) the package python-numpy-mkl.

@petronny Could you please elaborate a bit? I didn't compare the tests for gcc8 and gcc9 but I don't think there should be a difference.

@RunAndTumble I just want to make sure it's fully optimized by mkl.

Why do you specify the version of gcc? It works nicely with the system one.

Prebuilt binaries of this package can be found in arch4edu.

@alleut Could you confirm you have a working icc?

It doesn't build now:

Traceback (most recent call last): File "setup.py", line 505, in <module> setup_package() File "setup.py", line 501, in setup_package setup(metadata) File "/usr/lib/python3.7/site-packages/numpy/distutils/core.py", line 171, in setup return old_setup(new_attr) File "/usr/lib/python3.7/site-packages/setuptools/init.py", line 145, in setup return distutils.core.setup(**attrs) File "/usr/lib/python3.7/distutils/core.py", line 148, in setup dist.run_commands() File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands self.run_command(cmd) File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/usr/lib/python3.7/site-packages/numpy/distutils/command/build_clib.py", line 75, in run self.run_command('build_src') File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/usr/lib/python3.7/site-packages/numpy/distutils/command/build_src.py", line 148, in run self.build_sources() File "/usr/lib/python3.7/site-packages/numpy/distutils/command/build_src.py", line 165, in build_sources self.build_extension_sources(ext) File "/usr/lib/python3.7/site-packages/numpy/distutils/command/build_src.py", line 322, in build_extension_sources sources = self.generate_sources(sources, ext) File "/usr/lib/python3.7/site-packages/numpy/distutils/command/build_src.py", line 375, in generate_sources source = func(extension, build_dir) File "scipy/_lib/setup.py", line 32, in get_messagestream_config if config_cmd.check_func('open_memstream', decl=True, call=True): File "/usr/lib/python3.7/site-packages/numpy/distutils/command/config.py", line 321, in check_func self._check_compiler() File "/usr/lib/python3.7/site-packages/numpy/distutils/command/config.py", line 84, in _check_compiler self.fcompiler.customize(self.distribution) File "/usr/lib/python3.7/site-packages/numpy/distutils/fcompiler/init.py", line 511, in customize get_flags('opt', oflags) File "/usr/lib/python3.7/site-packages/numpy/distutils/fcompiler/init.py", line 502, in get_flags flags.extend(getattr(self.flag_vars, tag)) File "/usr/lib/python3.7/site-packages/numpy/distutils/fcompiler/environment.py", line 40, in getattr return self._get_var(name, conf_desc) File "/usr/lib/python3.7/site-packages/numpy/distutils/fcompiler/environment.py", line 56, in _get_var var = self._hook_handler(name, hook) File "/usr/lib/python3.7/site-packages/numpy/distutils/fcompiler/init.py", line 709, in _environment_hook return hook() File "/usr/lib/python3.7/site-packages/numpy/distutils/fcompiler/intel.py", line 128, in get_flags_opt v = self.get_version() File "/usr/lib/python3.7/site-packages/numpy/distutils/fcompiler/init.py", line 432, in get_version raise CompilerNotFound() numpy.distutils.fcompiler.CompilerNotFound

local/intel-fortran-compiler 2019.3.199-3 Intel Fortran compiler 19.0.3

@RunAndTumble I'm waiting for the update of scipy 1.3 in official repository. I'll update this ASAP since scipy 1.3 is in official repository now.

Hey, if you do not have time to support it properly, you can add me as a co-maintainer.

Do not forget that scipy 1.3 doesn't support python 2.7.

@petronny: Yesterday I upgraded to 1.2.0, which worked after manually patching my numpy/distutils/ccompiler.py with the workaround you provided in your pull request to numpy. Today, with 1.2.1, I'm getting the following error:

==> Starting prepare()...

patching file scipy/stats/_continuous_distns.py

Hunk #1 FAILED at 1.

1 out of 1 hunk FAILED -- saving rejects to file scipy/stats/_continuous_distns.py.rej

patching file scipy/stats/_stats_mstats_common.py

==> ERROR: A failure occurred in prepare().

Aborting...

@petronny I've added your patch to numpy-mkl, works like a charm with scipy 1.2.1

Good news, they're finally going to drop python2 support. See https://github.com/scipy/scipy/pull/9582

@RunAndTumble I'm getting that CompilerNotFound error too.
I create a workaround to fix it.
It's not a good solution but I think it's giving enough clues to the developers to fix it.

@petronny thanks for the SHA fix. However, I still have that issue with the compiler. I tried to build it locally & install directly but no result so far. Any ideas?

@petronny SHA sums check fails

P.S. since we do not use gcc, it's probably good to remove it from the dependency list. P.P.S. "CompilerNotFound()" again

Well, this is awkward: I started getting "CompilerNotFound()" as well. I'm pretty sure that I didn't change anything and I had it installed (had to remove to reinstall numpy)

UPD. I feel that this is related to icc somehow, since it's been updated recently. However, I do not see how this might change the outcome.

@Maverick You need to install the license for intel's compilers.

Still getting the "CompilerNotFound()" message despite installed mkl, as alleut has described. What should I do?

Long time passed, and I still don't find a way to fix the python2 version. Can we deprecate it?

Since setuptools changed a lot, please update the gcc part:

line 63, python3 setup.py build --fcompiler=gnu95 --compiler=unix

line 80, python2 setup.py build --fcompiler=gnu95 --compiler=unix

line 122, python3 setup.py config --compiler=unix config_fc --fcompiler=gnu95 install --prefix=/usr --root="${pkgdir}/" --optimize=1

line 145, python2 setup.py config --compiler=unix config_fc --fcompiler=gnu95 install --prefix=/usr --root="${pkgdir}/" --optimize=1

AFAIK, Python 3.7 works fine. The problem is the new version of numpy is not compatible with scipy when using python 2.7. I'm trying to solve this.

Somebody has figure out how to compile scipy-mkl with python 3.7 ????

python-scipy-mkl has a literal $pkgver in its provides. This happens because you have enclosed these variables in strong quotes.

Forwarded from https://github.com/arch4edu/arch4edu/issues/17

Hi, the build_python.sh fails the sha256sums check.

Please update the checksums.

@petronny TNX !!! :)

Hi, after intel-parallel-studio-xe has been updated to 2018.3.222, I finally figured out how to build this package and update it to 1.1.0.

Here is my PKGBUILD and please update it.

PS. I think we should use a patch rather than the sed command in this case. Please update my PKGBUILD.

How to tell builder what compiler to use?

Found executable /opt/intel/compilers_and_libraries_2018.1.163/linux/bin/intel64/icc customize IntelEM64TFCompiler
Found executable /opt/intel/compilers_and_libraries_2018.1.163/linux/bin/intel64/ifort
Traceback (most recent call last):
File "setup.py", line 418, in <module>setup_package()
File "setup.py", line 414, in setup_packagesetup(**metadata)
File "/usr/lib/python3.6/site-packages/numpy/distutils/core.py", line 169, in setupreturn old_setup(**new_attr)
File "/usr/lib/python3.6/site-packages/setuptools/__init__.py", line 129, in setupreturn distutils.core.setup(**attrs)
File "/usr/lib/python3.6/distutils/core.py", line 148, in setupdist.run_commands()
File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commandsself.run_command(cmd)
File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_commandcmd_obj.run()
File "/usr/lib/python3.6/site-packages/numpy/distutils/command/build_clib.py", line 75, in runself.run_command('build_src')
File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_commandself.distribution.run_command(command)
File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_commandcmd_obj.run()
File "/usr/lib/python3.6/site-packages/numpy/distutils/command/build_src.py", line 148, in runself.build_sources()
File "/usr/lib/python3.6/site-packages/numpy/distutils/command/build_src.py", line 165, in build_sourcesself.build_extension_sources(ext)
File "/usr/lib/python3.6/site-packages/numpy/distutils/command/build_src.py", line 324, in build_extension_sourcessources = self.generate_sources(sources, ext)
File "/usr/lib/python3.6/site-packages/numpy/distutils/command/build_src.py", line 377, in generate_sourcessource = func(extension, build_dir)
File "scipy/_lib/setup.py", line 32, in get_messagestream_configif config_cmd.check_func('open_memstream', decl=True, call=True):
File "/usr/lib/python3.6/site-packages/numpy/distutils/command/config.py", line 312, in check_funcself._check_compiler()
File "/usr/lib/python3.6/site-packages/numpy/distutils/command/config.py", line 83, in _check_compilerself.fcompiler.customize(self.distribution)
File "/usr/lib/python3.6/site-packages/numpy/distutils/fcompiler/__init__.py", line 499, in customizeget_flags('opt', oflags)
File "/usr/lib/python3.6/site-packages/numpy/distutils/fcompiler/__init__.py", line 490, in get_flagsflags.extend(getattr(self.flag_vars, tag))
File "/usr/lib/python3.6/site-packages/numpy/distutils/environment.py", line 39, in __getattr__return self._get_var(name, conf_desc)
File "/usr/lib/python3.6/site-packages/numpy/distutils/environment.py", line 53, in _get_varvar = self._hook_handler(name, hook)
File "/usr/lib/python3.6/site-packages/numpy/distutils/fcompiler/__init__.py", line 697, in _environment_hookreturn hook()
File "/usr/lib/python3.6/site-packages/numpy/distutils/fcompiler/intel.py", line 125, in get_flags_optv = self.get_version()
File "/usr/lib/python3.6/site-packages/numpy/distutils/fcompiler/__init__.py", line 430, in get_versionraise CompilerNotFound()
numpy.distutils.fcompiler.CompilerNotFound

The bug in icc 18.0.0 has been fixed in the icc 18.0.1.

If you are still using the icc 18.0.0 uncomment the lines 62 and 79. and comment out the lines 61 and 78.

Replacing .. with ${srcdir} worked for me:

# copy python3 build files
cp ${srcdir}/build_python3.sh scipy-${pkgver}

# copy python2 build files
cp -r scipy-${pkgver} scipy-${pkgver}-py2
cp ${srcdir}/build_python2.sh scipy-${pkgver}-py2

Yes, it probably is the BUILDDIR variable that changes the behaviour. According to the documentation a proper solution would be to put the copy processes into a prepare() section and use $srcdir as target folder: prepare() { # copy python3 build files cp build_python3.sh $srcdir/scipy-${pkgver} # copy python2 build files cp -r $srcdir/scipy-${pkgver} srcdir/scipy-${pkgver}-py2 cp build_python2.sh $srcdir/scipy-${pkgver}-py2 } An alternative would be to use the $startdir variable instead of '../'

@gwatterst well that did not work for me, could it be because I have set "BUILDDIR=/tmp/makepkg" ? this is the first pkgbuild that has failed for me with that setting, removing "../" worked for me.

@arvidsaur: No, that should be correct, since "makepkg will change the current directory to $srcdir before executing the build()" (from the wiki page). So, during build() we already are in src/ and the 2 shell scripts are still in ../ However, one might consider copying all the files during prepare()

I think there is a minor error in the PKGBUILD it should be "cp build_python3.sh scipy-${pkgver}" instead of "cp ../build_python3.sh scipy-${pkgver}" same for the python2 file.

Now it works also with icc. Thanks to all for the suggestions and the PKGBUILD !!!

I can confirm the hack of invik is working. The problem appears to be that the C sources need the set environment variable while compiling the C++ sources with the set environment variables returns an error. I have outsourced the ever-repeating builds into a shell script that make it possible to build the package with a single makepkg command. I'll send this version the the package maintainer.

There is a very dumb hack to get the package to compile with the 2018 Intel compiler (the one with the _Float128 error): loop-compile 8 or 10 times (sorry, lost count), one prepending the '__INTEL_PRE_CFLAGS="$__INTEL_PRE_CFLAGS -D_Float128=__float128"' variable, one without. I.e.: __INTEL_PRE_CFLAGS=" -D_Float128=__float128" python3 setup.py config --compiler=intelem --fcompiler=intelem build_clib --compiler=intelem --fcompiler=intelem build_ext --compiler=intelem --fcompiler=intelem python3 setup.py config --compiler=intelem --fcompiler=intelem build_clib --compiler=intelem --fcompiler=intelem build_ext --compiler=intelem --fcompiler=intelem [repeat these two another 4 times, to be sure, and it should be ready for packaging] In other words: try compiling setting the __INTEL_PRE_CFLAGS, and as soon as it fails, launch compilation WITHOUT setting the environment variable. It will continue from where the first try failed, and so on, until it gets it done. By the way: there is also a bug in the "package_python*-scipy-mkl()" functions in the PKGBUILD. Where it says: python* setup.py config_fc --fcompiler=intel install --prefix=/usr --root=${pkgdir} --optimize=1 It should be "--fcompiler=intelem". It needs to be corrected, or else the packaging will fail. You can find a fully working PKGBUILD here: https://pastebin.com/ncpPC1xY

Added 1.0 Unfortunately the fix for the _Float128 error does not works with scipy. So at the moment the solutions seems to compile with gcc (edit PKGBUILD) or downgrade the glibc.

For numpy adding export __INTEL_PRE_CFLAGS="$__INTEL_PRE_CFLAGS -D_Float128=__float128" to the PKGBUILD solved the _Float128 error. (compare https://aur.archlinux.org/packages/python-numpy-mkl/) Here the build process quits at a later stage with the same error, so I wonder if the FLAG is overridden somewhere?

Hello, I get the following error when trying to install this pkg: /usr/include/bits/floatn.h(73): error: invalid combination of type specifiers typedef __float128 _Float128; I am guessing this has something to do with this: https://software.intel.com/en-us/forums/intel-c-compiler/topic/742701 The full error: https://pastebin.com/L1GEKwWq Anyone got any ideas what I can do to solve this?

Install recently and got ifort warnings. My icc and ifort also use a AUR make version. ifort: command line remark #10148: option '-w90' not supported ifort: command line remark #10148: option '-w95' not supported ifort: command line remark #10411: option '-openmp' is deprecated and will be removed in a future release. Please use the replacement option '-qopenmp' I know this warnings are not problems and should be fixed easily :)

No site.cfg file included in this package: I had to copy over the site.cfg from python-numpy-mkl to be able to compile with mkl support.

@white1033 I had the same problem, edit PKGBUILD with --fcompiler=intelem if you use 64bit and --fcompiler=intel if you use 32bit. @trovao thanks for updating the PKGBUILD :)

@bred scipy 0.17 has been released. Would you be so kind and and update the PKGBUILD? Thanks. I've built mine by changing the following variables: pkgver=0.17.0 source=("https://github.com/scipy/scipy/releases/download/v${pkgver}/scipy-${pkgver}.tar.xz" ) md5sums=('ef0949640ee73f845dde6dd7f84d7691')

Getting errors while trying to build the thing here. Any ideas? Found executable /opt/intel/compilers_and_libraries_2016.1.150/linux/bin/intel64/ifort Traceback (most recent call last): File "setup.py", line 253, in <module> setup_package() File "setup.py", line 250, in setup_package setup(**metadata) File "/usr/lib/python3.5/site-packages/numpy/distutils/core.py", line 169, in setup return old_setup(**new_attr) File "/usr/lib/python3.5/distutils/core.py", line 148, in setup dist.run_commands() File "/usr/lib/python3.5/distutils/dist.py", line 955, in run_commands self.run_command(cmd) File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command cmd_obj.run() File "/usr/lib/python3.5/site-packages/numpy/distutils/command/install.py", line 60, in run r = old_install.run(self) File "/usr/lib/python3.5/distutils/command/install.py", line 539, in run self.run_command('build') File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command cmd_obj.run() File "/usr/lib/python3.5/site-packages/numpy/distutils/command/build.py", line 47, in run old_build.run(self) File "/usr/lib/python3.5/distutils/command/build.py", line 135, in run self.run_command(cmd_name) File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command cmd_obj.run() File "/usr/lib/python3.5/site-packages/numpy/distutils/command/build_clib.py", line 103, in run self._f_compiler.customize(self.distribution) File "/usr/lib/python3.5/site-packages/numpy/distutils/fcompiler/__init__.py", line 521, in customize linker_so_flags = self.flag_vars.linker_so File "/usr/lib/python3.5/site-packages/numpy/distutils/environment.py", line 39, in __getattr__ return self._get_var(name, conf_desc) File "/usr/lib/python3.5/site-packages/numpy/distutils/environment.py", line 53, in _get_var var = self._hook_handler(name, hook) File "/usr/lib/python3.5/site-packages/numpy/distutils/fcompiler/__init__.py", line 700, in _environment_hook return hook() File "/usr/lib/python3.5/site-packages/numpy/distutils/fcompiler/intel.py", line 66, in get_flags_linker_so v = self.get_version() File "/usr/lib/python3.5/site-packages/numpy/distutils/fcompiler/__init__.py", line 434, in get_version raise CompilerNotFound() numpy.distutils.fcompiler.CompilerNotFound ==> ERROR: A failure occurred in package_python-scipy-mkl(). Aborting... :: Installing python-scipy-mkl package(s)... :: failed to build python-scipy-mkl package(s)

scipy-0.16.0 is out. Probably it would be advisable update the pkgbuild to select compiler intelem or intel based on the arch as is done in python-numpy-mkl. Currently it's a mix.

scipy-0.13.2 is out

This replaced python*-numpy-mkl with python*-numpy since 1.7.1 is out. This then broke the build. Perhaps it would be better to make-depend this on *-numpy-mkl in the first place?

@ mSSM Sorry, but the python2 package is builded interdependently from the python3 package. If we use the python2 builder it uses the py2 interpreter and the py2 libs. Can you send me the full error log?

Ok, some guys on #Python suggested that that comes from the fact that the Python2 version of the package is build with Python3 sources. I have removed the Python2 arguments from the PKGBUILD and the package seems to build fine for Python3. :)

Getting errors when trying to build the thing here. Any ideas? compile options: '-I/usr/lib/python3.3/site-packages/numpy/core/include -I/usr/include/python3.3m -c' icc: scipy/stats/_rank.c icc -O3 -fPIC -fp-model strict -fomit-frame-pointer -openmp -xHost -w -shared build/temp.linux-x86_64-3.3/scipy/stats/_rank.o -L/usr/lib -Lbuild/temp.linux-x86_64-3.3 -lpython3.3m -o build/lib.linux-x86_64-3.3/scipy/stats/_rank.cpython-33m.so running scons Traceback (most recent call last): File "setup.py", line 208, in <module> setup_package() File "setup.py", line 145, in setup_package from numpy.distutils.core import setup File "/usr/lib/python3.3/site-packages/numpy/__init__.py", line 131, in <module> from ._import_tools import PackageLoader File "/usr/lib/python3.3/site-packages/numpy/_import_tools.py", line 263 print(str(mess), file=sys.stderr) ^ SyntaxError: invalid syntax

For this PKGBUILD you must use the imtel-mkl and intel-compilers provided in the Intel parallel studio XE: See: https://aur.archlinux.org/packages.php?ID=56337

Thanks for this and the numpy pkgbuild. May I suggest setting a local variable for these, so that for x86_64 builds: --compiler == --fcompiler = 'intelem' and otherwise --compiler == --fcompiler == 'intel'