Package Details: python-numpy-openblas 1.22.3-1

Git Clone URL: https://aur.archlinux.org/python-numpy-openblas.git (read-only, click to copy)
Package Base: python-numpy-openblas
Description: Scientific tools for Python - built with openblas
Upstream URL: http://numpy.org/
Licenses: custom
Conflicts: python-numpy, python3-numpy
Provides: python-numpy, python3-numpy
Submitter: xia0er
Maintainer: xia0er
Last Packager: xia0er
Votes: 25
Popularity: 0.52
First Submitted: 2013-11-21 20:38 (UTC)
Last Updated: 2022-04-10 20:25 (UTC)

Required by (1481)

Sources (1)

Latest Comments

bnavigator commented on 2022-01-06 20:35 (UTC)

You should determine the sitelib path dynamically:

https://wiki.archlinux.org/title/Python_package_guidelines

xia0er commented on 2022-01-06 20:24 (UTC)

@gnaggnoyil, fixed. Thanks for pointing this out.

gnaggnoyil commented on 2022-01-06 15:36 (UTC)

Current version of core/python is 3.10 while PYTHONPATH is set to /usr/lib/python3.9 in check of PKGBUILD. I think either PYTHONPATH needs to be updated to match current core/python version or a python=3.9 dependency needs to be explicitly specified in the dependencies section.

Xyne commented on 2020-12-05 18:10 (UTC)

Can you please add python-hypothesis to the makedepends array. It's required for the check function. Thanks!

mdeff commented on 2020-10-08 11:12 (UTC)

Yes a distribution-wide switch would be ideal (Arch does it for Java, and others like Debian have a generic method that they use for BLAS too).

bnavigator commented on 2020-10-05 11:49 (UTC)

Hi,

the mentioned ArchWiki page and an Archlinux wide switching approach would be greatly appreciated.

See also https://github.com/python-control/Slycot/issues/138#issuecomment-703573577

hairyass commented on 2020-05-29 02:46 (UTC)

I like this package because it just works. It removes all the conflicting packages, so please keep this package

E3LDDfrK commented on 2020-04-07 17:54 (UTC)

I was thinking about creating an ArchWiki page that explains what is BLAS & LAPACK, which packages use them, what are the available providers, and how to switch. This could be linked from the user package wikis (numpy, R, julia, etc.). Would that solve the issue in your opinion?

@mdeff That sounds great.

mdeff commented on 2020-04-07 11:35 (UTC)

@xia0er, thanks for your consideration. Isn't it overkill to have to rebuild the package for an additional dependency? Also, such a policy quickly explodes, as you'd need every <package>-<blas> combinations with {numpy, scipy, julia, r, pytorch, tensorflow, opencv, arpack, etc.} as packages and {netlib, openblas, mkl, blis, atlas} as blas. I think it's cleaner for user packages to depend on {blas, cblas, lapack, lapacke} and for provider packages to provide those. Users then choose which provider to install.

I was thinking about creating an ArchWiki page that explains what is BLAS & LAPACK, which packages use them, what are the available providers, and how to switch. This could be linked from the user package wikis (numpy, R, julia, etc.). Would that solve the issue in your opinion?

In the longer term, Arch should ideally allow the installation of multiple providers and have a mechanism to easily switch the default, like Debian, Gentoo, conda-forge (Arch does it for java).

Do you remember where the discussion about not including LAPACK in community/openblas happened? I searched as much as I could and didn't find any. In my opinion it's a serious issue (most serious being the omission of the C bindings), and the maintainer doesn't seem responsive. :/

xia0er commented on 2020-04-06 18:09 (UTC)

@mdeff Just checked out the FS#s you linked to. If the community/openblas eventually includes lapack, I agree this package can be sunset/removed, but I'd like to keep it active before that.

xia0er commented on 2020-04-06 18:04 (UTC)

@mdeff, the whole purpose of this package is to specify the dependency on aur/openblas-lapack. The community/openblas doesn't provide lapack (and will not, as I recall from an early discussion in the openblas package and why the aur/openblas-lapack package was created). As you pointed out, one can install aur/openblas-lapack and community/python-numpy, and have an optimized bumpy installation, but there is no clear way to specify the dependency than this package.

mdeff commented on 2020-04-06 11:34 (UTC) (edited on 2020-04-06 11:35 (UTC) by mdeff)

You can check what numpy is actually dynamically linked to with ldd /usr/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-x86_64-linux-gnu.so and ldd /usr/lib/python3.8/site-packages/numpy/linalg/_umath_linalg.cpython-38-x86_64-linux-gnu.so. There will be a line about BLAS, like libblas.so.3 => /usr/lib/libblas.so.3 and a line about LAPACK, like liblapack.so.3 => /usr/lib/liblapack.so.3. Numpy links to the system-provided BLAS and LAPACK. We can check where they come from.

1) If you have extra/blas, extra/cblas, and extra/lapack installed (i.e., BLAS and LAPACK from netlib), /usr/lib/libblas.so.3 will be a symlink to /usr/lib/libblas.so.3.9.0 (owned by extra/blas), and /usr/lib/liblapack.so.3 will be a symlink to /usr/lib/liblapack.so.3.9.0 (owned by extra/lapack).

2) If you have community/openblas, extra/cblas, and extra/lapack installed (i.e., BLAS from OpenBLAS and LAPACK from netlib), /usr/lib/libblas.so.3 will be a symlink to /usr/lib/libopenblasp-r0.3.9.so (owned by community/openblas) and /usr/lib/liblapack.so.3 will be a symlink to /usr/lib/liblapack.so.3.9.0 (owned by extra/lapack). (FYI, I don't recommended this configuration. See FS#66092, FS#59046, FS#63054, FS#62558. The community/openblas package should be fixed and could replace aur/openblas-lapack.)

3) If you have aur/openblas-lapack installed (i.e., BLAS and LAPACK from OpenBLAS), /usr/lib/libblas.so.3 and /usr/lib/liblapack.so.3 will be symlinks to /usr/lib/libopenblas.so (owned by aur/openblas-lapack).

4) You can actually symlink any BLAS and LAPACK provider (e.g., MKL, ATLAS, BLIS) to those locations, and numpy and scipy, (but also R, Julia, Octave, etc.) will use them. That's because BLAS and LAPACK are defined as APIs. Those are implemented by multiple providers which can be dynamically loaded at runtime.

E3LDDfrK commented on 2020-04-05 15:36 (UTC)

@mdeff Is there a better way to verify it instead of timing it?

mdeff commented on 2020-04-02 20:48 (UTC)

As @adfjjv pointed out, the stock extra/python-numpy will use OpenBLAS if installed (by community/openblas or aur/openblas-lapack).

You can verify it empirically:

sudo pacman -S blas lapack cblas python-numpy  # BLAS and LAPACK from netlib
time python -c "import numpy as np; x=np.random.random((2000, 2000)); np.dot(x, x.T)"
# about 4 seconds on my system, single thread
sudo pacman -S openblas lapack cblas python-numpy  # BLAS from OpenBLAS, LAPACK from netlib
time python -c "import numpy as np; x=np.random.random((2000, 2000)); np.dot(x, x.T)"
# about 0.3 second on my system, multiple threads
sudo pacman -S openblas-lapack python-numpy  # BLAS and LAPACK from OpenBLAS
time python -c "import numpy as np; x=np.random.random((2000, 2000)); np.dot(x, x.T)"
# about 0.3 second on my system, multiple threads

I suggest to remove this package to avoid further confusion (and aur/python-scipy-openblas).

masca commented on 2019-11-27 17:47 (UTC) (edited on 2019-11-29 13:41 (UTC) by masca)

@xia0er. I get this issue while I try to re-install (doing a clean build).

Edit: Solved after re-updating openblas-lapack!

xia0er commented on 2019-11-27 17:17 (UTC)

@masca, re-installing python-numpy-openblas should resolve your issue.

masca commented on 2019-11-27 14:23 (UTC)

After the update of the openblas-lapack packages, NumPy does not work:

Original error was: libopenblas.so.3: cannot open shared object file: No such file or directory

yodaembedding commented on 2019-10-16 17:45 (UTC)

numpy.test() seems to get stuck at 35%.

xia0er commented on 2019-10-06 21:20 (UTC) (edited on 2019-10-07 05:19 (UTC) by xia0er)

@adfjjv, the purpose of the package is exactly as you described - building and installing python-numpy after openblas-lapack. The stock python-numpy installed with pacman isn't linked to openblas-lapack.

adfjjv commented on 2019-10-05 08:29 (UTC)

What is the purpose of this package? I mean, doesn't installing regular python-numpy together with openblas-lapack work?

xia0er commented on 2019-05-09 02:45 (UTC)

@aviallon, good to know. Thanks for reporting back and hope it'd be helpful to others.

aviallon commented on 2019-05-08 19:45 (UTC)

@xia0er Hello, finally found my problem, after a VERY long time : it was caused by packages enum34 ! I just had to uninstall it and voilà ! See https://stackoverflow.com/a/45716067/6061951 for more info.

aviallon commented on 2019-03-13 16:15 (UTC)

@xiaOer : well, it happens at the same moment, but I did not have anything weird in my PATH.

xia0er commented on 2019-03-05 02:03 (UTC)

aviallon, I cannot replicate your problem. Is your problem related to what is described here: https://github.com/numpy/numpy/issues/6917

aviallon commented on 2019-03-03 00:59 (UTC)

I am getting RuntimeError:("Running cythonyze failed!") now. On two different systems, both with last cython version installed (even tried the git version, with no success).

xia0er commented on 2019-02-06 23:53 (UTC)

@thrasibule we now converted to use split package for both. Also, I follow the python-numpy and rename f2py installed by python2-numpy-openblas. They two can now be installed side-by-side.

thrasibule commented on 2019-02-02 19:49 (UTC)

Same issue with 1.16. python2-numpy-openblas conflicts with python-numpy-openblas as they both install f2py.

thrasibule commented on 2018-10-09 13:55 (UTC)

This package doesn't coinstall with python-numpy-openblas. Also any reason why not make python-numpy-openblas a split package which would install both?

thrasibule commented on 2018-09-27 14:06 (UTC)

python-numpy-openblas and python2-numpy-openblas can't be installed together anymore. /usr/bin/conv-template, /usr/bin/from-template and /usr/bin/f2py exist in both packages. Also any reason why not make it a split package and drop the python2-numpy-openblas package?

xia0er commented on 2018-05-09 20:49 (UTC)

@Pastafarianist From the discussions on the linked sklearn issue page, it doesn't seem to be an issue with python-numpy-openblas, as it also occurs with numpy-mkl and pip installed numpy? Hope you'll be able to diagnose the cause and come up with a fix soon.

domochevski commented on 2018-05-07 13:28 (UTC)

@solnce: You are correct. The mistake was on my part. After recompiling the OpenBLAS library 3 times from source, I realized that I was using the openblas-lapack package provided by AUR on this computer. Recompiling in the same order you did works totally fine. Apologies for any inconveniences my previous message may have caused.

solnce commented on 2018-05-07 12:30 (UTC)

@domochevski: I just recompiled it, works fine for me. The new packages from this morning might require a more thorough recompile. I did it in this order: (1) openblas-lapack, (2) python-numpy-openblas, (3) python-scipy-openblas.

domochevski commented on 2018-05-07 12:24 (UTC)

It seems like the latest update of the gcc-libs package is causing the whole compile process to crash, making it impossible to build and install this package.

Pastafarianist commented on 2018-05-06 11:21 (UTC)

Importing sklearn with python-numpy-openblas (or python-numpy-mkl) installed causes CPU affinity to be reset to {0}. Please see https://github.com/scikit-learn/scikit-learn/issues/11071 for more info and a PoC. The issue does not manifest with python-numpy.

solnce commented on 2018-01-30 10:18 (UTC)

@sakel: You can downgrade to the penultimate version of python-matplotlib, which solved it for me.

I'm contacting xia0er by mail now. If there is no answer by 2018-02-14, I will file an orphan request.

sakel commented on 2018-01-29 16:25 (UTC)

and 1.13.1 causes python-matplotlib to crash :'(

zhou13 commented on 2018-01-23 21:26 (UTC)

1.14 is out :)

anntzer commented on 2016-10-04 01:46 (UTC)

... and 1.11.2 :)

anntzer commented on 2016-07-22 05:17 (UTC)

... and another one to 1.11.1.

anntzer commented on 2016-04-13 16:44 (UTC)

Kindly requesting an update to 1.11.0...

xia0er commented on 2015-11-09 19:51 (UTC)

@ewtoombs Indeed. This seems to be a problem caused by putting depends inside the package() function - probably related to the problem @mnick experienced as well. I now moved it along with other parameters outside the function.

ewtoombs commented on 2015-10-18 21:33 (UTC)

For some reason, makepkg -si doesn't install cython as a dependency. This results in the build failure that you might expect to see. Is this a problem with makepkg or with this PKGBUILD?

xia0er commented on 2015-01-09 22:18 (UTC)

gcc-fortran is specified in the makedepends in package_python-numpy-openblas and package_python2-numpy-openblas function.

mnick commented on 2015-01-09 21:27 (UTC)

The PKGBUILD is missing a build dependency to gcc-fortran. The same holds for the python2 package.

xia0er commented on 2014-12-22 00:24 (UTC)

Just a note, you can check whether numpy built with this package is optimized utilizing openblas by checking whether numpy's lapack_lite library is linked with the openblas library: $ ldd /usr/lib/python2.7/site-packages/numpy/linalg/lapack_lite.so ... libopenblas.so.3 => /usr/lib/libopenblas.so.3 ... Of course running benchmarks would give you some idea as well.

sftrytry commented on 2014-10-06 01:38 (UTC)

It seems 1.9.0 is released.

xia0er commented on 2014-08-11 17:47 (UTC)

numpy now requires openblas to be configured in site.cfg for it to be successfully compiled with openblas.

xia0er commented on 2014-08-11 17:46 (UTC)

numpy now requires openblas to be configured in site.cfg for it to be successfully compiled with openblas.

sftrytry commented on 2014-06-18 04:02 (UTC)

@vidar How can I add cat command in PKGBUILD. I added your script above python2 setup.py stuff but got PKGBUILD syntax error of EOF.

sftrytry commented on 2014-05-17 17:10 (UTC)

2nd vidar. I updated my numpy on May 17. It was not optimized.

vidar commented on 2014-05-15 10:56 (UTC)

Hello. There is a little flaw in the PKGBUILD. numpy is NOT automatically linked against openblas, I added this to the PKGBUILD just above the python2 setup.py lines and it worked: (the standard prefix for my openblas-lapack package was /usr ) cat <<-EOF > site.cfg [openblas] libraries = openblas library_dirs = /usr/lib include_dirs = /usr/include EOF

xia0er commented on 2014-04-10 21:43 (UTC)

@stalafin, linking against openblas is now handled automatically when numpy is being built from source if openblas-lapack is installed (no change to numpy files needed, not even to site.cfg). I'd imagine it should similar for scipy.

Stalafin commented on 2014-02-25 19:51 (UTC)

How did you do to get Numpy to link against OpenBlas? I checked the Numpy sources, and in its site.cfg.example there is an example on how to get it to link against OpenBlas. However, your PKGBUILD appears to use the default site.cfg. I am trying to have Scipy link against OpenBlas as well, but I don't understand how to achieve that. Any hints?

uboot commented on 2014-02-20 12:45 (UTC)

options=("staticlibs") required, packages like scipy won't build otherwise...

uboot commented on 2014-02-20 12:45 (UTC)

options=("staticlibs") required, packages like scipy won't build otherwise...

xia0er commented on 2013-12-09 17:45 (UTC)

As compiling with openblas support is now handled upstream in numpy, this packages is only needed to indicate the dependency of openblas-lapack package (as openblas package doesn't do it) and do a build of numpy after openblas-lapack is installed.

xia0er commented on 2013-11-21 20:49 (UTC)

Thank you, Xyne! I incorporated your changes in 1.7.1-5. Since @sftrytry created an openblas-lapack package, this package can now depend on the new package. I deleted my previous comment. I am a bit ambivalent regarding separating PKGBUILD for python2 and 3. It is a pity that AUR doesn't support split package, but the single PKGBUILD works well on any archlinux box, and even though the python2 and 3 builds do not share any compilation, the commands are identical. So I am slightly leaning the current status - commenting out instead of removing the split package syntax.

xia0er commented on 2013-11-21 20:39 (UTC)

This package is split from python2-numpy-openblas. Since I don't use python3, I haven't tested it. Leave a comment if it doesn't work.

Xyne commented on 2013-11-21 02:03 (UTC)

Hi, I have cleaned up the PKGBUILD (removal of deprecated variables, removal of "replaces" array, inclusion of "prepare" function, etc.). You can find it here: http://xyne.archlinux.ca/tmp/pkgbuilds/ You will need to re-comment the pkgbase and pkgname variable. Let me know if you have any questions about my changes. Given that the split packages do not share any compilation I do not think it makes any sense to keep this as a split package on the AUR. You can always a single local PKGBUILD and change the comments before uploading the source tarball. Regarding the openblas PKGBUILD below, have you contacted the current maintainer of openblas to request default lapack and cblas support? Btw, it is against AUR policy to post PKGBUILDs in comments. Please use something like pastebin.com in the future. Aside from keep the page clean, it provides syntax highlighting, proper indentation and easy download of the original file. Thanks for maintaining this package. Regards, Xyne