AUR (en) - python-pytorch-git

Git Clone URL:	https://aur.archlinux.org/python-pytorch-git.git (read-only, click to copy)
Package Base:	python-pytorch-git
Description:	Tensors and Dynamic neural networks in Python with strong GPU acceleration (with MKL-DNN)
Upstream URL:	https://pytorch.org
Licenses:	BSD
Conflicts:	python-pytorch
Provides:	python-pytorch
Submitter:	zsrkmyn
Maintainer:	None
Last Packager:	hottea
Votes:	2
Popularity:	0.000000
First Submitted:	2017-03-15 20:57 (UTC)
Last Updated:	2019-11-28 13:10 (UTC)

Since a few days the git clone fails:

fatal: bad object refs/remotes/origin/clee2000/ignore-shard-1

trougnouf commented on 2021-12-20 21:38 (UTC) (edited on 2021-12-20 21:42 (UTC) by trougnouf)

I can't install and I don't understand how to resolve the conflict.

[trougnouf@d]: ~>$ sudo pacman -U .cache/pacaur/python-pytorch-git/python-pytorch-cuda-git-1.3.1.r42540.dd02af62835-1-x86_64.pkg.tar.xz
loading packages...
resolving dependencies...
looking for conflicting packages...
:: python-pytorch-cuda-git and python-pytorch-cuda are in conflict (python-pytorch). Remove python-pytorch-cuda? [y/N] y
error: failed to prepare transaction (could not satisfy dependencies)
:: removing python-pytorch-cuda breaks dependency 'python-pytorch-cuda' required by python-pyro-ppl
:: removing python-pytorch-cuda breaks dependency 'python-pytorch-cuda' required by python-torchvision-cuda

edit: removing python-pytorch-cuda first with sudo pacman -Rd --nodeps python-pytorch-cuda did it.

jiangjiechu commented on 2021-07-01 08:14 (UTC)

@iago-lito Actually I didn't build pytorch from this aur but from the vallina source from github. As for the TORCH_CUDA_ARCH_LIST,the cmakefile can automatically detec,so remove this line and let the program decide. There might be another reason for the slow compiling issue. In Aten submodule a CMakeFile creates tons of ".AVX.cpp..AVX.cpp" file recursively(almost 40,000+),which not only slows down the build, but also yeilds multi-declaration error at link time. You have to carefully remove all the duplicates of these files in Aten folder and caffe2 folder.

iago-lito commented on 2021-06-30 09:11 (UTC)

@jiangjiechu Thank you :) It does compile now with CC=gcc-10 CXX=g++-10 yay -S --editmenu python-pytorch-git. Obviously it takes a very long time.. too long yesterday before I actually had to go home. I read the following line from the PKGBUILD:

  # modify to your need, you don't need to compile for all GPU arch
  export TORCH_CUDA_ARCH_LIST="3.2;3.5;3.7;5.0;5.2;5.3;6.0;6.0+PTX;6.1;6.1+PTX;6.2;6.2+PTX;7.0;7.0+PTX;7.2;7.2+PTX;7.5;7.5+PTX"

.. so I guess I could speedup the speedup the process by only keeping the good item in the above list, but how do I know which one to keep?

jiangjiechu commented on 2021-06-29 08:55 (UTC)

@iago-lito, seems that gcc 11.1.0 changes the implementation of _mm_storeu_si()，to change to gcc-10, simply use CC=gcc-10 CXX=g++-10 python setup.py install can build.

iago-lito commented on 2021-06-29 08:22 (UTC)

@jiangjiechu Well, my gcc --version yields gcc (GCC) 11.1.0. But I do have gcc 10 installed and gcc-10 --version yields gcc-10 (Arch Linux 10.2.0-3) 10.2.0. Do you think the package is invoking the wrong compiler? If yes, is that a packaging bug? How can I work around it?

jiangjiechu commented on 2021-06-25 16:20 (UTC)

The problem is incompatibility of compiler gcc with XNNPACK,use gcc-10 can solve

iago-lito commented on 2021-06-17 13:51 (UTC)

Build failed today after yay -S --needed python-pytorch-cuda-git with

[845/5105] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmul-minmax-avx512f-x16.c.o
FAILED: confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmul-minmax-avx512f-x16.c.o 
/usr/bin/cc -DCPUINFO_SUPPORTED_PLATFORM=1 -DFXDIV_USE_INLINE_ASSEMBLY=0 -DXNN_ENABLE_ASSEMBLY=1 -DXNN_ENABLE_MEMOPT=1 -DXNN_ENABLE_SPARSE=1 -DXNN_LOG_LEVEL=0 -I../third_party/XNNPACK/include -I../third_party/XNNPACK/src -I../third_party/cpuinfo/deps/clog/include -I../third_party/cpuinfo/include -I../third_party/pthreadpool/include -I../third_party/FXdiv/include -I../third_party/FP16/include -isystem ../third_party/protobuf/src -isystem ../third_party/gemmlowp -isystem ../third_party/neon2sse -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -O3 -DNDEBUG -fPIC -Wno-psabi -pthread -std=gnu99  -mavx512f  -O2 -MD -MT confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmul-minmax-avx512f-x16.c.o -MF confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmul-minmax-avx512f-x16.c.o.d -o confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmul-minmax-avx512f-x16.c.o -c ../third_party/XNNPACK/src/f32-vbinary/gen/vmul-minmax-avx512f-x16.c
In file included from ../third_party/XNNPACK/src/f32-vbinary/gen/vmul-minmax-avx512f-x16.c:15:
../third_party/XNNPACK/src/xnnpack/intrinsics-polyfill.h:27:6: error: conflicting types for ‘_mm_storeu_si32’; have ‘void(const void *, __m128i)’
   27 | void _mm_storeu_si32(const void* address, __m128i v) {
      |      ^~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/xmmintrin.h:1316,
                 from /usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/immintrin.h:31,
                 from ../third_party/XNNPACK/src/f32-vbinary/gen/vmul-minmax-avx512f-x16.c:12:
/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/emmintrin.h:756:1: note: previous definition of ‘_mm_storeu_si32’ with type ‘void(void *, __m128i)’
  756 | _mm_storeu_si32 (void *__P, __m128i __B)
      | ^~~~~~~~~~~~~~~
[846/5105] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmulc-minmax-avx512f-x32.c.o
FAILED: confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmulc-minmax-avx512f-x32.c.o 
/usr/bin/cc -DCPUINFO_SUPPORTED_PLATFORM=1 -DFXDIV_USE_INLINE_ASSEMBLY=0 -DXNN_ENABLE_ASSEMBLY=1 -DXNN_ENABLE_MEMOPT=1 -DXNN_ENABLE_SPARSE=1 -DXNN_LOG_LEVEL=0 -I../third_party/XNNPACK/include -I../third_party/XNNPACK/src -I../third_party/cpuinfo/deps/clog/include -I../third_party/cpuinfo/include -I../third_party/pthreadpool/include -I../third_party/FXdiv/include -I../third_party/FP16/include -isystem ../third_party/protobuf/src -isystem ../third_party/gemmlowp -isystem ../third_party/neon2sse -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -O3 -DNDEBUG -fPIC -Wno-psabi -pthread -std=gnu99  -mavx512f  -O2 -MD -MT confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmulc-minmax-avx512f-x32.c.o -MF confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmulc-minmax-avx512f-x32.c.o.d -o confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmulc-minmax-avx512f-x32.c.o -c ../third_party/XNNPACK/src/f32-vbinary/gen/vmulc-minmax-avx512f-x32.c
In file included from ../third_party/XNNPACK/src/f32-vbinary/gen/vmulc-minmax-avx512f-x32.c:15:
../third_party/XNNPACK/src/xnnpack/intrinsics-polyfill.h:27:6: error: conflicting types for ‘_mm_storeu_si32’; have ‘void(const void *, __m128i)’
   27 | void _mm_storeu_si32(const void* address, __m128i v) {
      |      ^~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/xmmintrin.h:1316,
                 from /usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/immintrin.h:31,
                 from ../third_party/XNNPACK/src/f32-vbinary/gen/vmulc-minmax-avx512f-x32.c:12:
/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/emmintrin.h:756:1: note: previous definition of ‘_mm_storeu_si32’ with type ‘void(void *, __m128i)’
  756 | _mm_storeu_si32 (void *__P, __m128i __B)
      | ^~~~~~~~~~~~~~~
[847/5105] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vminc-avx512f-x32.c.o
FAILED: confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vminc-avx512f-x32.c.o 
/usr/bin/cc -DCPUINFO_SUPPORTED_PLATFORM=1 -DFXDIV_USE_INLINE_ASSEMBLY=0 -DXNN_ENABLE_ASSEMBLY=1 -DXNN_ENABLE_MEMOPT=1 -DXNN_ENABLE_SPARSE=1 -DXNN_LOG_LEVEL=0 -I../third_party/XNNPACK/include -I../third_party/XNNPACK/src -I../third_party/cpuinfo/deps/clog/include -I../third_party/cpuinfo/include -I../third_party/pthreadpool/include -I../third_party/FXdiv/include -I../third_party/FP16/include -isystem ../third_party/protobuf/src -isystem ../third_party/gemmlowp -isystem ../third_party/neon2sse -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -O3 -DNDEBUG -fPIC -Wno-psabi -pthread -std=gnu99  -mavx512f  -O2 -MD -MT confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vminc-avx512f-x32.c.o -MF confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vminc-avx512f-x32.c.o.d -o confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vminc-avx512f-x32.c.o -c ../third_party/XNNPACK/src/f32-vbinary/gen/vminc-avx512f-x32.c
In file included from ../third_party/XNNPACK/src/f32-vbinary/gen/vminc-avx512f-x32.c:15:
../third_party/XNNPACK/src/xnnpack/intrinsics-polyfill.h:27:6: error: conflicting types for ‘_mm_storeu_si32’; have ‘void(const void *, __m128i)’
   27 | void _mm_storeu_si32(const void* address, __m128i v) {
      |      ^~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/xmmintrin.h:1316,
                 from /usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/immintrin.h:31,
                 from ../third_party/XNNPACK/src/f32-vbinary/gen/vminc-avx512f-x32.c:12:
/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/emmintrin.h:756:1: note: previous definition of ‘_mm_storeu_si32’ with type ‘void(void *, __m128i)’
  756 | _mm_storeu_si32 (void *__P, __m128i __B)
      | ^~~~~~~~~~~~~~~
[848/5105] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmul-minmax-avx512f-x32.c.o
FAILED: confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmul-minmax-avx512f-x32.c.o 
/usr/bin/cc -DCPUINFO_SUPPORTED_PLATFORM=1 -DFXDIV_USE_INLINE_ASSEMBLY=0 -DXNN_ENABLE_ASSEMBLY=1 -DXNN_ENABLE_MEMOPT=1 -DXNN_ENABLE_SPARSE=1 -DXNN_LOG_LEVEL=0 -I../third_party/XNNPACK/include -I../third_party/XNNPACK/src -I../third_party/cpuinfo/deps/clog/include -I../third_party/cpuinfo/include -I../third_party/pthreadpool/include -I../third_party/FXdiv/include -I../third_party/FP16/include -isystem ../third_party/protobuf/src -isystem ../third_party/gemmlowp -isystem ../third_party/neon2sse -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -O3 -DNDEBUG -fPIC -Wno-psabi -pthread -std=gnu99  -mavx512f  -O2 -MD -MT confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmul-minmax-avx512f-x32.c.o -MF confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmul-minmax-avx512f-x32.c.o.d -o confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmul-minmax-avx512f-x32.c.o -c ../third_party/XNNPACK/src/f32-vbinary/gen/vmul-minmax-avx512f-x32.c
In file included from ../third_party/XNNPACK/src/f32-vbinary/gen/vmul-minmax-avx512f-x32.c:15:
../third_party/XNNPACK/src/xnnpack/intrinsics-polyfill.h:27:6: error: conflicting types for ‘_mm_storeu_si32’; have ‘void(const void *, __m128i)’
   27 | void _mm_storeu_si32(const void* address, __m128i v) {
      |      ^~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/xmmintrin.h:1316,
                 from /usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/immintrin.h:31,
                 from ../third_party/XNNPACK/src/f32-vbinary/gen/vmul-minmax-avx512f-x32.c:12:
/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/emmintrin.h:756:1: note: previous definition of ‘_mm_storeu_si32’ with type ‘void(void *, __m128i)’
  756 | _mm_storeu_si32 (void *__P, __m128i __B)
      | ^~~~~~~~~~~~~~~
[850/5105] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmulc-minmax-avx512f-x16.c.o
FAILED: confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmulc-minmax-avx512f-x16.c.o 
/usr/bin/cc -DCPUINFO_SUPPORTED_PLATFORM=1 -DFXDIV_USE_INLINE_ASSEMBLY=0 -DXNN_ENABLE_ASSEMBLY=1 -DXNN_ENABLE_MEMOPT=1 -DXNN_ENABLE_SPARSE=1 -DXNN_LOG_LEVEL=0 -I../third_party/XNNPACK/include -I../third_party/XNNPACK/src -I../third_party/cpuinfo/deps/clog/include -I../third_party/cpuinfo/include -I../third_party/pthreadpool/include -I../third_party/FXdiv/include -I../third_party/FP16/include -isystem ../third_party/protobuf/src -isystem ../third_party/gemmlowp -isystem ../third_party/neon2sse -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -O3 -DNDEBUG -fPIC -Wno-psabi -pthread -std=gnu99  -mavx512f  -O2 -MD -MT confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmulc-minmax-avx512f-x16.c.o -MF confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmulc-minmax-avx512f-x16.c.o.d -o confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmulc-minmax-avx512f-x16.c.o -c ../third_party/XNNPACK/src/f32-vbinary/gen/vmulc-minmax-avx512f-x16.c
In file included from ../third_party/XNNPACK/src/f32-vbinary/gen/vmulc-minmax-avx512f-x16.c:15:
../third_party/XNNPACK/src/xnnpack/intrinsics-polyfill.h:27:6: error: conflicting types for ‘_mm_storeu_si32’; have ‘void(const void *, __m128i)’
   27 | void _mm_storeu_si32(const void* address, __m128i v) {
      |      ^~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/xmmintrin.h:1316,
                 from /usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/immintrin.h:31,
                 from ../third_party/XNNPACK/src/f32-vbinary/gen/vmulc-minmax-avx512f-x16.c:12:
/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/emmintrin.h:756:1: note: previous definition of ‘_mm_storeu_si32’ with type ‘void(void *, __m128i)’
  756 | _mm_storeu_si32 (void *__P, __m128i __B)
      | ^~~~~~~~~~~~~~~
[852/5105] Generating src/x86_64-fma/2d-fourier-16x16.py.o
ninja: build stopped: subcommand failed.

ruestique commented on 2020-07-30 14:38 (UTC)

and after again

Building without cuda and without MKL-DNN
Building wheel torch-1.3.1
-- Building version 1.3.1
Could not find /home/chibo/.cache/pikaur/build/python-pytorch-git/src/pytorch-git/third_party/cpuinfo/CMakeLists.txt
Did you run 'git submodule update --init --recursive'?
==> ERROR: A failure occurred in build().
    Aborting...

ruestique commented on 2020-07-30 14:12 (UTC)

still having multiple sudo issue like this

cp: cannot create regular file 'pytorch-git/pytorch/.git/modules/third-party/cpuinfo/objects/pack/pack-3932b6e9b8a69893eddafdddbe1da7b17b17c4ea.idx': Permission denied

Arch Linux User Repository

Search Criteria

Package Details: python-pytorch-mkl-git 1.3.1.r22820.1350b99de4-1

Package Actions

Dependencies (24)

Required by (155)

Sources (1)

Latest Comments

belgium commented on 2022-05-16 14:20 (UTC)