Package Base Details: python-pytorch-git

Git Clone URL: https://aur.archlinux.org/python-pytorch-git.git (read-only, click to copy)
Submitter: zsrkmyn
Maintainer: None
Last Packager: hottea
Votes: 3
Popularity: 0.000427
First Submitted: 2017-03-15 20:57
Last Updated: 2019-11-28 13:10

Latest Comments

1 2 Next › Last »

jiangjiechu commented on 2021-07-01 08:14

@iago-lito Actually I didn't build pytorch from this aur but from the vallina source from github. As for the TORCH_CUDA_ARCH_LIST,the cmakefile can automatically detec,so remove this line and let the program decide. There might be another reason for the slow compiling issue. In Aten submodule a CMakeFile creates tons of ".AVX.cpp..AVX.cpp" file recursively(almost 40,000+),which not only slows down the build, but also yeilds multi-declaration error at link time. You have to carefully remove all the duplicates of these files in Aten folder and caffe2 folder.

iago-lito commented on 2021-06-30 09:11

@jiangjiechu Thank you :) It does compile now with CC=gcc-10 CXX=g++-10 yay -S --editmenu python-pytorch-git. Obviously it takes a very long time.. too long yesterday before I actually had to go home. I read the following line from the PKGBUILD:

  # modify to your need, you don't need to compile for all GPU arch
  export TORCH_CUDA_ARCH_LIST="3.2;3.5;3.7;5.0;5.2;5.3;6.0;6.0+PTX;6.1;6.1+PTX;6.2;6.2+PTX;7.0;7.0+PTX;7.2;7.2+PTX;7.5;7.5+PTX"

.. so I guess I could speedup the speedup the process by only keeping the good item in the above list, but how do I know which one to keep?

jiangjiechu commented on 2021-06-29 08:55

@iago-lito, seems that gcc 11.1.0 changes the implementation of _mm_storeu_si(),to change to gcc-10, simply use CC=gcc-10 CXX=g++-10 python setup.py install can build.

iago-lito commented on 2021-06-29 08:22

@jiangjiechu Well, my gcc --version yields gcc (GCC) 11.1.0. But I do have gcc 10 installed and gcc-10 --version yields gcc-10 (Arch Linux 10.2.0-3) 10.2.0. Do you think the package is invoking the wrong compiler? If yes, is that a packaging bug? How can I work around it?

jiangjiechu commented on 2021-06-25 16:20

The problem is incompatibility of compiler gcc with XNNPACK,use gcc-10 can solve

iago-lito commented on 2021-06-17 13:51

Build failed today after yay -S --needed python-pytorch-cuda-git with

[845/5105] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmul-minmax-avx512f-x16.c.o
FAILED: confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmul-minmax-avx512f-x16.c.o 
/usr/bin/cc -DCPUINFO_SUPPORTED_PLATFORM=1 -DFXDIV_USE_INLINE_ASSEMBLY=0 -DXNN_ENABLE_ASSEMBLY=1 -DXNN_ENABLE_MEMOPT=1 -DXNN_ENABLE_SPARSE=1 -DXNN_LOG_LEVEL=0 -I../third_party/XNNPACK/include -I../third_party/XNNPACK/src -I../third_party/cpuinfo/deps/clog/include -I../third_party/cpuinfo/include -I../third_party/pthreadpool/include -I../third_party/FXdiv/include -I../third_party/FP16/include -isystem ../third_party/protobuf/src -isystem ../third_party/gemmlowp -isystem ../third_party/neon2sse -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -O3 -DNDEBUG -fPIC -Wno-psabi -pthread -std=gnu99  -mavx512f  -O2 -MD -MT confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmul-minmax-avx512f-x16.c.o -MF confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmul-minmax-avx512f-x16.c.o.d -o confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmul-minmax-avx512f-x16.c.o -c ../third_party/XNNPACK/src/f32-vbinary/gen/vmul-minmax-avx512f-x16.c
In file included from ../third_party/XNNPACK/src/f32-vbinary/gen/vmul-minmax-avx512f-x16.c:15:
../third_party/XNNPACK/src/xnnpack/intrinsics-polyfill.h:27:6: error: conflicting types for ‘_mm_storeu_si32’; have ‘void(const void *, __m128i)’
   27 | void _mm_storeu_si32(const void* address, __m128i v) {
      |      ^~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/xmmintrin.h:1316,
                 from /usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/immintrin.h:31,
                 from ../third_party/XNNPACK/src/f32-vbinary/gen/vmul-minmax-avx512f-x16.c:12:
/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/emmintrin.h:756:1: note: previous definition of ‘_mm_storeu_si32’ with type ‘void(void *, __m128i)’
  756 | _mm_storeu_si32 (void *__P, __m128i __B)
      | ^~~~~~~~~~~~~~~
[846/5105] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmulc-minmax-avx512f-x32.c.o
FAILED: confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmulc-minmax-avx512f-x32.c.o 
/usr/bin/cc -DCPUINFO_SUPPORTED_PLATFORM=1 -DFXDIV_USE_INLINE_ASSEMBLY=0 -DXNN_ENABLE_ASSEMBLY=1 -DXNN_ENABLE_MEMOPT=1 -DXNN_ENABLE_SPARSE=1 -DXNN_LOG_LEVEL=0 -I../third_party/XNNPACK/include -I../third_party/XNNPACK/src -I../third_party/cpuinfo/deps/clog/include -I../third_party/cpuinfo/include -I../third_party/pthreadpool/include -I../third_party/FXdiv/include -I../third_party/FP16/include -isystem ../third_party/protobuf/src -isystem ../third_party/gemmlowp -isystem ../third_party/neon2sse -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -O3 -DNDEBUG -fPIC -Wno-psabi -pthread -std=gnu99  -mavx512f  -O2 -MD -MT confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmulc-minmax-avx512f-x32.c.o -MF confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmulc-minmax-avx512f-x32.c.o.d -o confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmulc-minmax-avx512f-x32.c.o -c ../third_party/XNNPACK/src/f32-vbinary/gen/vmulc-minmax-avx512f-x32.c
In file included from ../third_party/XNNPACK/src/f32-vbinary/gen/vmulc-minmax-avx512f-x32.c:15:
../third_party/XNNPACK/src/xnnpack/intrinsics-polyfill.h:27:6: error: conflicting types for ‘_mm_storeu_si32’; have ‘void(const void *, __m128i)’
   27 | void _mm_storeu_si32(const void* address, __m128i v) {
      |      ^~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/xmmintrin.h:1316,
                 from /usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/immintrin.h:31,
                 from ../third_party/XNNPACK/src/f32-vbinary/gen/vmulc-minmax-avx512f-x32.c:12:
/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/emmintrin.h:756:1: note: previous definition of ‘_mm_storeu_si32’ with type ‘void(void *, __m128i)’
  756 | _mm_storeu_si32 (void *__P, __m128i __B)
      | ^~~~~~~~~~~~~~~
[847/5105] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vminc-avx512f-x32.c.o
FAILED: confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vminc-avx512f-x32.c.o 
/usr/bin/cc -DCPUINFO_SUPPORTED_PLATFORM=1 -DFXDIV_USE_INLINE_ASSEMBLY=0 -DXNN_ENABLE_ASSEMBLY=1 -DXNN_ENABLE_MEMOPT=1 -DXNN_ENABLE_SPARSE=1 -DXNN_LOG_LEVEL=0 -I../third_party/XNNPACK/include -I../third_party/XNNPACK/src -I../third_party/cpuinfo/deps/clog/include -I../third_party/cpuinfo/include -I../third_party/pthreadpool/include -I../third_party/FXdiv/include -I../third_party/FP16/include -isystem ../third_party/protobuf/src -isystem ../third_party/gemmlowp -isystem ../third_party/neon2sse -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -O3 -DNDEBUG -fPIC -Wno-psabi -pthread -std=gnu99  -mavx512f  -O2 -MD -MT confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vminc-avx512f-x32.c.o -MF confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vminc-avx512f-x32.c.o.d -o confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vminc-avx512f-x32.c.o -c ../third_party/XNNPACK/src/f32-vbinary/gen/vminc-avx512f-x32.c
In file included from ../third_party/XNNPACK/src/f32-vbinary/gen/vminc-avx512f-x32.c:15:
../third_party/XNNPACK/src/xnnpack/intrinsics-polyfill.h:27:6: error: conflicting types for ‘_mm_storeu_si32’; have ‘void(const void *, __m128i)’
   27 | void _mm_storeu_si32(const void* address, __m128i v) {
      |      ^~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/xmmintrin.h:1316,
                 from /usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/immintrin.h:31,
                 from ../third_party/XNNPACK/src/f32-vbinary/gen/vminc-avx512f-x32.c:12:
/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/emmintrin.h:756:1: note: previous definition of ‘_mm_storeu_si32’ with type ‘void(void *, __m128i)’
  756 | _mm_storeu_si32 (void *__P, __m128i __B)
      | ^~~~~~~~~~~~~~~
[848/5105] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmul-minmax-avx512f-x32.c.o
FAILED: confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmul-minmax-avx512f-x32.c.o 
/usr/bin/cc -DCPUINFO_SUPPORTED_PLATFORM=1 -DFXDIV_USE_INLINE_ASSEMBLY=0 -DXNN_ENABLE_ASSEMBLY=1 -DXNN_ENABLE_MEMOPT=1 -DXNN_ENABLE_SPARSE=1 -DXNN_LOG_LEVEL=0 -I../third_party/XNNPACK/include -I../third_party/XNNPACK/src -I../third_party/cpuinfo/deps/clog/include -I../third_party/cpuinfo/include -I../third_party/pthreadpool/include -I../third_party/FXdiv/include -I../third_party/FP16/include -isystem ../third_party/protobuf/src -isystem ../third_party/gemmlowp -isystem ../third_party/neon2sse -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -O3 -DNDEBUG -fPIC -Wno-psabi -pthread -std=gnu99  -mavx512f  -O2 -MD -MT confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmul-minmax-avx512f-x32.c.o -MF confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmul-minmax-avx512f-x32.c.o.d -o confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmul-minmax-avx512f-x32.c.o -c ../third_party/XNNPACK/src/f32-vbinary/gen/vmul-minmax-avx512f-x32.c
In file included from ../third_party/XNNPACK/src/f32-vbinary/gen/vmul-minmax-avx512f-x32.c:15:
../third_party/XNNPACK/src/xnnpack/intrinsics-polyfill.h:27:6: error: conflicting types for ‘_mm_storeu_si32’; have ‘void(const void *, __m128i)’
   27 | void _mm_storeu_si32(const void* address, __m128i v) {
      |      ^~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/xmmintrin.h:1316,
                 from /usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/immintrin.h:31,
                 from ../third_party/XNNPACK/src/f32-vbinary/gen/vmul-minmax-avx512f-x32.c:12:
/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/emmintrin.h:756:1: note: previous definition of ‘_mm_storeu_si32’ with type ‘void(void *, __m128i)’
  756 | _mm_storeu_si32 (void *__P, __m128i __B)
      | ^~~~~~~~~~~~~~~
[850/5105] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmulc-minmax-avx512f-x16.c.o
FAILED: confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmulc-minmax-avx512f-x16.c.o 
/usr/bin/cc -DCPUINFO_SUPPORTED_PLATFORM=1 -DFXDIV_USE_INLINE_ASSEMBLY=0 -DXNN_ENABLE_ASSEMBLY=1 -DXNN_ENABLE_MEMOPT=1 -DXNN_ENABLE_SPARSE=1 -DXNN_LOG_LEVEL=0 -I../third_party/XNNPACK/include -I../third_party/XNNPACK/src -I../third_party/cpuinfo/deps/clog/include -I../third_party/cpuinfo/include -I../third_party/pthreadpool/include -I../third_party/FXdiv/include -I../third_party/FP16/include -isystem ../third_party/protobuf/src -isystem ../third_party/gemmlowp -isystem ../third_party/neon2sse -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -O3 -DNDEBUG -fPIC -Wno-psabi -pthread -std=gnu99  -mavx512f  -O2 -MD -MT confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmulc-minmax-avx512f-x16.c.o -MF confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmulc-minmax-avx512f-x16.c.o.d -o confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vbinary/gen/vmulc-minmax-avx512f-x16.c.o -c ../third_party/XNNPACK/src/f32-vbinary/gen/vmulc-minmax-avx512f-x16.c
In file included from ../third_party/XNNPACK/src/f32-vbinary/gen/vmulc-minmax-avx512f-x16.c:15:
../third_party/XNNPACK/src/xnnpack/intrinsics-polyfill.h:27:6: error: conflicting types for ‘_mm_storeu_si32’; have ‘void(const void *, __m128i)’
   27 | void _mm_storeu_si32(const void* address, __m128i v) {
      |      ^~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/xmmintrin.h:1316,
                 from /usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/immintrin.h:31,
                 from ../third_party/XNNPACK/src/f32-vbinary/gen/vmulc-minmax-avx512f-x16.c:12:
/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/include/emmintrin.h:756:1: note: previous definition of ‘_mm_storeu_si32’ with type ‘void(void *, __m128i)’
  756 | _mm_storeu_si32 (void *__P, __m128i __B)
      | ^~~~~~~~~~~~~~~
[852/5105] Generating src/x86_64-fma/2d-fourier-16x16.py.o
ninja: build stopped: subcommand failed.

ruestique commented on 2020-07-30 14:38

and after again

Building without cuda and without MKL-DNN
Building wheel torch-1.3.1
-- Building version 1.3.1
Could not find /home/chibo/.cache/pikaur/build/python-pytorch-git/src/pytorch-git/third_party/cpuinfo/CMakeLists.txt
Did you run 'git submodule update --init --recursive'?
==> ERROR: A failure occurred in build().
    Aborting...

ruestique commented on 2020-07-30 14:12

still having multiple sudo issue like this

cp: cannot create regular file 'pytorch-git/pytorch/.git/modules/third-party/cpuinfo/objects/pack/pack-3932b6e9b8a69893eddafdddbe1da7b17b17c4ea.idx': Permission denied

Iglu47 commented on 2020-07-14 23:05

impossible to install, conflicts with tbb libuv zstd packages

python-pytorch-cuda-git: /usr/lib/libtbb.so exists in filesystem (owned by tbb)
python-pytorch-cuda-git: /usr/lib/libuv.so exists in filesystem (owned by libuv)
python-pytorch-cuda-git: /usr/lib/libuv.so.1 exists in filesystem (owned by libuv)
python-pytorch-cuda-git: /usr/lib/libuv.so.1.0.0 exists in filesystem (owned by libuv)
python-pytorch-cuda-git: /usr/lib/libzstd.so exists in filesystem (owned by zstd)

ruestique commented on 2020-06-19 04:56

nah, caffe2 error again

Building without cuda and without MKL-DNN
Building wheel torch-1.3.1
-- Building version 1.3.1
cmake --build . --target install --config Release -- -j 8
make: *** No rule to make target 'install'.  Stop.
Traceback (most recent call last):
File "setup.py", line 732, in <module>
build_deps()
File "setup.py", line 311, in build_deps
build_caffe2(version=version,
File "/home/chibo/.cache/pikaur/build/python-pytorch-git/src/pytorch-git/tools/build_pytorch_libs.py", line 62, in build_caffe2
cmake.build(my_env)
File "/home/chibo/.cache/pikaur/build/python-pytorch-git/src/pytorch-git/tools/setup_helpers/cmake.py", line 345, in build
self.run(build_args, my_env)
File "/home/chibo/.cache/pikaur/build/python-pytorch-git/src/pytorch-git/tools/setup_helpers/cmake.py", line 141, in run
check_call(command, cwd=self.build_dir, env=env)
File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '8']' returned non-zero exit status 2.
==> ERROR: A failure occurred in build().
Aborting...

it also won't cuda compile, as far i know it necessary for rocm work