Package Details: ollama-cuda-git 0.4.7+r3696+gff6c2d6dc-1

Git Clone URL: https://aur.archlinux.org/ollama-cuda-git.git (read-only, click to copy)
Package Base: ollama-cuda-git
Description: Create, run and share large language models (LLMs) with CUDA
Upstream URL: https://github.com/ollama/ollama
Licenses: MIT
Conflicts: ollama
Provides: ollama, ollama-cuda
Submitter: sr.team
Maintainer: envolution
Last Packager: envolution
Votes: 4
Popularity: 0.92
First Submitted: 2024-02-22 23:22 (UTC)
Last Updated: 2024-12-01 01:18 (UTC)

Required by (26)

Sources (5)

Latest Comments

1 2 Next › Last »

envolution commented on 2024-12-05 15:38 (UTC)

until they merge https://github.com/ollama/ollama/pull/7499 there isn't a good way to manage this git package

envolution commented on 2024-12-01 02:18 (UTC)

@jfiguero @sarudosi

https://github.com/ollama/ollama/pull/7499 https://gitlab.archlinux.org/archlinux/packaging/packages/ollama-cuda/-/commits/main?ref_type=HEADS

It's being worked on. If anyone has a working build() to enable cuda, flag OOD and add it to comments please

jfiguero commented on 2024-11-29 18:41 (UTC) (edited on 2024-11-29 18:50 (UTC) by jfiguero)

I have installed ollama-cuda-git 0.4.6+r3691+gce7455a8e-1 and it does not use my GTX 1070 GPU but defaults to CPU. using ollama-cuda from extra and extra-testing does use it, but both packages are outdated.

I confirmed this using nvidia-smi, which won't show ollama as a running process, and see no change in power/RAM consumption while generating a response when using this package.

Here's my output for systemctl status ollama. Any suggestions on what I can look for to further debug?

ollama.service - Ollama Service Loaded: loaded (/usr/lib/systemd/system/ollama.service; enabled; preset: disabled) Drop-In: /etc/systemd/system/ollama.service.d └─override.conf Active: active (running) since Fri 2024-11-29 12:25:27 CST; 12min ago Invocation: 5fd8601c6424461a9f5c138297e19711 Main PID: 129202 (ollama) Tasks: 17 (limit: 37967) Memory: 60.4M (peak: 61.4M) CPU: 501ms CGroup: /system.slice/ollama.service └─129202 /usr/bin/ollama serve

Nov 29 12:25:27 hsxarch ollama[129202]: [GIN-debug] GET /api/tags --> github.com/ollama/ollama/server.(Server).ListHandler-fm (5 handlers) Nov 29 12:25:27 hsxarch ollama[129202]: [GIN-debug] GET /api/version --> github.com/ollama/ollama/server.(Server).GenerateRoutes.func2 (5 handlers) Nov 29 12:25:27 hsxarch ollama[129202]: [GIN-debug] HEAD / --> github.com/ollama/ollama/server.(Server).GenerateRoutes.func1 (5 handlers) Nov 29 12:25:27 hsxarch ollama[129202]: [GIN-debug] HEAD /api/tags --> github.com/ollama/ollama/server.(Server).ListHandler-fm (5 handlers) Nov 29 12:25:27 hsxarch ollama[129202]: [GIN-debug] HEAD /api/version --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers) Nov 29 12:25:27 hsxarch ollama[129202]: time=2024-11-29T12:25:27.215-06:00 level=INFO source=routes.go:1248 msg="Listening on [::]:11434 (version 0.4.6)" Nov 29 12:25:27 hsxarch ollama[129202]: time=2024-11-29T12:25:27.216-06:00 level=INFO source=common.go:135 msg="extracting embedded files" dir=/tmp/ollama649091418/runners Nov 29 12:25:27 hsxarch ollama[129202]: time=2024-11-29T12:25:27.300-06:00 level=INFO source=common.go:49 msg="Dynamic LLM libraries" runners="[cpu_avx2 cpu cpu_avx]" Nov 29 12:25:27 hsxarch ollama[129202]: time=2024-11-29T12:25:27.300-06:00 level=INFO source=gpu.go:221 msg="looking for compatible GPUs" Nov 29 12:25:27 hsxarch ollama[129202]: time=2024-11-29T12:25:27.492-06:00 level=INFO source=types.go:123 msg="inference compute" id=GPU-fe5de60e-7506-5f83-6fa3-4070b933c724 library=cuda variant=v12 compute=6.1 driver=12.7 name="NVIDIA GeForce GTX 1070" total="7.9 GiB" available="6.5 GiB"

sarudosi commented on 2024-11-26 06:48 (UTC)

I can not use GPU, using this pkg and extra/ollama-cuda. Do you have any idea to fix this issue? 

systemctl stutas ollama

11月 26 15:30:01 QC861 ollama[1474]: time=2024-11-26T15:30:01.430+09:00 level=WARN source=gpu.go:732 msg="unable to locate gpu dependency libraries"
11月 26 15:30:01 QC861 ollama[1474]: time=2024-11-26T15:30:01.430+09:00 level=DEBUG source=gpu.go:532 msg="gpu library search" globs="[libcudart.so* /var/lib/ollama/libcudart.so* /usr/local/cuda/lib64/libcuda>
11月 26 15:30:01 QC861 ollama[1474]: time=2024-11-26T15:30:01.439+09:00 level=DEBUG source=gpu.go:566 msg="discovered GPU libraries" paths=[/opt/cuda/lib64/libcudart.so.11.8.89]
11月 26 15:30:01 QC861 ollama[1474]: cudaSetDevice err: 100
11月 26 15:30:01 QC861 ollama[1474]: time=2024-11-26T15:30:01.444+09:00 level=DEBUG source=gpu.go:582 msg="Unable to load cudart library /opt/cuda/lib64/libcudart.so.11.8.89: cudart init failure: 100"
11月 26 15:30:01 QC861 ollama[1474]: time=2024-11-26T15:30:01.444+09:00 level=DEBUG source=amd_linux.go:416 msg="amdgpu driver not detected /sys/module/amdgpu"
11月 26 15:30:01 QC861 ollama[1474]: time=2024-11-26T15:30:01.444+09:00 level=INFO source=gpu.go:386 msg="no compatible GPUs were discovered"
11月 26 15:30:01 QC861 ollama[1474]: time=2024-11-26T15:30:01.444+09:00 level=INFO source=types.go:123 msg="inference compute" id=0 library=cpu variant=avx2 compute="" driver=0.0 name="" total="62.6 GiB" avai>
11月 26 15:44:48 QC861 ollama[1474]: [GIN] 2024/11/26 - 15:44:48 | 200 |     962.018µs |       127.0.0.1 | HEAD     "/"
11月 26 15:44:49 QC861 ollama[1474]: [GIN] 2024/11/26 - 15:44:49 | 200 |  572.186812ms |       127.0.0.1 | GET      "/api/tags"

sarudosi commented on 2024-11-26 06:43 (UTC) (edited on 2024-11-26 06:44 (UTC) by sarudosi)

Mr. brauliobo. You shuold modify PKGBUILD & makpkg -si.

- sed -i 's,T_CODE=on,T_CODE=on -D LLAMA_LTO=on -D CMAKE_BUILD_TYPE=Release,g' llm/generate/gen_linux.sh

+ sed -i 's,T_CODE=on,T_CODE=on -D LLAMA_LTO=on -D CMAKE_BUILD_TYPE=Release,g' scripts/build_linux.sh

brauliobo commented on 2024-11-14 18:59 (UTC)

got the error:

==> Validating source files with b2sums...
    ollama ... Skipped
    ollama.service ... Passed
    sysusers.conf ... Passed
    tmpfiles.d ... Passed
==> Removing existing $srcdir/ directory...
==> Extracting sources...
  -> Creating working copy of ollama git repo...
Cloning into 'ollama'...
done.
==> Starting prepare()...
sed: can't read llm/generate/gen_linux.sh: No such file or directory

sr.team commented on 2024-08-21 04:42 (UTC)

@JamesMowery you need preinstall makedepends before build package

JamesMowery commented on 2024-08-21 04:28 (UTC)

Getting the following error when installing on Nvidia 555 + Wayland + KDE Plasma.

+ gzip -n --best -f ../build/linux/x86_64/cpu/bin/ollama_llama_server
+ '[' -z '' ']'
+ '[' -d /usr/local/cuda/lib64 ']'
+ '[' -z '' ']'
+ '[' -d /opt/cuda/targets/x86_64-linux/lib ']'
+ CUDA_LIB_DIR=/opt/cuda/targets/x86_64-linux/lib
+ '[' -z '' ']'
+ CUDART_LIB_DIR=/opt/cuda/targets/x86_64-linux/lib
+ '[' -z '' -a -d /opt/cuda/targets/x86_64-linux/lib ']'
+ echo 'CUDA libraries detected - building dynamic CUDA library'
CUDA libraries detected - building dynamic CUDA library
+ init_vars
+ case "${GOARCH}" in
+ ARCH=x86_64
+ LLAMACPP_DIR=../llama.cpp
+ CMAKE_DEFS=-DCMAKE_SKIP_RPATH=on
+ CMAKE_TARGETS='--target ollama_llama_server'
+ echo '-march=native -mtune=generic -O2 -pipe -fno-plt'
+ grep -- -g
+ CMAKE_DEFS='-DCMAKE_BUILD_TYPE=Release -DLLAMA_SERVER_VERBOSE=off -DCMAKE_SKIP_RPATH=on'
+ case $(uname -s) in
++ uname -s
+ LIB_EXT=so
+ WHOLE_ARCHIVE=-Wl,--whole-archive
+ NO_WHOLE_ARCHIVE=-Wl,--no-whole-archive
+ GCC_ARCH=
+ DIST_BASE=../../dist/linux-amd64/
+ '[' -z '50;52;61;70;75;80' ']'
++ which pigz
++ echo gzip
+ GZIP=gzip
++ head -1
++ ls /opt/cuda/targets/x86_64-linux/lib/libcudart.so.12 /opt/cuda/targets/x86_64-linux/lib/libcudart.so.12.5.82
++ cut -f3 -d.
+ CUDA_MAJOR=12
+ '[' -n 12 -a -z '' ']'
+ CUDA_VARIANT=_v12
+ '[' x86_64 == arm64 ']'
+ '[' -n '' ']'
+ CMAKE_CUDA_DEFS='-DGGML_CUDA=on -DCMAKE_CUDA_ARCHITECTURES=50;52;61;70;75;80'
+ export CUDAFLAGS=-t8
+ CUDAFLAGS=-t8
+ CMAKE_DEFS='-DCMAKE_SKIP_RPATH=on -DBUILD_SHARED_LIBS=on -DCMAKE_POSITION_INDEPENDENT_CODE=on -D LLAMA_LTO=on -D CMAKE_BUILD_TYPE=Release -DGGML_NATIVE=off -DGGML_AVX=on -DGGML_AVX2=off -DGGML_AVX512=off -DGGML_FMA=off -DGGML_F16C=off -DGGML_OPENMP=off -DCMAKE_BUILD_TYPE=Release -DLLAMA_SERVER_VERBOSE=off -DCMAKE_SKIP_RPATH=on  -DGGML_CUDA=on -DCMAKE_CUDA_ARCHITECTURES=50;52;61;70;75;80 -DGGML_STATIC=off'
+ BUILD_DIR=../build/linux/x86_64/cuda_v12
+ export 'LLAMA_SERVER_LDFLAGS=-L/opt/cuda/targets/x86_64-linux/lib -lcudart -lcublas -lcublasLt -lcuda'
+ LLAMA_SERVER_LDFLAGS='-L/opt/cuda/targets/x86_64-linux/lib -lcudart -lcublas -lcublasLt -lcuda'
+ CUDA_DIST_DIR=../../dist/linux-amd64//lib/ollama
+ build
+ cmake -S ../llama.cpp -B ../build/linux/x86_64/cuda_v12 -DCMAKE_SKIP_RPATH=on -DBUILD_SHARED_LIBS=on -DCMAKE_POSITION_INDEPENDENT_CODE=on -D LLAMA_LTO=on -D CMAKE_BUILD_TYPE=Release -DGGML_NATIVE=off -DGGML_AVX=on -DGGML_AVX2=off -DGGML_AVX512=off -DGGML_FMA=off -DGGML_F16C=off -DGGML_OPENMP=off -DCMAKE_BUILD_TYPE=Release -DLLAMA_SERVER_VERBOSE=off -DCMAKE_SKIP_RPATH=on -DGGML_CUDA=on '-DCMAKE_CUDA_ARCHITECTURES=50;52;61;70;75;80' -DGGML_STATIC=off
-- The C compiler identification is GNU 14.2.1
-- The CXX compiler identification is GNU 14.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "2.46.0")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Using llamafile
-- Could not find nvcc, please set CUDAToolkit_ROOT.
CMake Warning at ggml/src/CMakeLists.txt:397 (message):
  CUDA not found


-- CUDA host compiler is GNU
CMake Error at ggml/src/CMakeLists.txt:984 (get_flags):
  get_flags Function invoked with incorrect arguments for function named:
  get_flags


-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- x86 detected
-- Configuring incomplete, errors occurred!
llm/generate/generate_linux.go:3: running "bash": exit status 1
==> ERROR: A failure occurred in build().
    Aborting...
 -> error making: ollama-cuda-git-exit status 4
 -> Failed to install the following packages. Manual intervention is required:
ollama-cuda-git - exit status 4

nmanarch commented on 2024-04-17 07:49 (UTC) (edited on 2024-04-29 10:12 (UTC) by nmanarch)

I have found a little trick showed by others on ollama git issue. So for those want ollama cuda run without avx try :

https://github.com/ollama/ollama/issues/2187#issuecomment-2082334649

Thanks to @sr.team and to all.

Hello. I apologized. Since 1.29 the gpu support without avx cpu is blocked in ollama. Did someone can help to have this to work again? https://github.com/ollama/ollama/issues/2187 and the bypass propose by dbzoo which work but not apply at the main. https://github.com/dbzoo/ollama/commit/45eb1048496780a78ed07cf39b3ce6b62b5a72e3 Many thanks.have a nice days.

nmanarch commented on 2024-04-04 14:23 (UTC)

Yes it is fixed and build and run.thanks.

1 2 Next › Last »