Arch Linux User Repository

I now get this compile error:

[ 84%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/unicode-data.cpp.o
/home/evert/Aur/whisper.cpp-cuda/src/whisper.cpp-1.7.6/examples/talk-llama/llama-graph.cpp: In member function ‘ggml_tensor* llm_graph_context::build_moe_ffn(ggml_tensor*, ggml_tensor*, ggml_tensor*, ggml_tensor*, ggml_tensor*, ggml_tensor*, int64_t, int64_t, llm_ffn_op_type, bool, bool, float, llama_expert_gating_func_type, int) const’:
/home/evert/Aur/whisper.cpp-cuda/src/whisper.cpp-1.7.6/examples/talk-llama/llama-graph.cpp:722:34: error: ‘ggml_repeat_4d’ was not declared in this scope; did you mean ‘ggml_repeat’?
  722 |         ggml_tensor * repeated = ggml_repeat_4d(ctx0, cur, n_embd, n_expert_used, n_tokens, 1);
      |                                  ^~~~~~~~~~~~~~
      |                                  ggml_repeat
[ 85%] Linking CXX executable ../../bin/vad-speech-segments
make[2]: *** [examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/build.make:205: examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-graph.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....

it the maintainer active? last update is more than a month ago...

I had to add this to make it build cos ggml_backend_sched_new() seemingly needs an extra arg now:

prepare() {
  cd "${srcdir}/${_pkgbase}-${pkgver}"
  sed -i '602s~.*~    sched = ggml_backend_sched_new(backends.data(), nullptr, backends.size(), WHISPER_MAX_NODES, false, false);~' src/whisper.cpp
  sed -i '9833s~.*~            ctx->sched.reset(ggml_backend_sched_new(backend_ptrs.data(), backend_buft.data(), backend_ptrs.size(), max_nodes, pipeline_parallel, false));~' examples/talk-llama/llama.cpp
  sed -i '4i #include <cstdint>' examples/talk-llama/llama-mmap.h
}

If you get this error:

[ 84%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-mmap.cpp.o
In file included from /home/henrique/.cache/pikaur/build/whisper.cpp-cuda/src/whisper.cpp-1.7.5/examples/talk-llama/llama-mmap.cpp:1:
/home/henrique/.cache/pikaur/build/whisper.cpp-cuda/src/whisper.cpp-1.7.5/examples/talk-llama/llama-mmap.h:26:5: error: ‘uint32_t’ does not name a type
26 |     uint32_t read_u32() const;
|     ^~~~~~~~
/home/henrique/.cache/pikaur/build/whisper.cpp-cuda/src/whisper.cpp-1.7.5/examples/talk-llama/llama-mmap.h:5:1: note: ‘uint32_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
4 | #include <vector>
+++ |+#include <cstdint>
5 |

Just add this to PKGBUILD:

prepare() {
  cd "${srcdir}/${_pkgbase}-${pkgver}"
  sed -i '4i #include <cstdint>' examples/talk-llama/llama-mmap.h
}

Adding -DCMAKE_CUDA_ARCHITECTURES=89-real is not always a good idea, because not every GPU supports such a high compute capability. My RTX2060 fails during runtime (not compilation) with this line. As for me, changing it to -DCMAKE_CUDA_ARCHITECTURES=75 (compute capability 7.5) fixes the problem. You may need to check https://developer.nvidia.com/cuda-gpus for the compute capability of a given GPU model.

whisper-cli: error while loading shared libraries: libcuda.so.1: cannot open shared object file: No such file or directory

libcuda.so.1 is in package nvidia-utils, so nvidai-utils should be a dependency

ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: 520

fixed by adding -DCMAKE_CUDA_ARCHITECTURES=89-real https://github.com/leejet/stable-diffusion.cpp/issues/541

tested

llama.cpp-cuda: /usr/lib/libggml-base.so exists in the file system (owned by whisper.cpp-cuda)
llama.cpp-cuda: /usr/lib/libggml-cpu.so exists in the file system (owned by whisper.cpp-cuda)
llama.cpp-cuda: /usr/lib/libggml-cuda.so exists in the file system (owned by whisper.cpp-cuda)
llama.cpp-cuda: /usr/lib/libggml.so exists in the file system (owned by whisper.cpp-cuda)
An error occurred, and no packages were updated.
-> Error during installation: [/home/chi/.cache/yay/llama.cpp-cuda/llama.cpp-cuda-b4762-1-x86_64.pkg.tar.zst] - exit status 1

whisper-cli: error while loading shared libraries: libcuda.so.1: cannot open shared object file: No such file or directory

$ ldd `which whisper-cli`
        linux-vdso.so.1 (0x00007169ac5d4000)
        libwhisper.so.1 => /usr/lib/libwhisper.so.1 (0x00007169ac4f8000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007169ac200000)
        libm.so.6 => /usr/lib/libm.so.6 (0x00007169ac108000)
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007169ac4ca000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007169abf16000)
        libggml.so => /usr/lib/libggml.so (0x00007169ac4bb000)
        libggml-cpu.so => /usr/lib/libggml-cpu.so (0x00007169abe76000)
        libggml-base.so => /usr/lib/libggml-base.so (0x00007169abdbc000)
        /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007169ac5d6000)
        libggml-cuda.so => /usr/lib/libggml-cuda.so (0x00007169a8c00000)
        libgomp.so.1 => /usr/lib/libgomp.so.1 (0x00007169abd69000)
        libcudart.so.12 => /opt/cuda/lib64/libcudart.so.12 (0x00007169a8800000)
        libcublas.so.12 => /opt/cuda/lib64/libcublas.so.12 (0x00007169a1600000)
        libcuda.so.1 => not found
        libdl.so.2 => /usr/lib/libdl.so.2 (0x00007169ac4b2000)
        libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007169ac4ad000)
        librt.so.1 => /usr/lib/librt.so.1 (0x00007169ac4a8000)
        libcublasLt.so.12 => /opt/cuda/lib64/libcublasLt.so.12 (0x000071696d800000)

$ fd libcuda
opt/cuda/targets/x86_64-linux/lib/libcudart.so
opt/cuda/targets/x86_64-linux/lib/libcudadevrt.a
opt/cuda/targets/x86_64-linux/lib/libcudart.so.12
opt/cuda/targets/x86_64-linux/lib/libcudart.so.12.8.57
opt/cuda/targets/x86_64-linux/lib/cmake/libcudacxx/
opt/cuda/targets/x86_64-linux/lib/cmake/libcudacxx/libcudacxx-config.cmake
opt/cuda/targets/x86_64-linux/lib/cmake/libcudacxx/libcudacxx-config-version.cmake
opt/cuda/targets/x86_64-linux/lib/cmake/libcudacxx/libcudacxx-header-search.cmake
opt/cuda/targets/x86_64-linux/lib/libcudart_static.a
opt/cuda/targets/x86_64-linux/lib/stubs/libcuda.so