Arch Linux User Repository

So close!

-- Installing: /home/evert/Aur/whisper.cpp-cuda/pkg/whisper.cpp-cuda/usr/lib/pkgconfig/whisper.pc
/home/evert/Aur/whisper.cpp-cuda/PKGBUILD: line 40: cd: /home/evert/Aur/whisper.cpp-cuda/src/build/bin: No such file or directory
==> ERROR: A failure occurred in package().
    Aborting...
 -> error making: whisper.cpp-cuda-exit status 4
 -> Failed to install the following packages. Manual intervention is required:
whisper.cpp-cuda - exit status 4
evert@Evert ~ [1]> 

I get this error at 95%:

/home/ashish/.cache/yay/whisper.cpp-cuda/src/whisper.cpp-1.7.6/examples/talk-llama/llama-model.cpp:224:42: error: too few arguments to function ‘ggml_tensor* ggml_ssm_scan(ggml_context*, ggml_tensor*, ggml_tensor*, ggml_tensor*, ggml_tensor*, ggml_tensor*, ggml_tensor*, ggml_tensor*)’
  224 |                 op_tensor = ggml_ssm_scan(ctx, s, x, dt, w, B, C);
      |                             ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~

Fails to build in a clean chroot with:

/usr/bin/ld: warning: libcuda.so.1, needed by /usr/lib/libggml-cuda.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: /usr/lib/libggml-cuda.so: undefined reference to `cuMemCreate'
/usr/bin/ld: /usr/lib/libggml-cuda.so: undefined reference to `cuMemAddressReserve'
/usr/bin/ld: /usr/lib/libggml-cuda.so: undefined reference to `cuMemUnmap'
/usr/bin/ld: /usr/lib/libggml-cuda.so: undefined reference to `cuMemSetAccess'
/usr/bin/ld: /usr/lib/libggml-cuda.so: undefined reference to `cuDeviceGet'
/usr/bin/ld: /usr/lib/libggml-cuda.so: undefined reference to `cuMemAddressFree'
/usr/bin/ld: /usr/lib/libggml-cuda.so: undefined reference to `cuGetErrorString'
/usr/bin/ld: /usr/lib/libggml-cuda.so: undefined reference to `cuDeviceGetAttribute'
/usr/bin/ld: /usr/lib/libggml-cuda.so: undefined reference to `cuMemMap'
/usr/bin/ld: /usr/lib/libggml-cuda.so: undefined reference to `cuMemRelease'
/usr/bin/ld: /usr/lib/libggml-cuda.so: undefined reference to `cuMemGetAllocationGranularity'
collect2: error: ld returned 1 exit status
make[2]: *** [tests/CMakeFiles/test-vad.dir/build.make:107: bin/test-vad] Error 1
make[1]: *** [CMakeFiles/Makefile2:1418: tests/CMakeFiles/test-vad.dir/all] Error 2
make: *** [Makefile:146: all] Error 2

Had to add nvidia-utils to dependencies to get it to build.

I now get this compile error:

[ 84%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/unicode-data.cpp.o
/home/evert/Aur/whisper.cpp-cuda/src/whisper.cpp-1.7.6/examples/talk-llama/llama-graph.cpp: In member function ‘ggml_tensor* llm_graph_context::build_moe_ffn(ggml_tensor*, ggml_tensor*, ggml_tensor*, ggml_tensor*, ggml_tensor*, ggml_tensor*, int64_t, int64_t, llm_ffn_op_type, bool, bool, float, llama_expert_gating_func_type, int) const’:
/home/evert/Aur/whisper.cpp-cuda/src/whisper.cpp-1.7.6/examples/talk-llama/llama-graph.cpp:722:34: error: ‘ggml_repeat_4d’ was not declared in this scope; did you mean ‘ggml_repeat’?
  722 |         ggml_tensor * repeated = ggml_repeat_4d(ctx0, cur, n_embd, n_expert_used, n_tokens, 1);
      |                                  ^~~~~~~~~~~~~~
      |                                  ggml_repeat
[ 85%] Linking CXX executable ../../bin/vad-speech-segments
make[2]: *** [examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/build.make:205: examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-graph.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....

it the maintainer active? last update is more than a month ago...

I had to add this to make it build cos ggml_backend_sched_new() seemingly needs an extra arg now:

prepare() {
  cd "${srcdir}/${_pkgbase}-${pkgver}"
  sed -i '602s~.*~    sched = ggml_backend_sched_new(backends.data(), nullptr, backends.size(), WHISPER_MAX_NODES, false, false);~' src/whisper.cpp
  sed -i '9833s~.*~            ctx->sched.reset(ggml_backend_sched_new(backend_ptrs.data(), backend_buft.data(), backend_ptrs.size(), max_nodes, pipeline_parallel, false));~' examples/talk-llama/llama.cpp
  sed -i '4i #include <cstdint>' examples/talk-llama/llama-mmap.h
}

If you get this error:

[ 84%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-mmap.cpp.o
In file included from /home/henrique/.cache/pikaur/build/whisper.cpp-cuda/src/whisper.cpp-1.7.5/examples/talk-llama/llama-mmap.cpp:1:
/home/henrique/.cache/pikaur/build/whisper.cpp-cuda/src/whisper.cpp-1.7.5/examples/talk-llama/llama-mmap.h:26:5: error: ‘uint32_t’ does not name a type
26 |     uint32_t read_u32() const;
|     ^~~~~~~~
/home/henrique/.cache/pikaur/build/whisper.cpp-cuda/src/whisper.cpp-1.7.5/examples/talk-llama/llama-mmap.h:5:1: note: ‘uint32_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
4 | #include <vector>
+++ |+#include <cstdint>
5 |

Just add this to PKGBUILD:

prepare() {
  cd "${srcdir}/${_pkgbase}-${pkgver}"
  sed -i '4i #include <cstdint>' examples/talk-llama/llama-mmap.h
}

Adding -DCMAKE_CUDA_ARCHITECTURES=89-real is not always a good idea, because not every GPU supports such a high compute capability. My RTX2060 fails during runtime (not compilation) with this line. As for me, changing it to -DCMAKE_CUDA_ARCHITECTURES=75 (compute capability 7.5) fixes the problem. You may need to check https://developer.nvidia.com/cuda-gpus for the compute capability of a given GPU model.

whisper-cli: error while loading shared libraries: libcuda.so.1: cannot open shared object file: No such file or directory

libcuda.so.1 is in package nvidia-utils, so nvidai-utils should be a dependency

ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: 520

fixed by adding -DCMAKE_CUDA_ARCHITECTURES=89-real https://github.com/leejet/stable-diffusion.cpp/issues/541

tested