Arch Linux User Repository

Alternate versions

llama.cpp
llama.cpp-vulkan
llama.cpp-sycl-fp16
llama.cpp-sycl-fp32
llama.cpp-cuda
llama.cpp-cuda-f16
llama.cpp-hip

Currently working as of current version. Not getting the build error anymore. I just want to say thanks to @txtsd

Update 2: Disregard the "fix" I posted below. All the models are not loading into memory now (seems like the block/layer loading is busted) and my computer is crashing often. So I'm guessing there really is a deeper problem. This is so darn frustrating.

Update: I got it working by changing this in the PKGBUILD:

-DLLAMA_USE_SYSTEM_GGML=OFF

and then I had to delete the following files:

llama.cpp-cuda-f16: /usr/include/ggml-alloc.h exists in filesystem (owned by libggml-cuda-f16-git)
llama.cpp-cuda-f16: /usr/include/ggml-backend.h exists in filesystem (owned by libggml-cuda-f16-git)
llama.cpp-cuda-f16: /usr/include/ggml-blas.h exists in filesystem (owned by libggml-cuda-f16-git)
llama.cpp-cuda-f16: /usr/include/ggml-cann.h exists in filesystem (owned by libggml-cuda-f16-git)
llama.cpp-cuda-f16: /usr/include/ggml-cpp.h exists in filesystem (owned by libggml-cuda-f16-git)
llama.cpp-cuda-f16: /usr/include/ggml-cpu.h exists in filesystem (owned by libggml-cuda-f16-git)
llama.cpp-cuda-f16: /usr/include/ggml-cuda.h exists in filesystem (owned by libggml-cuda-f16-git)
llama.cpp-cuda-f16: /usr/include/ggml-metal.h exists in filesystem (owned by libggml-cuda-f16-git)
llama.cpp-cuda-f16: /usr/include/ggml-opt.h exists in filesystem (owned by libggml-cuda-f16-git)
llama.cpp-cuda-f16: /usr/include/ggml-rpc.h exists in filesystem (owned by libggml-cuda-f16-git)
llama.cpp-cuda-f16: /usr/include/ggml-sycl.h exists in filesystem (owned by libggml-cuda-f16-git)
llama.cpp-cuda-f16: /usr/include/ggml-vulkan.h exists in filesystem (owned by libggml-cuda-f16-git)
llama.cpp-cuda-f16: /usr/include/ggml.h exists in filesystem (owned by libggml-cuda-f16-git)
llama.cpp-cuda-f16: /usr/include/gguf.h exists in filesystem (owned by libggml-cuda-f16-git)
llama.cpp-cuda-f16: /usr/lib/cmake/ggml/ggml-config.cmake exists in filesystem (owned by libggml-cuda-f16-git)
llama.cpp-cuda-f16: /usr/lib/cmake/ggml/ggml-version.cmake exists in filesystem (owned by libggml-cuda-f16-git)
llama.cpp-cuda-f16: /usr/lib/libggml-base.so exists in filesystem (owned by libggml-cuda-f16-git)
llama.cpp-cuda-f16: /usr/lib/libggml-cpu.so exists in filesystem (owned by libggml-cuda-f16-git)
llama.cpp-cuda-f16: /usr/lib/libggml.so exists in filesystem (owned by libggml-cuda-f16-git)

After doing that, It appears to be working just fine now.

I don't know how to modify the PKGBUILD to make it delete all those files automatically and stuff. I know it's probably very bad to do it this way. But I'm glad I got a temporary fix in order until this gets sorted.

Must be something about recent commits then. llama.cpp is a very fast moving software and is prone to bugs during building due to the nature of the package.

I'm getting the same error that @JamesMowery is getting. I've tried a rebuild of both lib.ggml and both versions of the llama.cpp-cuda as well.

@txtsd If it's at all helpful, I went ahead and tried to install the non fp16 package (as I had never used that; not even sure if fp16 is helpful with a 4090), and I also got the same exact error despite using those two completely new packages. So something is definitely a bit wrong.

AI says that some function is asking for a different number of arguments instead of what is being provided, likely due to a recent update? Not sure if that's at all accurate, but I hope it helps!

If you need me to provide any additional logging please let me know! (And if you would like that, please let me know how about to do that.) It's very much appreciated!

@txtsd I actually went ahead and deleted and re-added that package (I also deleted the cache) even before posting below and got that error. However, just to be sure, I just ran paru -S libggml-cuda-f16-git --rebuild and then after attempt to install paru llama.cpp-cuda-f16 again (also re-ran that with --rebuild too) and I got the same exact error. :(

@JamesMowery Rebuild libggml

This package started working a few days after my prior post. I just went to upgrade today (I usually update every Friday, today was an exception) and I'm getting this error.

[ 31%] Building CXX object examples/gguf-hash/CMakeFiles/llama-gguf-hash.dir/gguf-hash.cpp.o
[ 31%] Linking CXX executable ../../bin/rpc-server
[ 31%] Built target llama-gguf
/mnt/big/.cache/paru/clone/llama.cpp-cuda-f16/src/llama.cpp/src/llama-model.cpp: In function ‘bool weight_buft_supported(const llama_hparams&, ggml_tensor*, ggml_op, ggml_backend_buffer_type_t, ggml_backend_dev_t)’:
/mnt/big/.cache/paru/clone/llama.cpp-cuda-f16/src/llama.cpp/src/llama-model.cpp:231:42: error: too many arguments to function ‘ggml_tensor* ggml_ssm_scan(ggml_context*, ggml_tensor*, ggml_tensor*, ggml_tensor*, ggml_tensor*, ggml_tensor*, ggml_tensor*)’
  231 |                 op_tensor = ggml_ssm_scan(ctx, s, x, dt, w, B, C, ids);
      |                             ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /mnt/big/.cache/paru/clone/llama.cpp-cuda-f16/src/llama.cpp/src/../include/llama.h:4,
                 from /mnt/big/.cache/paru/clone/llama.cpp-cuda-f16/src/llama.cpp/src/llama-model.h:3,
                 from /mnt/big/.cache/paru/clone/llama.cpp-cuda-f16/src/llama.cpp/src/llama-model.cpp:1:
/usr/include/ggml.h:2009:35: note: declared here
 2009 |     GGML_API struct ggml_tensor * ggml_ssm_scan(
      |                                   ^~~~~~~~~~~~~
/mnt/big/.cache/paru/clone/llama.cpp-cuda-f16/src/llama.cpp/src/llama-model.cpp: In lambda function:
/mnt/big/.cache/paru/clone/llama.cpp-cuda-f16/src/llama.cpp/src/llama-model.cpp:9922:37: error: too many arguments to function ‘ggml_tensor* ggml_ssm_scan(ggml_context*, ggml_tensor*, ggml_tensor*, ggml_tensor*, ggml_tensor*, ggml_tensor*, ggml_tensor*)’
 9922 |                 return ggml_ssm_scan(ctx, ssm, x, dt, A, B, C, ids);
      |                        ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/ggml.h:2009:35: note: declared here
 2009 |     GGML_API struct ggml_tensor * ggml_ssm_scan(
      |                                   ^~~~~~~~~~~~~
/mnt/big/.cache/paru/clone/llama.cpp-cuda-f16/src/llama.cpp/src/llama-model.cpp: In lambda function:
/mnt/big/.cache/paru/clone/llama.cpp-cuda-f16/src/llama.cpp/src/llama-model.cpp:10046:37: error: too many arguments to function ‘ggml_tensor* ggml_ssm_scan(ggml_context*, ggml_tensor*, ggml_tensor*, ggml_tensor*, ggml_tensor*, ggml_tensor*, ggml_tensor*)’
10046 |                 return ggml_ssm_scan(ctx, ssm, x, dt, A, B, C, ids);
      |                        ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/ggml.h:2009:35: note: declared here
 2009 |     GGML_API struct ggml_tensor * ggml_ssm_scan(
      |                                   ^~~~~~~~~~~~~
[ 31%] Built target rpc-server
[ 32%] Linking CXX executable ../../bin/llama-gguf-hash
In function ‘SHA1Update’,
    inlined from ‘SHA1Final’ at /mnt/big/.cache/paru/clone/llama.cpp-cuda-f16/src/llama.cpp/examples/gguf-hash/deps/sha1/sha1.c:269:9:
/mnt/big/.cache/paru/clone/llama.cpp-cuda-f16/src/llama.cpp/examples/gguf-hash/deps/sha1/sha1.c:219:13: warning: ‘SHA1Transform’ reading 64 bytes from a region of size 0 [-Wstringop-overread]
  219 |             SHA1Transform(context->state, &data[i]);
      |             ^
/mnt/big/.cache/paru/clone/llama.cpp-cuda-f16/src/llama.cpp/examples/gguf-hash/deps/sha1/sha1.c:219:13: note: referencing argument 2 of type ‘const unsigned char[64]’
/mnt/big/.cache/paru/clone/llama.cpp-cuda-f16/src/llama.cpp/examples/gguf-hash/deps/sha1/sha1.c: In function ‘SHA1Final’:
/mnt/big/.cache/paru/clone/llama.cpp-cuda-f16/src/llama.cpp/examples/gguf-hash/deps/sha1/sha1.c:54:6: note: in a call to function ‘SHA1Transform’
   54 | void SHA1Transform(
      |      ^
[ 32%] Built target llama-gguf-hash
make[2]: *** [src/CMakeFiles/llama.dir/build.make:359: src/CMakeFiles/llama.dir/llama-model.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/Makefile2:1030: src/CMakeFiles/llama.dir/all] Error 2
make: *** [Makefile:136: all] Error 2
==> ERROR: A failure occurred in build().
    Aborting...
error: failed to build 'llama.cpp-cuda-f16-b5814-1':
error: packages failed to build: llama.cpp-cuda-f16-b5814-1