Package Details: llama.cpp-cuda-git b9893.r0.6f8895feec-1

Git Clone URL: https://aur.archlinux.org/llama.cpp-cuda-git.git (read-only, click to copy)
Package Base: llama.cpp-cuda-git
Description: Port of Facebook's LLaMA model in C/C++ (with NVIDIA CUDA optimizations)
Upstream URL: https://github.com/ggml-org/llama.cpp
Licenses: MIT
Conflicts: llama.cpp
Provides: llama.cpp
Submitter: Bink
Maintainer: Bink
Last Packager: Bink
Votes: 6
Popularity: 2.20
First Submitted: 2026-01-08 09:17 (UTC)
Last Updated: 2026-07-07 05:16 (UTC)

Dependencies (17)

Required by (13)

Sources (3)

Latest Comments

« First ‹ Previous 1 2 3 4 5 Next › Last »

Saluu commented on 2026-05-07 10:55 (UTC) (edited on 2026-05-07 10:57 (UTC) by Saluu)

OpenBLAS was added in commit bc45cc2 (3 May 2026). It causes a severe prompt-processing regression for any model that partially offloads layers to CPU (ngl < all layers). On a 14B Q4_K_M model with 36/49 layers on GPU, pp1024 drops from ~1450 t/s to ~120 t/s, roughly 12x slower.

The BLAS backend registers before the native CPU backend in ggml's scheduler (ggml-backend-reg.cpp). It claims MUL_MAT ops for CPU-side layers, forcing a dequantize-to-F32 + cblas_sgemm path instead of the native backend's optimized quantized vec_dot kernels. This is fundamentally slower for LLM inference on quantized models.

This is a known issue: https://github.com/ggml-org/llama.cpp/issues/5986

BLAS can benefit pure-CPU F32 inference, but for a CUDA package the common use case is partial GPU offload with quantized models, where BLAS is harmful. Making BLAS opt-in rather than the default would avoid this regression for most users while keeping it available for those who need it.

Suggested fix: remove -DGGML_BLAS=ON, -DGGML_BLAS_VENDOR=OpenBLAS, and the openblas dependency, or make them conditional on an opt-in variable (like the existing aur_llamacpp_build_universal pattern).

Workaround: aur_llamacpp_cmakeopts="-DGGML_BLAS=OFF" makepkg -si

aydintb commented on 2026-05-04 08:04 (UTC)

dependency build of cpp15 fails.

aydintb commented on 2026-05-04 08:03 (UTC)

gcc/lto1 differs make[2]: *** [Makefile:25123: compare] Error 1 make[1]: *** [Makefile:25103: stage3-bubble] Error 2 make: *** [Makefile:25166: bootstrap] Error 2 ==> ERROR: A failure occurred in build(). Aborting... -> error making: gcc15-exit status 4 -> nothing to install for gcc15 ==> Making package: llama.cpp-cuda-git b9010.r0.d05fe1d7da-2 (Pzt 04 May 2026 11:01:23) ==> Checking runtime dependencies... ==> Checking buildtime dependencies... ==> Missing dependencies: -> gcc15 ==> ERROR: Could not resolve all dependencies. -> error making: llama.cpp-cuda-git-exit status 8 -> Failed to install the following packages. Manual intervention is required: gcc15 - exit status 4 llama.cpp-cuda-git - exit status 8

ultramango commented on 2026-05-03 16:31 (UTC)

@Bink, thanks for the update! I confirm it now compiles without any tricks.

Bink commented on 2026-05-03 04:56 (UTC)

@ultramango, it should now be working again, with the specified gcc15 make dependency. Unfortunatly, for now, this means gcc15 needs to be compiled, which takes a long time. I suggest keeping gcc15 installed so you don't have to do that every update.

Hopefully GCC 16 support is implemented upstream soon.

Bink commented on 2026-05-03 02:59 (UTC)

The recent version bump of gcc from gcc15 to gcc16 has tripped this up. I'm reviewing options still. Ideally it'd still be using gcc15 for maximum optimisation, but that doesn't yet have an AUR package.

ultramango commented on 2026-05-02 20:38 (UTC) (edited on 2026-05-02 20:54 (UTC) by ultramango)

In case you encounter problems with nvcc/gcc compilation errors (/usr/include/c++/16.1.1/type_traits(1448): error: identifier "__f" is undefined):

# Note: you should have a GCC 13 installed
$ export CUDAHOSTCXX=/usr/bin/g++-13
$ makepkg

You might have a too new gcc (16 as of writing this comment) for nvcc.

TeddyHuang-00 commented on 2026-04-23 17:32 (UTC)

@Bink Thank you for the patch. I have verified the install and I believe it have resolved the installation issue.

Bink commented on 2026-04-23 10:08 (UTC)

Thanks for sharing these details! I don't have fish shell, but I've pushed an update that hopefully handles this better anyway. Let me know if that resolves it.

« First ‹ Previous 1 2 3 4 5 Next › Last »