| Git Clone URL: | https://aur.archlinux.org/llama.cpp-cuda.git (read-only, click to copy) |
|---|---|
| Package Base: | llama.cpp-cuda |
| Description: | Port of Facebook's LLaMA model in C/C++ (with NVIDIA CUDA optimizations) |
| Upstream URL: | https://github.com/ggerganov/llama.cpp |
| Licenses: | MIT |
| Conflicts: | ggml, libggml, llama.cpp |
| Provides: | llama.cpp |
| Replaces: | llama.cpp-cuda-f16 |
| Submitter: | txtsd |
| Maintainer: | envolution |
| Last Packager: | envolution |
| Votes: | 11 |
| Popularity: | 1.70 |
| First Submitted: | 2024-10-26 20:17 (UTC) |
| Last Updated: | 2025-10-17 01:49 (UTC) |
Hi, thanks for submitting this package (and llama.cpp-cuda-f16 too)!
Is it possible to add GGML_CUDA_FA_ALL_QUANTS=ON to the build options (for this package and llama.cpp-cuda-f16)? This option gives more flexible KV cache quantization options (and combinations). The build time is indeed longer, but I personally don't think it's too bad. It'll be a nice addition for users seeking lower VRAM usage.
Just wanted to say thank you for getting this working. It's finally building now without me having to edit the PKGBUILD every time to force it to point to gcc-13. :)
@AlxQ should be okay now - thanks for the report
@AlxQ it's picking up the commit from the aur repo somehow - I can have a look into this later today.
.git/logs/refs/remotes/origin/HEAD:0000000000000000000000000000000000000000 9ca426da1fdf involution involution@gmail.com 1754071863 -0400 clone: from ssh://aur.archlinux.org/llama.cpp-cuda.git
It's definitely the right upstream code - just not properly reflected in the version strings
The current PKGBUILD builds only b982-9ca426d build
$ llama-server --version
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 4060 Ti, compute capability 8.9, VMM: yes
version: 982 (9ca426d)
built with cc (GCC) 15.1.1 20250729 for x86_64-pc-linux-gnu
The current PKGBUILD no longer compiles a native build due to makepkg providing the SOURCE_DATE_EPOCH environment variable. This variable is checked in ggml/CMakeLists.txt:
if (CMAKE_CROSSCOMPILING OR DEFINED ENV{SOURCE_DATE_EPOCH})
message(STATUS "Setting GGML_NATIVE_DEFAULT to OFF")
set(GGML_NATIVE_DEFAULT OFF)
else()
set(GGML_NATIVE_DEFAULT ON)
endif()
@AlxQ thanks, fixed
==> Validating source files with sha256sums...
llama.cpp-cuda-b6045.tar.gz ... FAILED
llama.cpp.conf ... Passed
llama.cpp.service ... Passed
==> ERROR: One or more files did not pass the validity check!
-> error making: llama.cpp-cuda-exit status 1
-> Failed to install the following packages. Manual intervention is required:
llama.cpp-cuda - exit status 1
I've had cuda installed long before the last reboot, idk what's the issue then.
edit: it seems to be because I used the fish shell. Building from bash works.
edit: unrelated but I found llama.cpp-vulkan to be 2-4 x faster.
just a couple of minor changes in the build
python modules are now identified as optional for the conversion script
changed from git to tar.gz due to the git repo initial sync being so large
Pinned Comments
txtsd commented on 2024-10-26 20:17 (UTC) (edited on 2024-12-06 14:15 (UTC) by txtsd)
Alternate versions
llama.cpp
llama.cpp-vulkan
llama.cpp-sycl-fp16
llama.cpp-sycl-fp32
llama.cpp-cuda
llama.cpp-cuda-f16
llama.cpp-hip