AUR (en) - llama.cpp-cuda-git

Search Criteria

Enter search criteria

Search by

Keywords

Out of Date

Sort by

Sort order

Per page

Package Details: llama.cpp-cuda-git b9893.r0.6f8895feec-1

Package Actions

Git Clone URL:	https://aur.archlinux.org/llama.cpp-cuda-git.git (read-only, click to copy)
Package Base:	llama.cpp-cuda-git
Description:	Port of Facebook's LLaMA model in C/C++ (with NVIDIA CUDA optimizations)
Upstream URL:	https://github.com/ggml-org/llama.cpp
Licenses:	MIT
Conflicts:	llama.cpp
Provides:	llama.cpp
Submitter:	Bink
Maintainer:	Bink
Last Packager:	Bink
Votes:	6
Popularity:	2.20
First Submitted:	2026-01-08 09:17 (UTC)
Last Updated:	2026-07-07 05:16 (UTC)

Dependencies (17)

curl (curl-git^AUR, curl-c-ares^AUR)
gcc-libs (gcc-libs-git^AUR, gcc-libs-fast-optimized^AUR, gccrs-libs-git^AUR, gcc-libs-snapshot^AUR)
ggml-cuda-git^AUR (ggml-cuda-git^AUR)
glibc (glibc-git^AUR, glibc-git-native-pgo^AUR, glibc-eac^AUR)
openssl (openssl-git^AUR, openssl-aegis^AUR, openssl-static^AUR)
cmake (cmake3^AUR, cmake-git^AUR) (make)
cuda (cuda11.1^AUR, cuda-12.2^AUR, cuda12.0^AUR, cuda11.4^AUR, cuda-12.5^AUR, cuda-12.8^AUR, cuda-pascal^AUR, cuda-12.9^AUR) (make)
git (git-git^AUR, git-gl^AUR, git-wd40^AUR) (make)
ninja (ninja-git^AUR, ninja-mem^AUR, ninja-noemacs-git^AUR, ninja-kitware^AUR, ninja-fuchsia-git^AUR, n2-git^AUR) (make)
ccache (ccache-git^AUR) (optional) – greatly reduce package re-build time
nccl (nccl-git^AUR, nccl-cuda12.9^AUR) (optional) – needed for multi-GPU parallelism
python-numpy (python-numpy-git^AUR, python-numpy-mkl-bin^AUR, python-numpy1^AUR, python-numpy-mkl-tbb^AUR, python-numpy-mkl^AUR) (optional) – needed for convert_hf_to_gguf.py
python-pytorch (python-pytorch-cuda12.9^AUR, python-pytorch-opt-cuda12.9^AUR, python-pytorch-cuda, python-pytorch-opt, python-pytorch-opt-cuda, python-pytorch-opt-rocm, python-pytorch-opt-xpu, python-pytorch-rocm, python-pytorch-xpu) (optional) – needed for convert_hf_to_gguf.py
python-safetensors (optional) – needed for convert_hf_to_gguf.py
python-sentencepiece^AUR (python-sentencepiece-git^AUR, python-sentencepiece-bin^AUR) (optional) – needed for convert_hf_to_gguf.py
python-transformers^AUR (python-transformers-git^AUR) (optional) – needed for convert_hf_to_gguf.py
rdma-core (rdma-core-git^AUR) (optional) – RDMA transport for RPC backend

Required by (13)

assistd (requires llama.cpp) (optional)
assistd-git (requires llama.cpp) (optional)
llamaman-bin (requires llama.cpp) (optional)
llamastash (requires llama.cpp) (optional)
llamastash-bin (requires llama.cpp) (optional)
llamastash-git (requires llama.cpp) (optional)
manboster (requires llama.cpp) (optional)
manboster-bin (requires llama.cpp) (optional)
manboster-git (requires llama.cpp) (optional)
scmd-bin (requires llama.cpp)
voxd (requires llama.cpp) (optional)
voxd-bin (requires llama.cpp) (optional)
voxd-git (requires llama.cpp) (optional)

Sources (3)

Latest Comments

« First ‹ Previous 1 2 3 4 5 Next › Last »

jamesb6626 commented on 2026-05-21 16:29 (UTC) (edited on 2026-05-21 16:29 (UTC) by jamesb6626)

@Bink I figured out why the version keeps getting set wrong (described in my previous comment). When pkgver() changes the package version, the PKGBUILD is written to with the new $pkgver and is then rerun, overwriting $_build_number and $_commit_id with the blank values. This occurs after the run of prepare(), but only when the $pkgver changes. Hence why using makepkg a second time does not reproduce the issue, there has to be a change in the version of llama.cpp.

I don't have experience with PKGBUILD writing, but I think you need to treat variables set in prepare() as possibly overwritten in PKGBUILD. So either these should be set in build(), or maybe prepare() writes them to a file that build() will use, or something else. Idk, not my call as to what your view of a clean way to set those is. Hope this helps.

By the way, been getting a lot of use out of this, so thanks, super useful :)

yataro commented on 2026-05-21 08:46 (UTC) (edited on 2026-05-21 10:37 (UTC) by yataro)

With -DBUILD_SHARED_LIBS=OFF you will build all the libraries as static, I was talking about linking only ggml statically. If you keep llama library shared (libllama.so) I suspect there would be no blow out, but I didn't test it.

EDIT: Just tested it with static ggml, total size of package is 800MB, which is still way too big comparing to current setup. This happens because some executables link ggml directly, so I guess it is better to keep things in current state. Sorry for noise!

Bink commented on 2026-05-21 00:13 (UTC) (edited on 2026-05-21 00:14 (UTC) by Bink)

You might try experimenting with statically linking @yataro (-DBUILD_SHARED_LIBS=OFF). In my tests, the 43 statically linked binaries built for the package blow out in size, adding almost 7GB to the installed package size. Most executables go from being less than 100k in size, to now being about 170MB each.

If only a single executable is ever running at any given time (eg llama-server), that's probably acceptable, but I guess it depends how people are using this and other GGML related packages.

yataro commented on 2026-05-20 05:35 (UTC)

Okay, I see. Personally, I would go for static link of ggml (also in sake of fine-tuning), but considering state of libggml-cuda-git, providing working ggml is logical. Thank you for reply!

Bink commented on 2026-05-20 05:18 (UTC) (edited on 2026-05-20 05:20 (UTC) by Bink)

In one sense the decision, originally at least, was made for me.

I'd originally made this package as a -git version of the existing llama.cpp-cuda package. You'll note that package (currently) continues to take the same approach.
Building the GGML libraries in this package allows me to fine tune their optimisation, as they're the backbone to leveraging CUDA, which underpins the reason this package exists.

People had noted conflicts, but I addressed that to an extent by adding the provides section, allowing this package to perform as the dependency for those other packages (eg: satisfies libggml-cuda-git dependency for whisper.cpp-cuda).

Critically, there are unresolved build issue (in my testing at least) with libggml-cuda-git which would be the alternative.

yataro commented on 2026-05-20 04:51 (UTC)

@Bink may I know why you decided to go this way?

Bink commented on 2026-05-19 22:46 (UTC) (edited on 2026-05-19 22:48 (UTC) by Bink)

That's correct @yataro, it does build the ggml libs and acts as a provider for those. It's detailed in the PKGBUILD here:

provides=(
  "${_pkgname}"
  libggml-cuda-git
  libggml
  libggml.so
  ggml
)
conflicts=(
  "${_pkgname}"
  libggml
  ggml
)

That has been my intention so far, but I have been considering offloading that to a common dependency such as libggml-cuda-git.

yataro commented on 2026-05-19 09:16 (UTC)

This package installs ggml headers and libraries, I'm pretty sure that this would conflict with ggml packages.

$ pacman -Ql llama.cpp-cuda-git | grep ggml
llama.cpp-cuda-git /usr/bin/llama-convert-llama2c-to-ggml
llama.cpp-cuda-git /usr/include/ggml-alloc.h
llama.cpp-cuda-git /usr/include/ggml-backend.h
llama.cpp-cuda-git /usr/include/ggml-blas.h
llama.cpp-cuda-git /usr/include/ggml-cann.h
llama.cpp-cuda-git /usr/include/ggml-cpp.h
llama.cpp-cuda-git /usr/include/ggml-cpu.h
llama.cpp-cuda-git /usr/include/ggml-cuda.h
llama.cpp-cuda-git /usr/include/ggml-metal.h
llama.cpp-cuda-git /usr/include/ggml-openvino.h
llama.cpp-cuda-git /usr/include/ggml-opt.h
llama.cpp-cuda-git /usr/include/ggml-rpc.h
llama.cpp-cuda-git /usr/include/ggml-sycl.h
llama.cpp-cuda-git /usr/include/ggml-virtgpu.h
llama.cpp-cuda-git /usr/include/ggml-vulkan.h
llama.cpp-cuda-git /usr/include/ggml-webgpu.h
llama.cpp-cuda-git /usr/include/ggml-zendnn.h
llama.cpp-cuda-git /usr/include/ggml.h
llama.cpp-cuda-git /usr/lib/cmake/ggml/
llama.cpp-cuda-git /usr/lib/cmake/ggml/ggml-config.cmake
llama.cpp-cuda-git /usr/lib/cmake/ggml/ggml-version.cmake
llama.cpp-cuda-git /usr/lib/libggml-base.so
llama.cpp-cuda-git /usr/lib/libggml-base.so.0
llama.cpp-cuda-git /usr/lib/libggml-base.so.0.12.0
llama.cpp-cuda-git /usr/lib/libggml-cpu.so
llama.cpp-cuda-git /usr/lib/libggml-cpu.so.0
llama.cpp-cuda-git /usr/lib/libggml-cpu.so.0.12.0
llama.cpp-cuda-git /usr/lib/libggml-cuda.so
llama.cpp-cuda-git /usr/lib/libggml-cuda.so.0
llama.cpp-cuda-git /usr/lib/libggml-cuda.so.0.12.0
llama.cpp-cuda-git /usr/lib/libggml-rpc.so
llama.cpp-cuda-git /usr/lib/libggml-rpc.so.0
llama.cpp-cuda-git /usr/lib/libggml-rpc.so.0.12.0
llama.cpp-cuda-git /usr/lib/libggml.so
llama.cpp-cuda-git /usr/lib/libggml.so.0
llama.cpp-cuda-git /usr/lib/libggml.so.0.12.0

jamesb6626 commented on 2026-05-08 12:37 (UTC) (edited on 2026-05-08 12:41 (UTC) by jamesb6626)

On the first try with makepkg -si, it misses the build number & commit id. (current commit a7b116537675, but isn't a new issue)

The result is that e.g. llama-server --version confidently reports it is version: 0 (), with similar issues anywhere else a version is expected.

My current workaround is to interrupt it partway through a build, then rerun, somehow this gets the correct value.

Sticking in an echo to check the variables _commit_id/_build_number, they are set correctly in prepare(), but have been forgotten and reset to blank/0 when build() runs, in the PKGBUILD.

Anyone else getting this weird behaviour, or just me?

Bink commented on 2026-05-07 11:37 (UTC)

I'd been monitoring local performance too, and had wondered if that introduced a regression. Looks like I might have flown too close to the sun on that one. Thank you for providing a clear analysis @Saluu, I appreciate it.

I've now removed OpenBLAS.