Arch Linux User Repository

Alternate versions

llama.cpp
llama.cpp-vulkan
llama.cpp-sycl-fp16
llama.cpp-sycl-fp32
llama.cpp-cuda
llama.cpp-cuda-f16
llama.cpp-hip

The current PKGBUILD no longer compiles a native build due to makepkg providing the SOURCE_DATE_EPOCH environment variable. This variable is checked in ggml/CMakeLists.txt:

if (CMAKE_CROSSCOMPILING OR DEFINED ENV{SOURCE_DATE_EPOCH})
    message(STATUS "Setting GGML_NATIVE_DEFAULT to OFF")
    set(GGML_NATIVE_DEFAULT OFF)
else()
    set(GGML_NATIVE_DEFAULT ON)
endif()

@AlxQ thanks, fixed

==> Validating source files with sha256sums...
    llama.cpp-cuda-b6045.tar.gz ... FAILED
    llama.cpp.conf ... Passed
    llama.cpp.service ... Passed
==> ERROR: One or more files did not pass the validity check!
 -> error making: llama.cpp-cuda-exit status 1
 -> Failed to install the following packages. Manual intervention is required:
llama.cpp-cuda - exit status 1

I've had cuda installed long before the last reboot, idk what's the issue then.

edit: it seems to be because I used the fish shell. Building from bash works.

edit: unrelated but I found llama.cpp-vulkan to be 2-4 x faster.

just a couple of minor changes in the build

python modules are now identified as optional for the conversion script
changed from git to tar.gz due to the git repo initial sync being so large

just a couple of minor changes in the build

python modules are now identified as optional for the conversion script
changed from git to tar.gz due to the git repo initial sync being so large

@trougnouf you need to source /etc/profile or log out and back in after installing cuda

I can get through that error with -DCMAKE_CUDA_COMPILER=/opt/cuda/bin/nvcc but then I get several more during the compilation.

I get the following error:

-- Could not find nvcc, please set CUDAToolkit_ROOT.
CMake Error at ggml/src/ggml-cuda/CMakeLists.txt:183 (message):
  CUDA Toolkit not found

Finally I have a solution! Courtesy of Gemini Pro 2.5 and about 10 back and forths with it. Apparently the issue is that I have GCC 15 installed, and nvcc requires GCC 13. So this is what I had to do:

1. Had to install gcc13 from AUR (this took like 45 minutes)
2. Had to change the PKGBUILD options section:

options=(!lto !debug)

1. Had to add to the PKGBUILD build() function:

-DCMAKE_C_COMPILER=/usr/bin/gcc-13
-DCMAKE_CXX_COMPILER=/usr/bin/g++-13
-DCMAKE_CUDA_HOST_COMPILER=/usr/bin/g++-13

1. makepkg -si
2. Had to approve the removal of libggml-cuda-f16-git

So many insane hoops to go through to get this working. I don't really understand why I'm the only one having this issue? Is it maybe because I'm using CachyOS?

Either way, if anyone runs into the craziness I've gone through, I hope the above instructions help get you back up and running (at least temporarily)! I've tested it for about 30 minutes and everything seems pretty good.

@txtsd is there anything you can take from the above to maybe get this working with the existing PKGBUILD? Maybe somehow forcing nvcc + gcc to play together nicer? Much appreciated!

I just gave it another shot. Getting a new error now:

-- Adding CPU backend variant ggml-cpu: -march=native
-- Found CUDAToolkit: /opt/cuda/targets/x86_64-linux/include (found version "12.9.86")
-- CUDA Toolkit found
-- Using CUDA architectures: native
CMake Error at /usr/share/cmake/Modules/CMakeDetermineCompilerId.cmake:909 (message):
  Compiling the CUDA compiler identification source file
  "CMakeCUDACompilerId.cu" failed.

  Compiler: /opt/cuda/bin/nvcc

  Build flags:

  Id flags: --keep;--keep-dir;tmp -v



  The output was:

  2

  nvcc warning : Support for offline compilation for architectures prior to
  '<compute/sm/lto>_75' will be removed in a future release (Use
  -Wno-deprecated-gpu-targets to suppress warning).

  #$ _NVVM_BRANCH_=nvvm

  #$ _SPACE_=

  #$ _CUDART_=cudart

  #$ _HERE_=/opt/cuda/bin

  #$ _THERE_=/opt/cuda/bin

  #$ _TARGET_SIZE_=

  #$ _TARGET_DIR_=

  #$ _TARGET_DIR_=targets/x86_64-linux

  #$ TOP=/opt/cuda/bin/..

  #$ CICC_PATH=/opt/cuda/bin/../nvvm/bin

  #$ NVVMIR_LIBRARY_DIR=/opt/cuda/bin/../nvvm/libdevice

  #$ LD_LIBRARY_PATH=/opt/cuda/bin/../lib:/opt/cuda/lib64

  #$
  PATH=/opt/cuda/bin/../nvvm/bin:/opt/cuda/bin:/opt/cuda/bin:/home/james/miniforge3/condabin:/home/james/.local/bin:/opt/miniconda3/bin:/home/james/Scripts:/usr/local/bin:/usr/bin:/home/james/.lmstudio/bin

  #$ INCLUDES="-I/opt/cuda/bin/../targets/x86_64-linux/include"

  #$ LIBRARIES= "-L/opt/cuda/bin/../targets/x86_64-linux/lib/stubs"
  "-L/opt/cuda/bin/../targets/x86_64-linux/lib"

  #$ CUDAFE_FLAGS=

  #$ PTXAS_FLAGS=

  #$ rm tmp/a_dlink.reg.c

  #$ gcc -D__CUDA_ARCH_LIST__=520 -D__NV_LEGACY_LAUNCH -E -x c++ -D__CUDACC__
  -D__NVCC__ "-I/opt/cuda/bin/../targets/x86_64-linux/include"
  -D__CUDACC_VER_MAJOR__=12 -D__CUDACC_VER_MINOR__=9
  -D__CUDACC_VER_BUILD__=86 -D__CUDA_API_VER_MAJOR__=12
  -D__CUDA_API_VER_MINOR__=9 -D__NVCC_DIAG_PRAGMA_SUPPORT__=1
  -D__CUDACC_DEVICE_ATOMIC_BUILTINS__=1 -include "cuda_runtime.h" -m64
  "CMakeCUDACompilerId.cu" -o "tmp/CMakeCUDACompilerId.cpp4.ii"

  #$ cudafe++ --c++17 --gnu_version=150101 --display_error_number
  --orig_src_file_name "CMakeCUDACompilerId.cu" --orig_src_path_name
  "/mnt/big/.cache/paru/clone/llama.cpp-cuda/src/build/CMakeFiles/4.0.3-dirty/CompilerIdCUDA/CMakeCUDACompilerId.cu"
  --allow_managed --m64 --parse_templates --gen_c_file_name
  "tmp/CMakeCUDACompilerId.cudafe1.cpp" --stub_file_name
  "CMakeCUDACompilerId.cudafe1.stub.c" --gen_module_id_file
  --module_id_file_name "tmp/CMakeCUDACompilerId.module_id"
  "tmp/CMakeCUDACompilerId.cpp4.ii"

  /usr/include/c++/15.1.1/type_traits(554): error: type name is not allowed

        : public __bool_constant<__is_pointer(_Tp)>
                                              ^
...
...
...

  /usr/include/c++/15.1.1/bits/stl_algobase.h(1239): error: type name is not
  allowed

        || __is_pointer(_ValueType1)
                        ^



  /usr/include/c++/15.1.1/bits/stl_algobase.h(1412): error: type name is not
  allowed

      && __is_pointer(_II1) && __is_pointer(_II2)
                      ^



  /usr/include/c++/15.1.1/bits/stl_algobase.h(1412): error: type name is not
  allowed

      && __is_pointer(_II1) && __is_pointer(_II2)
                                            ^



  25 errors detected in the compilation of "CMakeCUDACompilerId.cu".

  # --error 0x2 --





Call Stack (most recent call first):
  /usr/share/cmake/Modules/CMakeDetermineCompilerId.cmake:8 (CMAKE_DETERMINE_COMPILER_ID_BUILD)
  /usr/share/cmake/Modules/CMakeDetermineCompilerId.cmake:53 (__determine_compiler_id_test)
  /usr/share/cmake/Modules/CMakeDetermineCUDACompiler.cmake:139 (CMAKE_DETERMINE_COMPILER_ID)
  ggml/src/ggml-cuda/CMakeLists.txt:43 (enable_language)


-- Configuring incomplete, errors occurred!
==> ERROR: A failure occurred in build().
    Aborting...
error: failed to build 'llama.cpp-cuda-b5849-1':
error: packages failed to build: llama.cpp-cuda-b5849-1

This is for the llama.cpp-cuda, but same exact error for f16 as well.