Arch Linux User Repository

NOTE: This package is a fork of koboldcpp by YellowRoseCx, which has slower releases. Please check first before flagging this package as out-of-date. https://github.com/YellowRoseCx/koboldcpp-rocm/releases

DISCLAIMER: I only maintain this package. Since I don't have an AMD GPU, I can only test the build process in a clean chroot. I cannot test if the result build would actually run. If this is not working for you, you can install koboldcpp and use Vulkan instead. It's faster to build and smaller to install.

Build configuration could be changed with environment variables. Or set via KEY=VALUE in ${XDG_CONFIG_HOME}/koboldcpp_build.conf file (default is ~/.config/koboldcpp_build.conf):

KOBOLDCPP_NO_PORTABLE - Disabling portable build (including failsafe and noavx2):

• '0' - Keep portable build enabled (default)
• '1' or any other value that isn't '0' - Disable portable build.

As of 1.69.yr0-3, I have added a desktop entry for GUI launcher, and you'll need to install customtkinter for it.

I try to run this through Distrobox and I do see the hipblas (ROCm) option, but it crashes with a Cuda error:

ggml_cuda_compute_forward: ADD failed
CUDA error: shared object initialization failed
  current device: 0, in function ggml_cuda_compute_forward at ggml/src/ggml-cuda.cu:2365
  err
ggml/src/ggml-cuda.cu:107: CUDA error

Vulkan seems to work though, not sure if that's effectively the same? I really wish ROCm would not be such a mess to get running. It's the one thing I really hate when it comes to AMD gpus on Linux.

@Smae Also check if the beginning of the console has these lines:

Initializing dynamic library: koboldcpp_hipblas.so

It should be koboldcpp_hipblas.so.

@Smae Yes, counterintuitively, you have to use --usecublas flag to enable hipblas with this fork. However, looking through the source code here, it will only use hipblas if and only if:

• /usr/share/koboldcpp/koboldcpp_cublas.so does not exist.
• And /usr/share/koboldcpp/koboldcpp_hipblas.so exists.

Could check if you some how has the straggler koboldcpp_cublas.so file? And what is the output of ldd /usr/share/koboldcpp/koboldcpp_hipblas.so?

@TheBill2001 Pretty much the moment after launch, in all fairness I'm after the hipblas as I do have an AMD GPU, but it's not showing up. From a screenshot on YellowRose's github (which admittedly is just copypasta) it seems to imply to select 'cuBlas' but I'm assuming this is incorrect and where my confusion is stemming from.

Thus my real problem is more, 'why isn't the hipblas backend showing up'.

If i just launch koboldcpp from the command line, it comes up with the GUI, so I alter my preferences accordingly and set it to cuBlas, then click launch. Within around 1-5 milliseconds it exits out as it can't find the nvidia stuff. But lemme wind back a second.

As per previous the back ends I have available (despite having installed koboldcpp-hipblas) are 'OpenBlas, CLBlas, cuBlas, Vulkan, NoBlas....' no hipblas present.

Additional info AMD RX 590 series.

@Smae That should not be the case. libcuda.so.1 belongs to CUDA in the nvidia-utils package and this fork does not use really use anything from NVIDIA. I have just tested building the package in a clean chroot and none of the libraries in koboldcpp is linked against it. However, I cannot test further as I don't have AMD GPU. Where and when in the execution of koboldcpp did you encounter this error message?

I can't seem to get this to work? I've installed it but I get

'libcuda.so.1 cannot open shared object file: no such file or directory' . What else do I need to install?

cblas is 3.12.0-5 rocblas is 6.0.2-1 clbast is 1.6.3 hipblas 6.0.2-1 (koboldcpp-hipblas 1.71.1.yr0-1) openblas 0.3.27-1

The options I have are: OpenBLAS, CLBlas, cuBlas, Vulkan, NoBlas, CLBlas NoAVX2 (old cpu) NoAVX2 mode (old cup) Failsafe mode

There is no hipblas option surprisingly, if I hover over, it does say that it's not available, but as far as I'm aware I've installed all the dependencies.

NOTE: This package is a fork of koboldcpp by YellowRoseCx, which has slower releases. Please check first before flagging this package as out-of-date. https://github.com/YellowRoseCx/koboldcpp-rocm/releases

DISCLAIMER: I only maintain this package. Since I don't have an AMD GPU, I can only test the build process in a clean chroot. I cannot test if the result build would actually run. If this is not working for you, you can install koboldcpp and use Vulkan instead. It's faster to build and smaller to install.

Build configuration could be changed with environment variables. Or set via KEY=VALUE in ${XDG_CONFIG_HOME}/koboldcpp_build.conf file (default is ~/.config/koboldcpp_build.conf):

KOBOLDCPP_NO_PORTABLE - Disabling portable build (including failsafe and noavx2):

• '0' - Keep portable build enabled (default)
• '1' or any other value that isn't '0' - Disable portable build.

As of 1.69.yr0-3, I have added a desktop entry for GUI launcher, and you'll need to install customtkinter for it.

I believe the current checksum is wrong. I get:

ad1490d792ad26d5d9f065166f726b91b058b41509282ee362ed84065b545668  koboldcpp-rocm-1.67.yr0-ROCm.tar.gz

However, PKGBUILD states:

sha256sums = 7cfcd2323d78a97461fd263595eea1ee5c34e1b1496a9181ba68ea2e20cb7961

koboldcpp-hipblas needs to be removed from the provides array, since this package is already named this.

Please add =${pkgver} to the koboldcpp and koboldcpp-rocm provides-entries.