Arch Linux User Repository

Portable/NoAVX2 build has been separated to koboldcpp-hipblas-portable.

As of 1.69.yr0-3, I have added a desktop entry for GUI launcher, and you'll need to install customtkinter for it.

@astrella The reason is that I don't know. I adopted this package along with the -cuda and the minimal one. To be honest, even though I have an NVidia GPU, I mostly use the Vulkan backend since it is faster on my machine. And without an AMD GPU, I couldn't really test this package, either. I've been considering dropping this package and let someone else maintain it.

If there is demand, I'll switch over. Aside from checking if it is buildable in a clean chroot environment, I cannot guarantee that it'll work probably. Others might still want this fork, so another set of packages would have to be created.

Is there any reason to not switch to upstream LostRuins/koboldcpp since it now supports hipblas build?

Portable/NoAVX2 build has been separated to koboldcpp-hipblas-portable.

@random_click KoboldCpp does not use tk directly, but via customtkinter to display the GUI Launcher. It is optional as you can just use KoboldCpp directly from the terminal without ever touching the launcher.

If you install either python-customtkinter or python-customtkinter-git, tk should be automatically install with it. If it isn't, please report to those packages' maintainers.

Isn't this package missing dependecy to tk? I couldn't start the app without it

I try to run this through Distrobox and I do see the hipblas (ROCm) option, but it crashes with a Cuda error:

ggml_cuda_compute_forward: ADD failed
CUDA error: shared object initialization failed
  current device: 0, in function ggml_cuda_compute_forward at ggml/src/ggml-cuda.cu:2365
  err
ggml/src/ggml-cuda.cu:107: CUDA error

Vulkan seems to work though, not sure if that's effectively the same? I really wish ROCm would not be such a mess to get running. It's the one thing I really hate when it comes to AMD gpus on Linux.

@Smae Also check if the beginning of the console has these lines:

Initializing dynamic library: koboldcpp_hipblas.so

It should be koboldcpp_hipblas.so.

@Smae Yes, counterintuitively, you have to use --usecublas flag to enable hipblas with this fork. However, looking through the source code here, it will only use hipblas if and only if:

• /usr/share/koboldcpp/koboldcpp_cublas.so does not exist.
• And /usr/share/koboldcpp/koboldcpp_hipblas.so exists.

Could check if you some how has the straggler koboldcpp_cublas.so file? And what is the output of ldd /usr/share/koboldcpp/koboldcpp_hipblas.so?

@TheBill2001 Pretty much the moment after launch, in all fairness I'm after the hipblas as I do have an AMD GPU, but it's not showing up. From a screenshot on YellowRose's github (which admittedly is just copypasta) it seems to imply to select 'cuBlas' but I'm assuming this is incorrect and where my confusion is stemming from.

Thus my real problem is more, 'why isn't the hipblas backend showing up'.

If i just launch koboldcpp from the command line, it comes up with the GUI, so I alter my preferences accordingly and set it to cuBlas, then click launch. Within around 1-5 milliseconds it exits out as it can't find the nvidia stuff. But lemme wind back a second.

As per previous the back ends I have available (despite having installed koboldcpp-hipblas) are 'OpenBlas, CLBlas, cuBlas, Vulkan, NoBlas....' no hipblas present.

Additional info AMD RX 590 series.

@Smae That should not be the case. libcuda.so.1 belongs to CUDA in the nvidia-utils package and this fork does not use really use anything from NVIDIA. I have just tested building the package in a clean chroot and none of the libraries in koboldcpp is linked against it. However, I cannot test further as I don't have AMD GPU. Where and when in the execution of koboldcpp did you encounter this error message?