Arch Linux User Repository

Since 1.95.1, I have switched over to using upstream KobloldCpp rather than YellowRoseCx's fork.

Portable/NoAVX2 build has been separated to koboldcpp-hipblas-portable.

As of 1.69.yr0-3, I have added a desktop entry for GUI launcher, and you'll need to install customtkinter for it.

Long build time indeed... That was several hours on my 5700X3D. lol

Unfortunately it seems to be the same result with the portable version.

@TheBill2001 Yes, it's all built inside the distrobox. I'm on Bazzite, which is immutable, which usually makes it impossible to build stuff outside of a separate distrobox due to the lack of dependencies that can't be installed into the main OS.

Koboldcpp itself detects & lists my gpu properly, but I suppose the package itself might not? I'll give the portable a shot I guess.

@DarkThoughts I don't know much about distrobox aside from what Arch Wiki said. Were you building the package inside distrobox as well? It could be that the build process failed to detect your GPU's architecture. Since, this package will build for your specific GPU architecture and skip others (to reduce build time and package size). If the architecture cannot be determined, it will create a generically optimized binary which might not perform the best on your machine.

This is purely speculation, though, as I have no experience with distrobox, nor I know how your system is set up. Try building and running the package without any containerization and see if there is any difference.

Alternatively, the package koboldcpp-hipblas-portable is configured to build with multiple architectures which does include the 6650 XT. The downsides are long build time and larger package size since it also build for other GPU archs.

Lastly, you could also try running the binary built by upstream GitHub Action (https://koboldai.org/cpplinuxrocm) to see if it works. If that binary does not work either, you might want to file a bug report upstream.

So, I've tried this package again to see if there's any changes to the ROCm option. Now it does not crash anymore with a CUDA error while loading koboldcpp with ROCm, BUT it does not seem to progress at all during the first initialization process. It does say the first one may take "a few minutes", but I've waited many minutes at this point with the GPU (6650 XT) running at 100% - but it just sits at one dot in the terminal (I assume it's meant to be a progress bar?). I pretty much have to stop the distrobox itself to terminate the process too.

Anyone having more success than me?

@lightdot Fixed. Thanks.

Since 1.51.1, I have switched over to using upstream...

A typo, should be 1.95.1.

Since 1.95.1, I have switched over to using upstream KobloldCpp rather than YellowRoseCx's fork.

@astrella The reason is that I don't know. I adopted this package along with the -cuda and the minimal one. To be honest, even though I have an NVidia GPU, I mostly use the Vulkan backend since it is faster on my machine. And without an AMD GPU, I couldn't really test this package, either. I've been considering dropping this package and let someone else maintain it.

If there is demand, I'll switch over. Aside from checking if it is buildable in a clean chroot environment, I cannot guarantee that it'll work probably. Others might still want this fork, so another set of packages would have to be created.

Is there any reason to not switch to upstream LostRuins/koboldcpp since it now supports hipblas build?

Portable/NoAVX2 build has been separated to koboldcpp-hipblas-portable.