Package Details: ollama-rocm-git 0.11.6.r3.g4ae4f47b16fd-1

Git Clone URL: https://aur.archlinux.org/ollama-rocm-git.git (read-only, click to copy)
Package Base: ollama-rocm-git
Description: Create, run and share large language models (LLMs) with ROCm
Upstream URL: https://github.com/ollama/ollama
Licenses: MIT
Conflicts: ollama
Provides: ollama
Submitter: sr.team
Maintainer: wgottwalt
Last Packager: wgottwalt
Votes: 5
Popularity: 0.003217
First Submitted: 2024-02-28 00:40 (UTC)
Last Updated: 2025-08-21 09:08 (UTC)

Required by (64)

Sources (4)

Pinned Comments

wgottwalt commented on 2024-11-09 10:46 (UTC) (edited on 2024-11-26 15:23 (UTC) by wgottwalt)

Looks like the ROCm 6.2.2-1 SDK has a malfunctioning compiler. It produces a broken ollama binary (fp16 issues). You may need to stay with ROCm 6.0.2 for now. I don't know if this got fixed in a newer build release. But the initial SDK version "-1" is broken.

ROCm 6.2.4 fixes this issue completely.

Latest Comments

1 2 3 4 Next › Last »

EndlessEden commented on 2025-10-22 02:19 (UTC) (edited on 2025-10-22 06:20 (UTC) by EndlessEden)

I havent kept up with this package, i switched from NVIDIA to AMD (GFX1201) earlier this year.

@wgottwalt But i can confirm the configure script is not detecting HIP at all now, ROCM is properly installed as is hipblas, both are 6.4.4 (and from arch-packaging afaik).

I will debug this, but can you also test it?

UPDATE: Ok, confirmed, while the HIP libraries are in my user path(due to edits i did to rocm.sh), the current profile.d/rocm.sh is a mess. it relies a update_path function. -- i didnt reboot after editing it however.

After adding

 "-DOLLAMA_LLVM_DIR=/opt/rocm/llvm -DHIP_COMPILER=hipcc -DHIP_PATH=/opt/rocm \"

and 

export PATH=/opt/rocm/bin:$PATH
export HIP_PATH=/opt/rocm
export CMAKE_PREFIX_PATH=/opt/rocm

to the PKGBUILD, HIP was detected, and it started building. I also installed vulkan-headers just to be safe. Vulkan backend was added a while ago, and while im not sure "ROCM" builds would find it necessary, i think its a good option as a fallback (As you leave CPU interference enabled, it doesnt make much sense to not add another generic backend)

Ill report back after testing to confirm its all working.

UPDATE1: While it successfully built with no issues, at runtime it ignores ROCM (atleast for my RX 9070XT | GFX1201), it does work in windows (with patched rocm libraries), but refuses here. -- ive tried everything and while it will work with vulkan (NOT inside of systemd for some reason... despite every effort), it simply refuses to use ROCM.

UPDATE2: After double-checking the ROCM support matrix, RDNA4/CDNA4 is NOT supported by ROCM-6.4.4 (which Archlinux is packaging), current ROCM is 7.0.2... its significantly out of date. -- trying with opencl-amd*

UPDATE3: for some reason this package tries to force hip-runtime-amd(due to requesting newer than a ancient version?) that doesnt align with opencl-amd, so i explicitly specified opencl-amd; Maintainer, you should look into this as well | Discovered Ollama doesnt detect ROCM 7.x due to a change in layout... it doesnt even try with symlinks and file copies. (keyboard headbashing continues)

chb commented on 2025-03-13 07:06 (UTC)

@wgottwalt Thank you for your insight, I have explored this further if you are interested in taking a look as it sounds like you are far more experienced than myself, though I am aware of the explicit deprecation of the arch. It seems some have had success building from SDK or even in this docker (below) though I do not believe any are on so recent a build as 6.3+ (I think current SDK build is 6.1.2.0). I wonder if any patches from the SDK builder should be rolled in upstream as they seem to address many aspects of compatibility for older GCN. My use case is whisperx + llm+cot+rag . I can accept that I may just need to update my card. I was able to utilise vulkan for llama via kobold (I think https://github.com/LostRuins/koboldcpp - I think someone mentions that in one of the sdk issues) and performance was middling, though for the price of the cards as people upgrade, I feel some reasonable performance could be achieved across several and could be ideal for eg raspi+pcie switch (not sure if that's a thing) or even just single card. arm probably runs it more efficiently lol. i think the 580's performance was about equal to 7700k but with the advantage of not completely bricking your system, maybe i need to try a different scheduler or limit cores but i feel that somewhat defeats the purpose. my lspci -vvv is in one of these:

https://github.com/lamikr/rocm_sdk_builder https://github.com/lamikr/rocm_sdk_builder/issues/220 https://github.com/lamikr/rocm_sdk_builder/issues/173 https://github.com/robertrosenbusch/gfx803_rocm/issues/6#issuecomment-2719117249

edtoml commented on 2025-03-05 13:33 (UTC) (edited on 2025-03-05 13:48 (UTC) by edtoml)

the PKGBUILD's makedepends should be updated to include hipblas-common. It is not getting installed when rocm is upgraded and is needed for the build.

With the latest hipblas

yay -Qi hipblas
Name            : hipblas
Version         : 6.3.2-2

the package is not finding a cmake file

-- Detecting HIP compile features - done
CMake Error at /usr/share/cmake/Modules/CMakeFindDependencyMacro.cmake:76 (find_package):
  By not providing "Findhipblas-common.cmake" in CMAKE_MODULE_PATH this
  project has asked CMake to find a package configuration file provided by
  "hipblas-common", but CMake did not find one.

And fails to build

wgottwalt commented on 2025-01-13 11:05 (UTC)

@chb I'm not into building ROCm myself, though, I don't know much about it. But the ROCm docs always stated, that the GCN 5.0 arch is the minimum requirement. The RX 580 is Polaris and this is GCN 4. The crash looks like a floating point exception to me and it is very likely, that GCN 4 has an incomplete floating point model. It is sufficient for rasterization, but may not be enough for GPU-compute workloads. Though, that are just my assumptions based on my experience and may mean nothing.

chb commented on 2025-01-13 10:41 (UTC) (edited on 2025-01-13 10:42 (UTC) by chb)

@wgottwalt it seems like tensile and rocblas are the main issues, I'm trying to rebuild rocblas with which rm -f "$srcdir/$dirname/library/src/blas3/Tensile/Logic/asm_full/r9nano*.yaml" https://github.com/xuhuisheng/rocm-build/blob/master/gfx803/README.md

xuhuisheng commented Oct 23, 2020 • What is the expected behavior

Dont crash and return correct loss on gfx803

What actually happens

Invalid argument: indices[5,284] = 997212422 is not in [0, 5001) (text classification)
Low accuracy with loss NaN (mnist)

How to reproduce

ROCm-3.7+ on gfx803, run tensorflow text classification sample. Tensorflow offical sample could reproduce this issue, almost 90%. https://www.tensorflow.org/tutorials/keras/text_classification
There are many people get this error, please refer here :

ROCm-3.7+ broken on gfx803 ROCm#1265 Workaround 1: I rebuild rocBLAS with BUILD_WITH_TENSILE_HOST=false, and the problem dispeared, Maybe the gfx803 r9nano_*.yml is out-of-date? This way caused compiling failure on ROCm-3.9. Workaround 2: keep BUILD_WITH_TENSILE_HOST=true, delete library/src/blas3/Tensile/Logic/asm_full/r9nano_Cijk_Ailk_Bljk_SB.yaml, and issue resolved. If I just keep one solution of this file, issue reproduced.

https://github.com/ROCm/rocBLAS/issues/1172

xuhuisheng has a docker with working ROCm

OS linux Python ROCm GPU

Ubuntu-20.04.5 5.15 3.8.10 5.4.1 RX580 https://github.com/xuhuisheng/rocm-gfx803

But I think my issue was then ctranslate2 for whisperx

wgottwalt commented on 2025-01-13 09:33 (UTC)

@chb I see, though I can imagine the performance won't be that good. The support for the interesting types like 8 bit ints and 16 bit floats is quite limited on that old hardware. Combined with the small local memory (only max 8GiB) you may be better off with a modern CPU. Hmm, could you build ROCm for aarch64, too? I test my ollama cpu-only packages against my Ampera Altra Max systems, which can easily deal with 405B models. Would be nice if I could spread the load over GFX cards, too.

chb commented on 2025-01-13 04:01 (UTC)

I'm currently trying to get rocm to compile for gfx803 (RX580), options exist for gfx900 and other archs https://github.com/lamikr/rocm_sdk_builder/issues/173

This project may be of assistance to people with 'unsupported' cards. If I'm able to complete this I will discuss with the author if this can be hosted.

wgottwalt commented on 2024-12-21 17:18 (UTC)

No, I will not change that. The ROCm documentation is very clear about the gfx900 target: "Unsupported - The current ROCm release does not support this hardware. The HIP runtime might continue to run applications for an unsupported GPU, but prebuilt ROCm libraries are not officially supported and will cause runtime errors."

In short: The target is deprecated for a while now and is in the process of getting removed.

pbordron commented on 2024-12-19 17:57 (UTC)

Crash on my Vega 56 when querying a model with an invalid device function current device: 0, in function ggml_cuda_compute_forward ....

Need to enable gfx900 target and remove the sed on Makefile.rocm in PKGBUILD in order to solve the problem

1 2 3 4 Next › Last »