Package Details: ollama-rocm-git 0.6.1.git+eb2b22b0-1

Git Clone URL: https://aur.archlinux.org/ollama-rocm-git.git (read-only, click to copy)
Package Base: ollama-rocm-git
Description: Create, run and share large language models (LLMs) with ROCm
Upstream URL: https://github.com/ollama/ollama
Licenses: MIT
Conflicts: ollama
Provides: ollama
Submitter: sr.team
Maintainer: wgottwalt
Last Packager: wgottwalt
Votes: 5
Popularity: 0.25
First Submitted: 2024-02-28 00:40 (UTC)
Last Updated: 2025-03-14 15:00 (UTC)

Required by (40)

Sources (4)

Pinned Comments

wgottwalt commented on 2024-11-09 10:46 (UTC) (edited on 2024-11-26 15:23 (UTC) by wgottwalt)

Looks like the ROCm 6.2.2-1 SDK has a malfunctioning compiler. It produces a broken ollama binary (fp16 issues). You may need to stay with ROCm 6.0.2 for now. I don't know if this got fixed in a newer build release. But the initial SDK version "-1" is broken.

ROCm 6.2.4 fixes this issue completely.

Latest Comments

1 2 3 4 Next › Last »

chb commented on 2025-03-13 07:06 (UTC)

@wgottwalt Thank you for your insight, I have explored this further if you are interested in taking a look as it sounds like you are far more experienced than myself, though I am aware of the explicit deprecation of the arch. It seems some have had success building from SDK or even in this docker (below) though I do not believe any are on so recent a build as 6.3+ (I think current SDK build is 6.1.2.0). I wonder if any patches from the SDK builder should be rolled in upstream as they seem to address many aspects of compatibility for older GCN. My use case is whisperx + llm+cot+rag . I can accept that I may just need to update my card. I was able to utilise vulkan for llama via kobold (I think https://github.com/LostRuins/koboldcpp - I think someone mentions that in one of the sdk issues) and performance was middling, though for the price of the cards as people upgrade, I feel some reasonable performance could be achieved across several and could be ideal for eg raspi+pcie switch (not sure if that's a thing) or even just single card. arm probably runs it more efficiently lol. i think the 580's performance was about equal to 7700k but with the advantage of not completely bricking your system, maybe i need to try a different scheduler or limit cores but i feel that somewhat defeats the purpose. my lspci -vvv is in one of these:

https://github.com/lamikr/rocm_sdk_builder https://github.com/lamikr/rocm_sdk_builder/issues/220 https://github.com/lamikr/rocm_sdk_builder/issues/173 https://github.com/robertrosenbusch/gfx803_rocm/issues/6#issuecomment-2719117249

edtoml commented on 2025-03-05 13:33 (UTC) (edited on 2025-03-05 13:48 (UTC) by edtoml)

the PKGBUILD's makedepends should be updated to include hipblas-common. It is not getting installed when rocm is upgraded and is needed for the build.

With the latest hipblas

yay -Qi hipblas
Name            : hipblas
Version         : 6.3.2-2

the package is not finding a cmake file

-- Detecting HIP compile features - done
CMake Error at /usr/share/cmake/Modules/CMakeFindDependencyMacro.cmake:76 (find_package):
  By not providing "Findhipblas-common.cmake" in CMAKE_MODULE_PATH this
  project has asked CMake to find a package configuration file provided by
  "hipblas-common", but CMake did not find one.

And fails to build

wgottwalt commented on 2025-01-13 11:05 (UTC)

@chb I'm not into building ROCm myself, though, I don't know much about it. But the ROCm docs always stated, that the GCN 5.0 arch is the minimum requirement. The RX 580 is Polaris and this is GCN 4. The crash looks like a floating point exception to me and it is very likely, that GCN 4 has an incomplete floating point model. It is sufficient for rasterization, but may not be enough for GPU-compute workloads. Though, that are just my assumptions based on my experience and may mean nothing.

chb commented on 2025-01-13 10:41 (UTC) (edited on 2025-01-13 10:42 (UTC) by chb)

@wgottwalt it seems like tensile and rocblas are the main issues, I'm trying to rebuild rocblas with which rm -f "$srcdir/$dirname/library/src/blas3/Tensile/Logic/asm_full/r9nano*.yaml" https://github.com/xuhuisheng/rocm-build/blob/master/gfx803/README.md

xuhuisheng commented Oct 23, 2020 • What is the expected behavior

Dont crash and return correct loss on gfx803

What actually happens

Invalid argument: indices[5,284] = 997212422 is not in [0, 5001) (text classification)
Low accuracy with loss NaN (mnist)

How to reproduce

ROCm-3.7+ on gfx803, run tensorflow text classification sample. Tensorflow offical sample could reproduce this issue, almost 90%. https://www.tensorflow.org/tutorials/keras/text_classification
There are many people get this error, please refer here :

ROCm-3.7+ broken on gfx803 ROCm#1265 Workaround 1: I rebuild rocBLAS with BUILD_WITH_TENSILE_HOST=false, and the problem dispeared, Maybe the gfx803 r9nano_*.yml is out-of-date? This way caused compiling failure on ROCm-3.9. Workaround 2: keep BUILD_WITH_TENSILE_HOST=true, delete library/src/blas3/Tensile/Logic/asm_full/r9nano_Cijk_Ailk_Bljk_SB.yaml, and issue resolved. If I just keep one solution of this file, issue reproduced.

https://github.com/ROCm/rocBLAS/issues/1172

xuhuisheng has a docker with working ROCm

OS linux Python ROCm GPU

Ubuntu-20.04.5 5.15 3.8.10 5.4.1 RX580 https://github.com/xuhuisheng/rocm-gfx803

But I think my issue was then ctranslate2 for whisperx

wgottwalt commented on 2025-01-13 09:33 (UTC)

@chb I see, though I can imagine the performance won't be that good. The support for the interesting types like 8 bit ints and 16 bit floats is quite limited on that old hardware. Combined with the small local memory (only max 8GiB) you may be better off with a modern CPU. Hmm, could you build ROCm for aarch64, too? I test my ollama cpu-only packages against my Ampera Altra Max systems, which can easily deal with 405B models. Would be nice if I could spread the load over GFX cards, too.

chb commented on 2025-01-13 04:01 (UTC)

I'm currently trying to get rocm to compile for gfx803 (RX580), options exist for gfx900 and other archs https://github.com/lamikr/rocm_sdk_builder/issues/173

This project may be of assistance to people with 'unsupported' cards. If I'm able to complete this I will discuss with the author if this can be hosted.

wgottwalt commented on 2024-12-21 17:18 (UTC)

No, I will not change that. The ROCm documentation is very clear about the gfx900 target: "Unsupported - The current ROCm release does not support this hardware. The HIP runtime might continue to run applications for an unsupported GPU, but prebuilt ROCm libraries are not officially supported and will cause runtime errors."

In short: The target is deprecated for a while now and is in the process of getting removed.

pbordron commented on 2024-12-19 17:57 (UTC)

Crash on my Vega 56 when querying a model with an invalid device function current device: 0, in function ggml_cuda_compute_forward ....

Need to enable gfx900 target and remove the sed on Makefile.rocm in PKGBUILD in order to solve the problem

rakatan commented on 2024-11-19 14:25 (UTC)

@wgottwalt

You are aware what the function pkgver() in the PKGBUILD does, right? I'm really waiting for the day someone uses the outdated flag...

not really, having seen no other place referencing a commit I assumed this would be the place - how else is the commit pinned if it's only referenced here? My assumption got sort of confirmed by it just working with ROCm (as I saw there is a commit adding compiler flags wrt fp16 in between these two).

what outdated flag do you mean?

In short, I'm rather new to Arch, so still learning the structure, and the community expectations.

And you also recognized the ollama.service file and that it includes the line Environment='LD_LIBRARY_PATH=/usr/lib/ollama' right?

likewise, I was not aware - is that the only expected way to use ollama? ollama serve is a command available on a binary on the PATH - should I be aware to not use it and only use it via systemd?

I'm aware that my answer is a but harsh, but damn it, why?!?

harsh and informative is good - at times. why what? ;)

1 2 3 4 Next › Last »