| Git Clone URL: | https://aur.archlinux.org/llama.cpp-cuda.git (read-only, click to copy) |
|---|---|
| Package Base: | llama.cpp-cuda |
| Description: | Port of Facebook's LLaMA model in C/C++ (with NVIDIA CUDA optimizations) |
| Upstream URL: | https://github.com/ggerganov/llama.cpp |
| Licenses: | MIT |
| Conflicts: | libggml, llama.cpp |
| Provides: | llama.cpp |
| Submitter: | txtsd |
| Maintainer: | txtsd |
| Last Packager: | txtsd |
| Votes: | 5 |
| Popularity: | 2.45 |
| First Submitted: | 2024-10-26 20:17 (UTC) |
| Last Updated: | 2024-12-22 11:12 (UTC) |
diff --git a/llama.cpp.service b/llama.cpp.service
index 4678d85..be89f9b 100644
--- a/llama.cpp.service
+++ b/llama.cpp.service
@@ -7,7 +7,7 @@ Type=simple
EnvironmentFile=/etc/conf.d/llama.cpp
ExecStart=/usr/bin/llama-server $LLAMA_ARGS
ExecReload=/bin/kill -s HUP $MAINPID
-Restart=never
+Restart=no
[Install]
WantedBy=multi-user.target
Also your sysetmd service file is wrong. Did you even test your package?
diff --git a/PKGBUILD b/PKGBUILD
index ad448a7..3fdc20f 100644
--- a/PKGBUILD
+++ b/PKGBUILD
@@ -50,7 +50,8 @@ build() {
local _cmake_options=(
-B build
-S "${_pkgname}"
- -DCMAKE_BUILD_TYPE=None
+ -DCMAKE_BUILD_TYPE=MinSizeRel
+ -DCMAKE_CUDA_ARCHITECTURES=native
-DCMAKE_INSTALL_PREFIX='/usr'
-DGGML_NATIVE=OFF
-DGGML_AVX2=OFF
@@ -59,8 +60,8 @@ build() {
-DGGML_FMA=OFF
-DGGML_ALL_WARNINGS=OFF
-DGGML_ALL_WARNINGS_3RD_PARTY=OFF
- -DBUILD_SHARED_LIBS=OFF
- -DGGML_STATIC=ON
+ -DBUILD_SHARED_LIBS=ON
+ -DGGML_STATIC=OFF
-DGGML_LTO=ON
-DGGML_RPC=ON
-DLLAMA_CURL=ON
@@ -75,7 +76,6 @@ build() {
package() {
DESTDIR="${pkgdir}" cmake --install build
rm "${pkgdir}/usr/include/"ggml*
- rm "${pkgdir}/usr/lib/"lib*.a
install -Dm644 "${_pkgname}/LICENSE" "${pkgdir}/usr/share/licenses/${pkgname}/LICENSE"
this patch reduces the package size from 37G to 82M
Maybe don't enable static linking? IDK
❯ ls -lh /usr/bin/llama*
-rwxr-xr-x 1 root root 815M Nov 23 00:02 /usr/bin/llama-batched
-rwxr-xr-x 1 root root 815M Nov 23 00:02 /usr/bin/llama-batched-bench
-rwxr-xr-x 1 root root 815M Nov 23 00:02 /usr/bin/llama-bench
-rwxr-xr-x 1 root root 816M Nov 23 00:02 /usr/bin/llama-cli
-rwxr-xr-x 1 root root 814M Nov 23 00:02 /usr/bin/llama-convert-llama2c-to-ggml
-rwxr-xr-x 1 root root 815M Nov 23 00:02 /usr/bin/llama-cvector-generator
-rwxr-xr-x 1 root root 815M Nov 23 00:02 /usr/bin/llama-embedding
-rwxr-xr-x 1 root root 815M Nov 23 00:02 /usr/bin/llama-eval-callback
-rwxr-xr-x 1 root root 814M Nov 23 00:02 /usr/bin/llama-export-lora
-rwxr-xr-x 1 root root 814M Nov 23 00:02 /usr/bin/llama-gbnf-validator
-rwxr-xr-x 1 root root 111K Nov 23 00:02 /usr/bin/llama-gguf
-rwxr-xr-x 1 root root 131K Nov 23 00:02 /usr/bin/llama-gguf-hash
-rwxr-xr-x 1 root root 814M Nov 23 00:02 /usr/bin/llama-gguf-split
-rwxr-xr-x 1 root root 815M Nov 23 00:02 /usr/bin/llama-gritlm
-rwxr-xr-x 1 root root 815M Nov 23 00:02 /usr/bin/llama-imatrix
-rwxr-xr-x 1 root root 816M Nov 23 00:02 /usr/bin/llama-infill
-rwxr-xr-x 1 root root 816M Nov 23 00:02 /usr/bin/llama-llava-cli
-rwxr-xr-x 1 root root 816M Nov 23 00:02 /usr/bin/llama-lookahead
-rwxr-xr-x 1 root root 816M Nov 23 00:02 /usr/bin/llama-lookup
-rwxr-xr-x 1 root root 815M Nov 23 00:02 /usr/bin/llama-lookup-create
-rwxr-xr-x 1 root root 31K Nov 23 00:02 /usr/bin/llama-lookup-merge
-rwxr-xr-x 1 root root 815M Nov 23 00:02 /usr/bin/llama-lookup-stats
-rwxr-xr-x 1 root root 816M Nov 23 00:02 /usr/bin/llama-minicpmv-cli
-rwxr-xr-x 1 root root 816M Nov 23 00:02 /usr/bin/llama-parallel
-rwxr-xr-x 1 root root 815M Nov 23 00:02 /usr/bin/llama-passkey
-rwxr-xr-x 1 root root 816M Nov 23 00:02 /usr/bin/llama-perplexity
-rwxr-xr-x 1 root root 814M Nov 23 00:02 /usr/bin/llama-quantize
-rwxr-xr-x 1 root root 815M Nov 23 00:02 /usr/bin/llama-quantize-stats
-rwxr-xr-x 1 root root 815M Nov 23 00:02 /usr/bin/llama-retrieval
-rwxr-xr-x 1 root root 815M Nov 23 00:02 /usr/bin/llama-save-load-state
-rwxr-xr-x 1 root root 817M Nov 23 00:02 /usr/bin/llama-server
-rwxr-xr-x 1 root root 815M Nov 23 00:02 /usr/bin/llama-simple
-rwxr-xr-x 1 root root 815M Nov 23 00:02 /usr/bin/llama-simple-chat
-rwxr-xr-x 1 root root 816M Nov 23 00:02 /usr/bin/llama-speculative
-rwxr-xr-x 1 root root 815M Nov 23 00:02 /usr/bin/llama-tokenize
Wait can you show me individual file sizes? That's not right.
@brauliobo Such is life with CUDA :( I can't even build this on my CI because of the size requirements
the build is taking 45gb! just the build/bin folder takes 38gb:
braulio @ whitebeast ➜ bin git:(master) pwd
/home/braulio/.cache/yay/llama.cpp-cuda/src/build/bin
braulio @ whitebeast ➜ bin git:(master) du -h --max-depth=1
38G .
braulio @ whitebeast ➜ bin git:(master) ls
llama-batched llama-gbnf-validator llama-lookup llama-quantize llama-vdot test-grammar-integration test-sampling
llama-batched-bench llama-gguf llama-lookup-create llama-quantize-stats rpc-server test-grammar-parser test-tokenizer-0
llama-bench llama-gguf-hash llama-lookup-merge llama-retrieval test-arg-parser test-json-schema-to-grammar test-tokenizer-1-bpe
llama-cli llama-gguf-split llama-lookup-stats llama-save-load-state test-autorelease test-llama-grammar test-tokenizer-1-spm
llama-convert-llama2c-to-ggml llama-gritlm llama-minicpmv-cli llama-server test-backend-ops test-log
llama-cvector-generator llama-imatrix llama-parallel llama-simple test-barrier test-model-load-cancel
llama-embedding llama-infill llama-passkey llama-simple-chat test-c test-quantize-fns
llama-eval-callback llama-llava-cli llama-perplexity llama-speculative test-chat-template test-quantize-perf
llama-export-lora llama-lookahead llama-q8dot llama-tokenize test-grad0 test-rope
Pinned Comments
txtsd commented on 2024-10-26 20:17 (UTC) (edited on 2024-12-06 14:15 (UTC) by txtsd)
Alternate versions
llama.cpp
llama.cpp-vulkan
llama.cpp-sycl-fp16
llama.cpp-sycl-fp32
llama.cpp-cuda
llama.cpp-cuda-f16
llama.cpp-hip