Arch Linux User Repository

Install polly and use this -fplugin=LLVMPolly.so instead of -Xclang -load -Xclang LLVMPolly.so. Also that PGO script doesn't work with Clang, only GCC.

hi,

it seems something isn't ok with makepkg-optimize-mold :( and pgo also doesn't work

error: unable to load plugin 'LLVMPolly.so': 'LLVMPolly.so: cannot open shared object file: No such file or directory'
clang (LLVM option parsing): Unknown command line argument '-polly'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--color'?
clang (LLVM option parsing): Unknown command line argument '-polly-num-threads=8'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--xcore-max-threads=8'?
clang (LLVM option parsing): Unknown command line argument '-polly-parallel'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--enable-pre'?
make[1]: *** [<builtin>: conn_bomb] Error 1
make[1]: *** Waiting for unfinished jobs....

clang -march=native -O2 -pipe -pthread -lgomp -mllvm -polly -mllvm -polly-num-threads=8 -mllvm -polly-parallel -Xclang -load -Xclang LLVMPolly.so -flto=auto -fprofile-generate -fprofile-dir=/home/tom/compile/bpftune-git/bpftune-git.gen -lpthread -Wl,-O1 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,-z,pack-relative-relocs -flto=auto -lgcov --coverage conn_bomb.c -o conn_bomb

@ptr1337 I think this should have proper upstream (instead of wiki) and replace makepkg-optimize which is not maintained.

Peter @ptr1337, consider using oxipng instead of optipng

Why?

1) It does optimize better.

You can see that optipng is one of the dependencies of Calibre. Indeed, it's used internally by it:

https://github.com/kovidgoyal/calibre/blob/eb161166b05f33eb0ff6bda97bac1fafadca77c5/src/calibre/utils/img.py#L621-L625

CLI utility of calibre ebook-polish is using it. My experience is that after running epub-polish on EPUB book with PNG images, the latter can be further optimized with oxipng:

oxipng --opt max extracted_epub/**.png

So it means that optipng doesn't optimally encodes PNGs. Worth noting that oxipng with --opt max doesn't use Zopfli, there is separate CLI switch for it, i.e. it's not why oxipng compresses files better.

2) oxipng is faster (multithreaded, uses all avaliable cores/threads by default).

As stated in the project's README:

The core goal of rewriting OptiPNG was to implement multithreading, which would be very difficult to do within the existing C codebase of OptiPNG. This also served as an opportunity to choose a more modern, safer language (Rust).

Multi-threading would be especially useful if you need to optimize only a few images (< $(nproc)), because you won't be able to utilize all your threads with single-threaded optipng, unlike with oxipng

makepkg-optimize.conf isn't properly created

CFLAGS="-march=native -O3 -pipe -fno-plt -fexceptions \
#CFLAGS="-march=native -O3 -pipe -fstack-protector-strong --param=ssp-buffer-size=4 -fno-plt -fopenmp -pthread -Wno-error -w" #
#CFLAGS="-march=native -Og -pipe -fstack-protector-strong --param=ssp-buffer-size=4 -fno-plt" #
        -Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security \
        -fstack-clash-protection -fcf-protection"

Mold --separate-debug-file and -fuse-ld=mold needs to be in one section with -Wl

#!/usr/bin/bash
#
#   mold.sh - Compile with the fast mold linker
#

[[ -n "$LIBMAKEPKG_BUILDENV_MOLD_SH" ]] && return
LIBMAKEPKG_BUILDENV_MOLD_SH=1

LIBRARY=${LIBRARY:-'/usr/share/makepkg'}

source "$LIBRARY/util/option.sh"

build_options+=('mold')
buildenv_functions+=('buildenv_mold')

buildenv_mold() {
    if
    check_buildoption "mold" "y"; then
            LDFLAGS="${LDFLAGS/-Wl,/-fuse-ld=mold -Wl,}"
        LDFLAGS+=",--separate-debug-file"
    fi
}

clang pgo: https://clang.llvm.org/docs/UsersManual.html#profile-guided-optimization isn't easy like pgo in gcc because more opportunity

@brikler

Actually I have never changed anything on the PGO thing from the upstream makepkg-optimize and I did tested it just with gcc - im sorry. Clang uses/needs other options for that. If I have time I will look into this!

it doesn't work as expected. the first compilation breaks with:

clang-14: warning: argument unused during compilation: '-fprofile-dir=/home/tom/compile/upx/upx.used' [-Wunused-command-line-argument]
clang-14: warning: argument unused during compilation: '-fprofile-dir=/home/tom/compile/upx/upx.used' [-Wunused-command-line-argument]
error: error: Could not read profile default.profdata: No such file or directoryCould not read profile default.profdata: No such file or directory

1 error generated.
make[1]: *** [Makefile:92: help.o] Error 1
make[1]: *** Waiting for unfinished jobs....
1 error generated.
make[1]: *** [Makefile:92: main.o] Error 1
make: *** [Makefile:31: all] Error 2
==> ERROR: A failure occurred in build().
    Aborting...

PROFDEST was set in the PKGBUILD, and upx.genwas on the correct place.

there is something wrong in general, because its not on me to create gen and used folder.

edit1: it works with gcc, but not with 'clang'

edit2: a propos UNITY_BUILD it is more or less a developer tool, and it will not work on every project, but if, the time to compile a project can be reduced up to 50%. more about unity build: https://mesonbuild.com/Unity-builds.html

Cause of incremental builds I have something here: https://github.com/ptr1337/dotfiles/blob/master/scripts/toolchain/build_llvm.bash#L242

But it is not very well tested. It does work so for but has some issues and it also uses "check" at the compilation.