Arch Linux User Repository

makepkg-optimize is a collection of libmakepkg tidy, buildenv, and executable extensions, and a supplement to pacman. These enable various optimization routines for building and packaging such as upx compression, profile guided optimization, polyhedral model optimization, etc..

Note: Over-optimization is a thing, and it is not good.

See the wiki article for details.

Note to packagers: makepkg-optmize's macros may be enabled or disabled in options() as well!

to day i wanted to try lto-thin but it seems clang doesn't work with graphite

  clang-10: error: unknown argument: '-fgraphite-identity'
    clang-10: error: unknown argument: '-floop-interchange'
    clang-10: error: unknown argument: '-floop-nest-optimize'
    clang-10: error: unknown argument: '-floop-parallelize-all'
    clang-10: error: unknown argument: '-ftree-loop-distribution'
    clang-10: error: unknown argument: '-ftree-parallelize-loops=2'
    clang-10: error: unsupported argument 'auto' to option 'flto='

BUILDENV=(!distcc color !ccache check !sign lto-thin graphite !pgo)

[tom@donar compiz]$ pacman -Q clang
clang 10.0.0-3

@quequotion

if saw this in makepkg.conf and it's not necessary

-T "$(getconf _NPROCESSORS_ONLN)"
and
zstd -c -z -q --ultra -T0 -22 -)

because xz can do the same with:

#xz --long-help
  -T, --threads=NUM   use at most NUM threads; the default is 1; set to 0
                      to use as many threads as there are processor cores

#man zstd
zstdmt is equivalent to zstd -T0

@Anonymous Descriptions added

@brikler I've filed an issue upstream to see if we can get some clarification about what mimetypes are safe to compress. Also, UPX just had its first new release in two years, so there may be new capabilities. Looking at the code, it seems like application/x-sharedlib should be covered. Still unsure about application/x-pie-executable.

Could you add descriptions for the optdepends and what they are used for?

@quequotion always outputs "warning, not compressing... this message should appear when 'application/x-executable', 'application/x-dosexec', 'application/x-sharedlib' should be compressed and the system are using more x-sharedlib then other types… so it's probably a x-sharedlib to compress

[tom@donar ~]$ file --brief --mime-type /usr/bin/* | grep application/x-sharedlib | wc
   1109    1109   26616
[tom@donar ~]$ file --brief --mime-type /usr/bin/* | grep application/x-executable | wc
    427     427   10675
[tom@donar ~]$ file --brief --mime-type /usr/bin/* | grep application/x-sharedlib | wc
   1109    1109   26616
[tom@donar ~]$ file --brief --mime-type /usr/lib/* | grep application/x-executable | wc
      1       1      25

https://aur.archlinux.org/packages/makepkg-optimize/?O=20&PP=10#comment-675946 https://aur.archlinux.org/packages/makepkg-optimize/?O=20&PP=10#comment-675582

@brikler

I wonder if stripping even works on a compressed binary..

The scripts should be executed in alphanumerical order (this is why lto.sh.in became ZZ-lto.sh.in); strip.sh should be happening before upx.sh anyway..

I'll look into it.

EDIT: I have been unable to compress any binaries in any packages with upx, as makepkg-optimize always outputs "warning, not compressing..." even when I deleted that code from upx.sh. I experienced an alternate reality in which no changes to upx.sh had any effect on the execution of upx.sh and I am at a loss to explain that.

EDIT again: fixed the "alternate reality" problem; apparently packaging had duplicated the tidy scripts as the executable scripts; should also fix strip and upx running out of order

  -> Compressing binaries with UPX...
  -> Stripping unneeded symbols from binaries and libraries...

imo should the binaries be stripped before they will be compressed because less to compress when stripped, right?

@bartus

I've made two updates since, changing how old used profiles are handled. I suspect this was the weak point: the old conditions allowed for a scenario in which both a directory for generated profiles and an old directory of used profiles may exist simultaneously at build time--in which case pgo would silently fail to utilize the generated profiles and build an unoptimized package. Now, if a folder for newly generated files exists, and a folder of old used profiles also exists, the old used profiles will be deleted and the previously generated profiles will be used to optimize.

Perhaps it has been the case that I always deleted the old profiles by hand, so it had not been a problem for me before.