Arch Linux User Repository

@mhx @KokaKiwi My problem was more packaging/build system related: When I download the git repo of the AUR pkg, enter git reset --hard HEAD^, therefore dropping 57421aacd756 , which to my understanding excludes the perfmon test, extra_x86-64_build and the CD passed, since the tests depending on fuse were disable (3079 tests run). Now with the latest commit, the testsuite fails. Also, you'd expect it to have one less test, in total now 3078. But it does have 3087 tests, which can't be right. When you provide the -E <regex> arg twice to ctest, it silently drops the first -E <regex>. That's why those tests are enabled now after the latest commit of the AUR repo (I think that's where you got confused), the CD build fails while it worked fine in February.

@KokaKiwi Can you please fix this issue and change it to -E '(dwarfs/tools_test\..*|dwarfsextract_test\.perfmon)' ?

FWIW, I've added Arch (and Fedora) builds to the DwarFS CI pipeline: https://github.com/mhx/dwarfs/actions/runs/8526321352/job/23355308124

What image did you use?

I'm using this Dockerfile. The Makefile in the same directory has build_arch and run_arch targets for building/running the container.

The --cap-add SYS_ADMIN --device /dev/fuse --privileged arguments are necessary for FUSE to work within the container, and I'm assuming a lack of these arguments to be the root cause of this issue. Running without the arguments:

99% tests passed, 9 tests failed out of 3090

Total Test time (real) =   5.72 sec

The following tests FAILED:
        2965 - dwarfs/tools_test.end_to_end/standalone (Subprocess aborted)
        2966 - dwarfs/tools_test.end_to_end/universal-tool (Subprocess aborted)
        2967 - dwarfs/tools_test.end_to_end/universal-symlink (Subprocess aborted)
        2968 - dwarfs/tools_test.mutating_and_error_ops/standalone (Subprocess aborted)
        2969 - dwarfs/tools_test.mutating_and_error_ops/universal-tool (Subprocess aborted)
        2970 - dwarfs/tools_test.mutating_and_error_ops/universal-symlink (Subprocess aborted)
        2971 - dwarfs/tools_test.categorize/standalone (Subprocess aborted)
        2972 - dwarfs/tools_test.categorize/universal-tool (Subprocess aborted)
        2973 - dwarfs/tools_test.categorize/universal-symlink (Subprocess aborted)
Errors while running CTest

However, with these arguments:

100% tests passed, 0 tests failed out of 3090

Total Test time (real) =   6.10 sec

@mhx: The failed tests using extra-x86_64-build and on my CD are exactly the same. Version 0.9.6-1 (The 1st one, thought the git hooks on the AUR servers block commits without any version increase until now) was built by my CD in February fine: http://repo.oscloud.info/

So either the last commit, which doesn't make sense to me or some change in Arch causes the error that I have. I'm gonna investigate, but it'll take a week or 2 as I don't have time for it right now.

(in which dwarfs builds/tests just fine otherwise),

What image did you use? archlinux/archlinux is the one updated frequently, which my CD uses. Using a fresh docker container should behave very similar to a extra-x86_64-build. If the image is a couple days old you could use https://gist.github.com/bionade24/b3aa8240958937f80d86f4d4c8686971 to create an Arch image from scratch. It needs arch-install-scripts and pacman, which most Linux distros have in their repos. If you just update inside the container, most config updates would have to be applied by manual intervention and reside in *.pacnew files. But I wouldn't rely that running pacdiff, choosing overwrite with .pacnew on every prompt results in the same setting as a fresh image.

@bionade24 Oh, that's a completely different issue :)

I have to admit I'm not too familiar with Arch and all its development tools. I've only got Arch installed on a Raspberry Pi 4 (so no extra-x86_64-build, and apparently also no ARM equivalent) and my attempts to setup a chroot build environment got stuck at

$ arch-nspawn $CHROOT/root pacman -Syu
:: Synchronizing package databases...
 core
 core.db failed to downloaderror: failed retrieving file 'core.db' from mirror.archlinuxarm.org : The requested URL returned error: 404
error: failed to synchronize all databases (failed to retrieve some files)

I've also tried to run extra-x86_64-build in an x86_64 Arch docker container (in which dwarfs builds/tests just fine otherwise), but again, no luck:

[root@3cbf804f27ef dwarfs]# extra-x86_64-build
Failed to retrieve machine ID: No such file or directory
Attempted to remove disk file system under "/run/systemd/nspawn/propagate/arch-nspawn-1714", and we can't allow that.
==> ERROR: Aborting...

Just by looking at which tests failed in your log file, my hunch is that the build environment somehow prevents the FUSE driver from working.

@mhx The build fails for me in my CD with clean build environments, too. Full build log: https://abs-cd.oscloud.info/cd_manager/dwarfs The build also fails for me using extra-x86_64-build script, you should be able to reproduce it with that.

@QTaKs & @Lerin, can you please post the full output of the failing test? I'd expect there to be additional output after the Expected: true line, like:

/home/mhx/git/github/dwarfs/test/tool_main_test.cpp:544: Failure
Value of: std::regex_search(errs, perfmon_re)
  Actual: false
Expected: true
I 11:14:13.484382 extraction finished without errors
[filesystem_v2.readv_future]
      samples: 5
      overall: 51.02us
  avg latency: 10.2us
  p50 latency: 1.024us
  p90 latency: 65.54us
  p99 latency: 65.54us
[...]

This would help a lot in figuring out the root cause, as I'm unable to reproduce the problem, even on arch:

[mhx@archpi build]$ ./tool_main_test "--gtest_filter=dwarfsextract_test.perfmon"
Running main() from /build/gtest/src/googletest-1.14.0/googletest/src/gtest_main.cc
Note: Google Test filter = dwarfsextract_test.perfmon
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from dwarfsextract_test
[ RUN      ] dwarfsextract_test.perfmon
[       OK ] dwarfsextract_test.perfmon (147 ms)
[----------] 1 test from dwarfsextract_test (147 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (147 ms total)
[  PASSED  ] 1 test.

Same error. Everything updated. I also tried to build a git version, nothing has changed.

2833/3079 Testing: dwarfsextract_test.perfmon
2833/3079 Test: dwarfsextract_test.perfmon
Command: "/mnt/tmpfs/makepkg/dwarfs/src/build/tool_main_test" "--gtest_filter=dwarfsextract_test.perfmon" "--gtest_also_run_disabled_tests"
Directory: /mnt/tmpfs/makepkg/dwarfs/src/build
"dwarfsextract_test.perfmon" start time: Mar 07 20:27 +04
Output:
----------------------------------------------------------

Running main() from /usr/src/debug/gtest/googletest-1.14.0/googletest/src/gtest_main.cc
Note: Google Test filter = dwarfsextract_test.perfmon
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from dwarfsextract_test
[ RUN      ] dwarfsextract_test.perfmon
/mnt/tmpfs/makepkg/dwarfs/src/dwarfs/test/tool_main_test.cpp:544: Failure
Value of: std::regex_search(errs, perfmon_re)
  Actual: false
Expected: true

99% tests passed, 1 tests failed out of 3079

Total Test time (real) = 124.68 sec

The following tests FAILED: 2833 - dwarfsextract_test.perfmon (Failed) Errors while running CTest Output from these tests are in: /home/lerin/.cache/yay/dwarfs/src/build/Testing/Temporary/LastTest.log Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely. make: *** [Makefile:91: test] Błąd 8 ==> BŁĄD: Wystąpił błąd w check(). Przerywam... -> błąd podczas budowania: dwarfs-exit status 4 -> Failed to install the following packages. Manual intervention is required: dwarfs - exit status 4

LastTestsFailed.log :: 2833:dwarfsextract_test.perfmon

Well i just want report

Yes, this works for me as well. Thank you.