After updating to 2.13.2 I've found it impossible to mount properly:
-> ls /cvmfs/lhcb.cern.ch
-> sudo mount -t cvmfs lhcb.cern.ch /cvmfs/lhcb.cern.ch
CernVM-FS: running with credentials 949:948
CernVM-FS: loading Fuse module... done
CernVM-FS: mounted cvmfs on /cvmfs/lhcb.cern.ch
-> ls /cvmfs/lhcb.cern.ch
conda/ containers/ etc/ group_login.csh@ group_login.csh.pre-LBCORE-1949@ group_login.sh@ group_login.sh.pre-LBCORE-1949@ lhcbdirac/ lib/ README
-> ls /cvmfs/lhcb.cern.ch/etc
This then simply hangs.
In parallel I can search for any cmvfs processes, but don't see anything odd:
-> ps aux | grep cvmfs
cvmfs 6122 0.0 0.0 37540 3292 ? S 13:26 0:00 /usr/bin/cvmfs2 -o rw,system_mount,fsname=cvmfs2,allow_other,grab_mountpoint,uid=949,gid=948 lhcb.cern.ch /cvmfs/lhcb.cern.ch
cvmfs 6124 0.0 0.0 48128 12008 ? Sl 13:26 0:00 /usr/bin/cvmfs2 __cachemgr__ . 11 12 4194304000 2097152000 1 3 -1 :
cvmfs 6128 0.0 0.0 37540 3264 ? S 13:26 0:00 /usr/bin/cvmfs2 __cachemgr__ . 11 12 4194304000 2097152000 1 3 -1 :
cvmfs 6133 0.0 0.0 636512 31672 ? Sl 13:26 0:00 /usr/bin/cvmfs2 -o rw,system_mount,fsname=cvmfs2,allow_other,grab_mountpoint,uid=949,gid=948 lhcb.cern.ch /cvmfs/lhcb.cern.ch
dan 6188 0.0 0.0 5872 3852 pts/0 S+ 13:26 0:00 ls --color=auto -F /cvmfs/lhcb.cern.ch/etc
Any tips?
Pinned Comments
ktht commented on 2025-09-30 18:12 (UTC)
@DiV21dSREL I had almost the exact same issue: after the Manjaro update on 29/09/25 I was unable to access
/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase, although its parent directory was accessible. After a fair bit of debugging (rebuilding cvmfs with debug symbols; settingCVMFS_HTTP_TRACING=onand specifyingCVMFS_DEBUGLOGin/etc/cvmfs/default.local), it finally clicked for me that the offending package must be curl, as it got stuck waiting for the response from cvmfs servers. It also lines up with the fact that curl was upgraded from version 8.15 to 8.16 with the Manjaro update. After downgrading curl to version 8.15 (withsudo manjaro-downgrade curlif you're on Manjaro), I was able to access cvmfs just like before the update.I also ran
git bisectbetween tags 8.15 and 8.16 in the curl repo to figure out the commit that broke cvmfs. The offending commit happens to be this one, which is part of this PR. Following the discussion there, it seems that the PR introduced a regression in curl, which effectively caused it to run into a deadlock in some scenarios (cf this issue for details). The bug was fixed 2 weeks ago, and the problem is now gone with the current master branch. So we'll just have to wait for the next release of curl.TLDR: downgrade curl to version 8.15; skip 8.16; wait for version 8.17.