Package Details: python2-scandir 1.10.0-8

Git Clone URL: https://aur.archlinux.org/python2-scandir.git (read-only, click to copy)
Package Base: python2-scandir
Description: Better directory iterator and faster os.walk() alternative
Upstream URL: https://github.com/benhoyt/scandir
Licenses: BSD
Submitter: hexd
Maintainer: hexd
Last Packager: hexd
Votes: 4
Popularity: 0.022430
First Submitted: 2022-06-12 05:39 (UTC)
Last Updated: 2022-06-18 04:43 (UTC)

Dependencies (2)

Required by (1)

Sources (1)

Latest Comments

« First ‹ Previous 1 2 3 Next › Last »

MarsSeed commented on 2022-06-16 11:53 (UTC)

Thanks to both of, you, @jfk and @ravenblackdusk.

@hexd, I've changed the test locale to LC_ALL=C.UTF-8 in python2-pyopenssl's check().

According to @jfk's comment, it resolved the UnicodeEncodeError there.

Based on input from both @jfk and @ravenblackdusk, as well as my own testing, it seems that if we set any <lang>.UTF-8 locale that is not available on a particular system (neither in glibc nor via locale-gen), python2 will fall back to the ancient ANSI_X3.4-1968 encoding for its filesystem access. (The same as if we were forcing LC_ALL=C).

Only locales POSIX, C and C.UTF-8 are universally present on up-to-date Arch-based systems. The latter has just been added recently to glibc, on 6 June 2022, with package version 2.35-6 (systemd needs that as fallback now, and "forced" the Arch devs to include it, according to them).

It is possible some users still don't have it; they'll have to update to at least glibc 2.35-6.

Fortunately even Manjaro's roughly one-month-delayed stable-branch repos received this glibc version 2 days after Arch, as a fast-tracked package upgrade.

So @hexd, please change your testing locale to C.UTF-8 and cross your fingers! :D

ravenblackdusk commented on 2022-06-16 11:36 (UTC)

I'm using manjaro I don't use testing repositories AFAIK.

git clone https://aur.archlinux.org/python2-scandir.git
cd python2-scandir
makepkg --syncdeps --rmdeps --install

seemingly same error: UnicodeEncodeError: 'ascii' codec can't encode character u'\u018f' in position 72: ordinal not in range(128)

$ env | grep -E '^(LANG|LC_)' | sort
LANG=en_GB.UTF-8
LC_CTYPE=en_GB.utf8
$ python2 -c 'import sys; print("default: " + sys.getdefaultencoding()); print("filesystem: " + sys.getfilesystemencoding())'
default: ascii
filesystem: UTF-8
$ LC_ALL=en_US.UTF-8 python2 -c 'import sys; print("default: " + sys.getdefaultencoding()); print("filesystem: " + sys.getfilesystemencoding())'
default: ascii
filesystem: ANSI_X3.4-1968
$ locale --all-locales
C
C.UTF-8
en_GB.utf8
POSIX

tpaniaki commented on 2022-06-16 07:45 (UTC) (edited on 2022-06-16 09:27 (UTC) by tpaniaki)

$ LC_ALL=en_US.UTF-8 python2 -c 'import sys; print("default: " + sys.getdefaultencoding()); print("filesystem: " + sys.getfilesystemencoding())'
default: ascii
filesystem: ANSI_X3.4-1968
$ python2 -c 'import sys; print("default: " + sys.getdefaultencoding()); print("filesystem: " + sys.getfilesystemencoding())'
default: ascii
filesystem: UTF-8
$ locale --all-locales
C
C.UTF-8
fr_CH.utf8
POSIX

EDIT: build eventually went through after installing python2-pyopenssl (see MarsSeed's thread for details). I cannot pinpoint a reason though.

Daerun commented on 2022-06-16 07:39 (UTC)

Just wanted to say that in my case (vanilla Arch and locale set to catalan) I had to comment the line containing "LC_ALL" for it to work.

MarsSeed commented on 2022-06-16 00:20 (UTC)

+1: I'm also interested in the output of this command:

locale --all-locales

MarsSeed commented on 2022-06-16 00:09 (UTC) (edited on 2022-06-16 01:18 (UTC) by MarsSeed)

Thanks for both of you for your replies. On the surface, everything looks fine with those LANG settings.

But could you please also share the output of the following statement? (I have a hunch and I've already found some potential discrepancy; will discuss later).

python2 -c 'import sys; print("default: " + sys.getdefaultencoding()); print("filesystem: " + sys.getfilesystemencoding())'

EDIT: And also please report the results of the following variant:

LC_ALL=en_US.UTF-8 python2 -c 'import sys; print("default: " + sys.getdefaultencoding()); print("filesystem: " + sys.getfilesystemencoding())'

tpaniaki commented on 2022-06-15 17:00 (UTC) (edited on 2022-06-15 17:36 (UTC) by tpaniaki)

I'm running Gnome on Arch with default repos (meaning I haven't set anything in particular).

$ env | grep -E '^(LANG|LC_)' | sort
LANG=fr_CH.UTF-8
$ findmnt -T ~/.cache/yay/python2-scandir 
TARGET SOURCE                  FSTYPE OPTIONS
/home  /dev/mapper/volume-home ext4   rw,relatime

robertc commented on 2022-06-15 16:07 (UTC)

I am getting the same errors both with Pamac and Yay, I am using EndeavourOS.

$ env | grep -E '^(LANG|LC_)' | sort
LANG=en_GB.UTF-8
LC_COLLATE=en_GB.UTF-8

MarsSeed commented on 2022-06-15 13:53 (UTC)

@ravenblackdusk, are you using Manjaro? In that case, which branch?

@jfk, same question to you - which distro are you using? And within that, do you use the default 'branch' / default repos, or maybe 'testing' / 'unstable' etc.?

Also to both of you: what is the output of the following command on your system(s)?

env | grep -E '^(LANG|LC_)' | sort

Thank you for your responses in advance.

tpaniaki commented on 2022-06-15 13:09 (UTC)

Still getting the same errors on my system as well.