Package Details: python-safetensors 0.4.5-2

Git Clone URL: https://aur.archlinux.org/python-safetensors.git (read-only, click to copy)
Package Base: python-safetensors
Description: Simple, safe way to store and distribute tensors
Upstream URL: https://github.com/huggingface/safetensors
Licenses: Apache-2.0
Submitter: Henry-ZHR
Maintainer: envolution
Last Packager: envolution
Votes: 6
Popularity: 1.04
First Submitted: 2023-01-08 09:54 (UTC)
Last Updated: 2024-12-18 15:54 (UTC)

Sources (1)

Latest Comments

1 2 3 4 Next › Last »

actionless commented on 2024-12-20 13:53 (UTC)

weird, i've rebuilt related libraries from (flax, jax, jaxlib), and i still have exception in the same test, although this time it's with stacktrace instead of being just a crash:

tests/test_flax_comparison.py [genesect:2646977:0:2647130] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x61616274)
==== backtrace (tid:2647130) ====
 0 0x000000000004c822 ucs_event_set_fd_get()  ???:0
 1 0x000000000004c9ed ucs_event_set_fd_get()  ???:0
 2 0x000000000003d1d0 __sigaction()  ???:0
 3 0x00000000000b0681 memcpy()  ???:0
 4 0x000000000730b167 PyInit_xla_extension()  ???:0
 5 0x000000000730ead0 PyInit_xla_extension()  ???:0
 6 0x000000000077ce25 PyInit_xla_extension()  ???:0
 7 0x00000000015d5b78 PyInit_xla_extension()  ???:0
 8 0x0000000004cdc3af PyInit_xla_extension()  ???:0
 9 0x0000000004cdc105 PyInit_xla_extension()  ???:0
10 0x0000000004cddd6b PyInit_xla_extension()  ???:0
11 0x0000000004cc2737 PyInit_xla_extension()  ???:0
12 0x0000000004cc3959 PyInit_xla_extension()  ???:0
13 0x0000000004cc76a8 PyInit_xla_extension()  ???:0
14 0x0000000004cd5840 PyInit_xla_extension()  ???:0
15 0x0000000004cbbe9f PyInit_xla_extension()  ???:0
16 0x0000000004cbaf02 PyInit_xla_extension()  ???:0
17 0x0000000004cc3ed6 PyInit_xla_extension()  ???:0
18 0x0000000004cc4663 PyInit_xla_extension()  ???:0
19 0x0000000004cc4a67 PyInit_xla_extension()  ???:0
20 0x0000000004cc4d3a PyInit_xla_extension()  ???:0
21 0x00000000015c60d6 PyInit_xla_extension()  ???:0
22 0x000000000078837e PyInit_xla_extension()  ???:0
23 0x000000000078019f PyInit_xla_extension()  ???:0
24 0x0000000007fdc0b5 PyInit_xla_extension()  ???:0
25 0x0000000007fdb95e PyInit_xla_extension()  ???:0
26 0x0000000007fd15da PyInit_xla_extension()  ???:0
27 0x000000000009439d pthread_condattr_setpshared()  ???:0
28 0x000000000011949c __clone()  ???:0
=================================
Fatal Python error: Segmentation fault

Thread 0x0000764a61b22b80 (most recent call first):
  File "/usr/lib/python3.12/ast.py", line 257 in iter_fields
  File "/usr/lib/python3.12/ast.py", line 380 in walk
  File "/usr/lib/python3.12/site-packages/_pytest/_code/source.py", line 148 in get_statement_startend2
<... here the same as before ...>

[mutex.cc : 453] RAW: Lock blocking 0x598c93524070   @ 0x74bca89619dc 0x74bca8961701 0x74bca0d29ad3 0x74bca1dd5b1a 0x74bca54dc3af 0x74bca54dc105 0x74bca54ddd6b 0x74bca54c2737 0x74bca54c3959 0x74bca54c76a8 0x74bca54d5840 0x74bca54bbe9f 0x74bca54baf02 0x74bca54c3ed6 0x74bca54c4663 0x74bca54c4a67 0x74bca54c4d3a 0x74bca1dc60d6 0x74bca0f8837e 0x74bca0f8019f 0x74bca87dc0b5 0x74bca87db95e 0x74bca87d15da 0x74bcbeaa339d 0x74bcbeb2849c

and sometimes:

tests/test_flax_comparison.py corrupted size vs. prev_size while consolidating
Fatal Python error: Aborted

do you have any further ideas where to debug that? although i'm not using jax/flax backend so just skipping that particular test only, make the other tests pass, so prolly i don't even need fixing that

actionless commented on 2024-12-18 19:37 (UTC)

thanks for confirming the expected result, i'll try it, i suspect python-flax pkg

envolution commented on 2024-12-18 19:36 (UTC)

@actionless I can't reproduce on clean chroot, nor on my normal environment. I'd suggest trying to rebuild a handful of the dependencies and - failing that - report upstream:

pikaur -S python-numpy python-torch python-jaxlib python-zstandard python-yaml python-pytest

my clean chroot log:

==> Starting check()...
============================= test session starts ==============================
platform linux -- Python 3.12.7, pytest-8.3.3, pluggy-1.5.0
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /build/python-safetensors/src/safetensors-0.4.5/bindings/python
configfile: setup.cfg
plugins: anyio-4.6.2.post1, benchmark-4.0.0, hypothesis-6.118.7
collected 57 items

tests/test_flax_comparison.py ...                                        [  5%]
tests/test_mlx_comparison.py sss                                         [ 10%]
tests/test_pt_comparison.py s..s...s......sss....                        [ 47%]
tests/test_pt_model.py .............                                     [ 70%]
tests/test_simple.py .................                                   [100%]

=============================== warnings summary ===============================
... trimmed ...

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================== 48 passed, 9 skipped, 3 warnings in 4.41s ===================

actionless commented on 2024-12-18 16:54 (UTC)

the log doesn't change

python pkgs list: https://0x0.st/XCaU.txt

envolution commented on 2024-12-18 16:42 (UTC)

@actionless can you please try without the aur helper (pikaur)? I'm not able to reproduce - can you confirm you're using python-h5py and not python-h5py-openmpi? 

pip list > /tmp/python-packages.txt && pacman -Q | grep '^python' >> /tmp/python-packages.txt && curl -F'file=@/tmp/python-packages.txt' https://0x0.st

if you could run that command and share the link it would be appreciated

actionless commented on 2024-12-18 16:18 (UTC)

here it is:

Successfully built safetensors-0.4.5-cp312-cp312-linux_x86_64.whl
==> Starting check()...
================================================================ test session starts =================================================================
platform linux -- Python 3.12.7, pytest-8.3.3, pluggy-1.5.0
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/lie/.cache/pikaur/build/python-safetensors/src/safetensors-0.4.5/bindings/python
configfile: setup.cfg
plugins: hypothesis-6.118.7, benchmark-4.0.0, xonsh-0.18.3, typeguard-4.3.0, anyio-4.6.2.post1
collected 57 items

tests/test_flax_comparison.py double free or corruption (out)
Fatal Python error: Aborted

Thread 0x00007b1c41085b80 (most recent call first):
  File "/usr/lib/python3.12/ast.py", line 52 in parse
  File "/usr/lib/python3.12/site-packages/_pytest/_code/source.py", line 181 in getstatementrange_ast
  File "/usr/lib/python3.12/site-packages/_pytest/_code/code.py", line 262 in getsource
  File "/usr/lib/python3.12/site-packages/_pytest/_code/code.py", line 828 in _getentrysource
  File "/usr/lib/python3.12/site-packages/_pytest/_code/code.py", line 930 in repr_traceback_entry
  File "/usr/lib/python3.12/site-packages/_pytest/_code/code.py", line 992 in repr_traceback
  File "/usr/lib/python3.12/site-packages/_pytest/_code/code.py", line 1058 in repr_excinfo
  File "/usr/lib/python3.12/site-packages/_pytest/_code/code.py", line 690 in getrepr
  File "/usr/lib/python3.12/site-packages/_pytest/nodes.py", line 456 in _repr_failure_py
  File "/usr/lib/python3.12/site-packages/_pytest/python.py", line 1669 in repr_failure
  File "/usr/lib/python3.12/site-packages/_pytest/reports.py", line 376 in from_item_and_call
  File "/usr/lib/python3.12/site-packages/_pytest/runner.py", line 368 in pytest_runtest_makereport
  File "/usr/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/usr/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/usr/lib/python3.12/site-packages/_pytest/runner.py", line 244 in call_and_report
  File "/usr/lib/python3.12/site-packages/_pytest/runner.py", line 132 in runtestprotocol
  File "/usr/lib/python3.12/site-packages/_pytest/runner.py", line 113 in pytest_runtest_protocol
  File "/usr/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/usr/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/usr/lib/python3.12/site-packages/_pytest/main.py", line 362 in pytest_runtestloop
  File "/usr/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/usr/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/usr/lib/python3.12/site-packages/_pytest/main.py", line 337 in _main
  File "/usr/lib/python3.12/site-packages/_pytest/main.py", line 283 in wrap_session
  File "/usr/lib/python3.12/site-packages/_pytest/main.py", line 330 in pytest_cmdline_main
  File "/usr/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/usr/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/usr/lib/python3.12/site-packages/_pytest/config/__init__.py", line 175 in main
  File "/usr/lib/python3.12/site-packages/_pytest/config/__init__.py", line 201 in console_main
  File "/usr/bin/pytest", line 8 in <module>

Extension modules: ujson, jaxlib.cpu_feature_guard, numpy._core._multiarray_umath, numpy.linalg._umath_linalg, zstandard.backend_c, msgpack._cmsgpack, yaml._yaml, torch._C, torch._C._dynamo.autograd_compiler, torch._C._dynamo.eval_frame, torch._C._dynamo.guards, torch._C._dynamo.utils, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special (total: 18)
/home/lie/.cache/pikaur/build/python-safetensors/PKGBUILD: line 55: 1400329 Aborted                 (core dumped) PYTHONPATH="${PWD}/test_build/usr/lib/python${python_version}/site-packages" pytest tests/ --ignore=tests/test_paddle_comparison.py --ignore=tests/test_tf_comparison.py
==> ERROR: A failure occurred in check().
    Aborting...

envolution commented on 2024-12-18 15:41 (UTC)

@actionless - it would be great if you'd share the actual failed check. It's possible one of the optional depends should be included in checkdepends - but without seeing your error it's difficult to know which.

If you don't want to bother helping troubleshoot, just install the optional depends and let us know if it's solved.

actionless commented on 2024-12-18 12:16 (UTC)

numpy problem is gone after all arch packages got eventually updated or rebuilt with same newer numpy version, however check() is still failing now (but with just crash without any traceback) - do anyone else have the same issue?

mane.andrea commented on 2024-08-11 03:00 (UTC)

For context: the error seems to be with Tensorflow. There is a report in Arch

https://gitlab.archlinux.org/archlinux/packaging/packages/tensorflow/-/issues/11

I upgraded the package and skipped the tensorflow tests, unfortunately.

gatsby commented on 2024-07-23 07:19 (UTC)

Package isn't passing the check. Seems to be related to the NumPy 2.0 update.

1 2 3 4 Next › Last »