Package Details: python-safetensors 0.4.5-2

Git Clone URL: https://aur.archlinux.org/python-safetensors.git (read-only, click to copy)
Package Base: python-safetensors
Description: Simple, safe way to store and distribute tensors
Upstream URL: https://github.com/huggingface/safetensors
Licenses: Apache-2.0
Submitter: Henry-ZHR
Maintainer: envolution
Last Packager: envolution
Votes: 6
Popularity: 1.04
First Submitted: 2023-01-08 09:54 (UTC)
Last Updated: 2024-12-18 15:54 (UTC)

Dependencies (28)

Sources (1)

Latest Comments

« First ‹ Previous 1 2 3 4 Next › Last »

dylondark commented on 2024-07-16 05:47 (UTC) (edited on 2024-07-16 05:47 (UTC) by dylondark)

Package isn't passing the check. Seems to be related to the NumPy 2.0 update.

ImportError: 
A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.


============================================= short test summary info ==============================================
ERROR tests/test_tf_comparison.py - AttributeError: `np.complex_` was removed in the NumPy 2.0 release. Use `np.complex128` instead.. Did you mean:...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================= 1 error in 2.70s =================================================

mane.andrea commented on 2024-04-23 16:07 (UTC)

@hashworks I tried to build it in two different computers and in a clean chroot and I could not reproduce the issue. Is your system up-to-date? Can you try to build it in a clean chroot?

hashworks commented on 2024-04-21 17:08 (UTC)

I'm getting a test failure at tests/test_pt_comparison.py saying RuntimeError: HIP error: shared object initialization failed. Can you reproduce that?

https://fb.hash.works/JThvZA/

mane.andrea commented on 2024-04-17 17:57 (UTC)

@carsme Many thanks, I have patched it as you say

carsme commented on 2024-04-17 17:51 (UTC)

Two issues:

  • Re-building without cleaning the build directory fails due to the mkdir invocation.
  • Building with rustup instead of rust fails, due to that no toolchain version is specified.

These issues can be fixed by the following patch:

diff --git a/PKGBUILD b/PKGBUILD
index 03774ec..e069144 100644
--- a/PKGBUILD
+++ b/PKGBUILD
@@ -39,11 +39,13 @@ source=("${pkgname}-${pkgver}.tar.gz::${url}/archive/refs/tags/v${pkgver}.tar.gz
 sha512sums=('278469d7bc8a4285519cf8020891d0f0e107d8124ad2c24686ca8a19ef2f210c49d9483c502bd5aa5d73940a775ac3eb5e3b90f62825dd7845c351bbdc8c82c3')

 prepare() {
-  mkdir "safetensors-${pkgver}/.git"
+  mkdir -p "safetensors-${pkgver}/.git"
 }

 build() {
   cd "safetensors-${pkgver}/bindings/python"
+
+  export RUSTUP_TOOLCHAIN=stable
   python -m build --wheel --no-isolation
 }

mane.andrea commented on 2023-12-12 18:29 (UTC)

With @daskol patch now builds fine. I tried a chroot build in a machine with cuda and in a machine without cuda

mane.andrea commented on 2023-12-09 03:42 (UTC)

Sorry for the delay. I overlooked the notification. Will work on an updated PKGBUILD asap

daskol commented on 2023-12-09 02:41 (UTC)

In my optinion, the best solution is not put any .gitignore with glob pattern in AUR repo since there is no actual need in blacklisting files at all.

@h3ss I believe that it'd be better to create an empty directory .git at $srcdir/safetensors-$pkgver rather than patching or sed'ing (see ignore::WalkBuilder). Anyway, we should create an issue in PyO3/maturin. It is stupid that it traverses out of repo root. What about submodules? Why I should tweak include/ignore filters in pyproject.toml? Weird.

diff --git a/PKGBUILD b/PKGBUILD
index cb94dc8..4d04fcf 100644
--- a/PKGBUILD
+++ b/PKGBUILD
@@ -38,6 +38,10 @@ checkdepends=('python-pytorch'
 source=("${pkgname}-${pkgver}.tar.gz::${url}/archive/refs/tags/v${pkgver}.tar.gz")
 sha512sums=('da38f184f2c0730a59b1d4ca490758b71cb5ed3e1e16741bbf4b7bfac35fa1a4f24e5acbc35f3b38f10e4c46d81554896fbd95accfd671ee066c9fcbd50551ab')

+prepare() {
+  mkdir "safetensors-${pkgver}/.git"
+}
+
 build() {
   cd "safetensors-${pkgver}/bindings/python"
   python -m build --wheel --no-isolation

@xiota There is the same issue with python-tokenziers.

h3ss commented on 2023-12-05 03:34 (UTC)

The problem is that the AUR package includes a .gitignore file, and the maturin build backend will ignore anything specified in any .gitignore file in the path.

See: - Wheels for mixed Python/Rust project does not include Python source code #885

The contents of the AUR .gitignore are:

**/*
!.SRCINFO
!.gitignore
!PKGBUILD

That first line is the problem, it basically excludes everything from inclusion (except what the Rust part of the build creates apparently).

I have been able to get around this by editing the src/safetensors-0.4.0/bindings/python/pyproject.toml file. Specifically, by adding this under the [tool.maturin] section:

include = ["**/*"]

See: - Maturin User Guide - Configuration - Specifically the include part

When this is done, the resulting .whl file reliably contains the needed Python source files.

So, how can we get around this in the PKGBUILD? By using sed to add the include line to the pyproject.toml in the build() function:

build() {
  cd "safetensors-${pkgver}/bindings/python"
  sed -i 's/\[tool.maturin\]/[tool.maturin]\ninclude = \["**\/*"\]/g' pyproject.toml
  python -m build --wheel --no-isolation
}

After adding that sed line, the package reliably passes all checks and builds.

@mane.andrea Can we get this or something equivalent incorporated into the PKGBUILD? I'm confident this should fix this longstanding issue.

PhotonX commented on 2023-11-26 18:14 (UTC) (edited on 2023-11-26 18:15 (UTC) by PhotonX)

I'm getting the check() errors as well. I don't understand what the underlying problem is, though. One of the errors is, exemlarily:

________________ ERROR collecting tests/test_flax_comparison.py ________________
ImportError while importing test module '/home/photon/.cache/yay/python-safetensors/src/safetensors-0.4.0/bindings/python/tests/test_flax_comparison.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.11/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_flax_comparison.py:12: in <module>
    from safetensors import safe_open
E   ImportError: cannot import name 'safe_open' from 'safetensors' (unknown location)

So it searches for safe_open from... itself? I think, I have all the necessary dependencies installed, but none of them provides this...

« First ‹ Previous 1 2 3 4 Next › Last »