Arch Linux User Repository

Search Criteria

Enter search criteria

Search by

Keywords

Out of Date

Sort by

Sort order

Per page

Package Details: python-tokenizers 0.21.0-1

Package Actions

Git Clone URL:	https://aur.archlinux.org/python-tokenizers.git (read-only, click to copy)
Package Base:	python-tokenizers
Description:	Fast State-of-the-Art Tokenizers optimized for Research and Production
Upstream URL:	https://github.com/huggingface/tokenizers
Keywords:	huggingface
Licenses:	Apache-2.0
Submitter:	filipg
Maintainer:	xiota (daskol)
Last Packager:	xiota
Votes:	8
Popularity:	0.96
First Submitted:	2021-10-23 11:17 (UTC)
Last Updated:	2024-12-21 18:21 (UTC)

Dependencies (14)

python (python37^AUR, python311^AUR, python310^AUR)
clang (llvm-rocm-git^AUR, llvm-git^AUR, clang-minimal-git^AUR, clang17-bin^AUR) (make)
python-build (make)
python-installer (python-installer-git^AUR) (make)
python-maturin (python-maturin-git^AUR) (make)
python-setuptools-rust (python-setuptools-rust-git^AUR) (make)
python-wheel (make)
rust-bindgen (make)
python-datasets^AUR (check)
python-numpy (python-numpy-flame^AUR, python-numpy-git^AUR, python-numpy-mkl-bin^AUR, python-numpy-mkl-tbb^AUR, python-numpy-mkl^AUR, python-numpy1^AUR) (check)
python-pyarrow (check)
python-pytest (check)
python-requests (check)
python-setuptools-rust (python-setuptools-rust-git^AUR) (check)

Required by (11)

Sources (3)

Pinned Comments

xiota commented on 2024-08-30 16:15 (UTC) (edited on 2024-08-30 16:59 (UTC) by xiota)

Problems:

Try building in a clean chroot.
Provide logs (in a pastebin) to demonstrate the problem.

Latest Comments

« First ‹ Previous 1 2 3 4

xiota commented on 2023-02-14 03:02 (UTC)

@groo This package should build in a clean chroot (extra-x86_64-build). If you're having problems, try running makepkg directly instead of using AUR helpers.

groo commented on 2022-12-16 18:09 (UTC)

The package does not compile for me. It fails with following message:

[...output ommitted...] running build_rust cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module -- --crate-type cdylib error: 'json-render-diagnostics' isn't a valid value for '--message-format <FMT>' [possible values: human, json, short]

    Did you mean 'json'?

USAGE: cargo rustc --features <FEATURES> --lib --manifest-path <PATH> --message-format <FMT> --release --verbose

For more information try --help error: cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module -- --crate-type cdylib failed with code 1 ==> FEHLER: Ein Fehler geschah in build(). Breche ab... -> Fehler beim Erstellen: python-tokenizers

filipg commented on 2022-12-10 08:07 (UTC)

It depends on python-setuptools-rust which, in turn, depends on python-setuptools, so I assume it is not needed to add it explicitly.

CorvetteCole commented on 2022-10-06 00:06 (UTC)

This package should depend on python-setuptools

« First ‹ Previous 1 2 3 4