@groo This package should build in a clean chroot (extra-x86_64-build). If you're having problems, try running makepkg directly instead of using AUR helpers.
Search Criteria
Package Details: python-tokenizers 0.23.1-1
Package Actions
| Git Clone URL: | https://aur.archlinux.org/python-tokenizers.git (read-only, click to copy) |
|---|---|
| Package Base: | python-tokenizers |
| Description: | Fast State-of-the-Art Tokenizers optimized for Research and Production |
| Upstream URL: | https://github.com/huggingface/tokenizers |
| Keywords: | huggingface |
| Licenses: | Apache-2.0 |
| Submitter: | filipg |
| Maintainer: | xiota (daskol) |
| Last Packager: | daskol |
| Votes: | 16 |
| Popularity: | 1.10 |
| First Submitted: | 2021-10-23 11:17 (UTC) |
| Last Updated: | 2026-04-28 21:24 (UTC) |
Dependencies (14)
- python
- clang (llvm-gitAUR, clang-minimal-gitAUR, clang17-binAUR, clang-static-gitAUR) (make)
- python-build (make)
- python-installer (make)
- python-maturin (python-maturin-gitAUR) (make)
- python-setuptools-rust (make)
- python-wheel (make)
- rust-bindgen (rust-bindgen-gitAUR) (make)
- python-datasets (check)
- python-numpy (python-numpy-gitAUR, python-numpy-mkl-binAUR, python-numpy1AUR, python-numpy-mkl-tbbAUR, python-numpy-mklAUR) (check)
- python-pyarrow (check)
- python-pytest (check)
- python-requests (check)
- python-setuptools-rust (check)
Required by (22)
- comfyui-system
- gaia-amd
- immich-machine-learning
- litellm
- muzaiten-features-clap
- pix2tex
- python-accelerate (check)
- python-auralis
- python-bark-git
- python-chromadb
- python-cohere
- python-crewai
- python-curated-transformers
- python-fastembed
- python-faster-whisper
- python-model2vec
- python-skorch (check)
- python-transformers
- python-transformers-git
- python-wordllama
- Show 2 more...
Sources (3)
xiota commented on 2023-02-14 03:02 (UTC)
groo commented on 2022-12-16 18:09 (UTC)
The package does not compile for me. It fails with following message:
[...output ommitted...] running build_rust cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module -- --crate-type cdylib error: 'json-render-diagnostics' isn't a valid value for '--message-format <FMT>' [possible values: human, json, short]
Did you mean 'json'?
USAGE: cargo rustc --features <FEATURES> --lib --manifest-path <PATH> --message-format <FMT> --release --verbose
For more information try --help
error: cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module -- --crate-type cdylib failed with code 1
==> FEHLER: Ein Fehler geschah in build().
Breche ab...
-> Fehler beim Erstellen: python-tokenizers
filipg commented on 2022-12-10 08:07 (UTC)
It depends on python-setuptools-rust which, in turn, depends on python-setuptools, so I assume it is not needed to add it explicitly.
CorvetteCole commented on 2022-10-06 00:06 (UTC)
This package should depend on python-setuptools
Pinned Comments
xiota commented on 2024-08-30 16:15 (UTC) (edited on 2024-08-30 16:59 (UTC) by xiota)
Problems: