Package Details: python-vllm 0.12.0-1

Git Clone URL: https://aur.archlinux.org/python-vllm.git (read-only, click to copy)
Package Base: python-vllm
Description: high-throughput and memory-efficient inference and serving engine for LLMs
Upstream URL: https://github.com/vllm-project/vllm
Licenses: Apache-2.0
Submitter: envolution
Maintainer: envolution
Last Packager: envolution
Votes: 2
Popularity: 0.067775
First Submitted: 2024-12-01 16:07 (UTC)
Last Updated: 2025-12-03 13:53 (UTC)

Dependencies (56)

Required by (2)

Sources (1)

Pinned Comments

envolution commented on 2025-08-04 01:31 (UTC)

@nipsky looks like some new incompatibility with python 3.13 - I was able to reproduce. Unfortunately the only feasible method at the moment is to run in a virtualenv using python 3.9-3.12. Upstream is working on 3.13 support but it's not quite there yet.

To be honest, we'll probably be on 3.14 by the time they support 3.13. I'll try have a look to see if I can patch it to initialize, but this week is kind of busy for me so it wouldn't be quick.

Latest Comments

1 2 Next › Last »

Clemy commented on 2025-09-22 18:47 (UTC) (edited on 2025-09-22 18:53 (UTC) by Clemy)

inhale it needs openai_harmony module. just use pip guys or well wait till thy fix the dependencie. ORRR you can compile the openai_harmony yourself if it's possible.

davispuh commented on 2025-08-17 20:02 (UTC)

Need to add python-cbor2 dependency.

eback (most recent call last):
  File "/bin/vllm", line 5, in <module>
    from vllm.entrypoints.cli.main import main
  File "/usr/lib/python3.13/site-packages/vllm/entrypoints/cli/__init__.py", line 3, in <module>
    from vllm.entrypoints.cli.benchmark.latency import BenchmarkLatencySubcommand
  File "/usr/lib/python3.13/site-packages/vllm/entrypoints/cli/benchmark/latency.py", line 5, in <module>
    from vllm.benchmarks.latency import add_cli_args, main
  File "/usr/lib/python3.13/site-packages/vllm/benchmarks/latency.py", line 16, in <module>
    from vllm import LLM, SamplingParams
  File "/usr/lib/python3.13/site-packages/vllm/__init__.py", line 64, in __getattr__
    module = import_module(module_name, __package__)
  File "/usr/lib/python3.13/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/vllm/entrypoints/llm.py", line 17, in <module>
    from vllm.beam_search import (BeamSearchInstance, BeamSearchOutput,
                                  BeamSearchSequence,
                                  create_sort_beams_key_function)
  File "/usr/lib/python3.13/site-packages/vllm/beam_search.py", line 8, in <module>
    from vllm.sequence import Logprob
  File "/usr/lib/python3.13/site-packages/vllm/sequence.py", line 18, in <module>
    from vllm.inputs import SingletonInputs
  File "/usr/lib/python3.13/site-packages/vllm/inputs/__init__.py", line 9, in <module>
    from .registry import (DummyData, InputContext, InputProcessingContext,
                           InputRegistry)
  File "/usr/lib/python3.13/site-packages/vllm/inputs/registry.py", line 14, in <module>
    from vllm.utils import resolve_mm_processor_kwargs
  File "/usr/lib/python3.13/site-packages/vllm/utils/__init__.py", line 55, in <module>
    import cbor2
ModuleNotFoundError: No module named 'cbor2'

And yep currently borked, need this PR https://github.com/vllm-project/vllm/pull/13164 it was recently approved so I think in next release it will work :)

envolution commented on 2025-08-04 16:22 (UTC)

@nipsky i was suggesting you use pip/conda/etc (from within your venv) to install as it exists on pypi - https://pypi.org/project/vllm/

installing this package will only try build it for current system python

nipsky commented on 2025-08-04 12:11 (UTC)

@envolution Thanks for the tip, I’ve tried this with a couple of older Python versions (3.9 to 3.12) within fresh Conda environments but got the exact same error for each. Or do I need to rebuild the package with Python <3.13 somehow?

envolution commented on 2025-08-04 01:31 (UTC)

@nipsky looks like some new incompatibility with python 3.13 - I was able to reproduce. Unfortunately the only feasible method at the moment is to run in a virtualenv using python 3.9-3.12. Upstream is working on 3.13 support but it's not quite there yet.

To be honest, we'll probably be on 3.14 by the time they support 3.13. I'll try have a look to see if I can patch it to initialize, but this week is kind of busy for me so it wouldn't be quick.

nipsky commented on 2025-08-04 00:09 (UTC) (edited on 2025-08-04 00:11 (UTC) by nipsky)

Installation works fine, but running vllm gives me this error:


INFO 08-04 02:06:53 [__init__.py:235] Automatically detected platform cpu.
Traceback (most recent call last):
  File "/usr/lib/python3.13/inspect.py", line 1087, in findsource
    lnum = vars(object)['__firstlineno__'] - 1
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
KeyError: '__firstlineno__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/bin/vllm", line 8, in <module>
    sys.exit(main())
             ~~~~^^
  File "/usr/lib/python3.13/site-packages/vllm/entrypoints/cli/main.py", line 46, in main
    cmd.subparser_init(subparsers).set_defaults(
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/vllm/entrypoints/cli/serve.py", line 66, in subparser_init
    serve_parser = make_arg_parser(serve_parser)
  File "/usr/lib/python3.13/site-packages/vllm/entrypoints/openai/cli_args.py", line 235, in make_arg_parser
    parser = FrontendArgs.add_cli_args(parser)
  File "/usr/lib/python3.13/site-packages/vllm/entrypoints/openai/cli_args.py", line 168, in add_cli_args
    frontend_kwargs = get_kwargs(FrontendArgs)
  File "/usr/lib/python3.13/site-packages/vllm/engine/arg_utils.py", line 261, in get_kwargs
    return copy.deepcopy(_compute_kwargs(cls))
                         ~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/site-packages/vllm/engine/arg_utils.py", line 148, in _compute_kwargs
    cls_docs = get_attr_docs(cls)
  File "/usr/lib/python3.13/site-packages/vllm/config.py", line 150, in get_attr_docs
    cls_node = ast.parse(textwrap.dedent(inspect.getsource(cls))).body[0]
                                         ~~~~~~~~~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/inspect.py", line 1258, in getsource
    lines, lnum = getsourcelines(object)
                  ~~~~~~~~~~~~~~^^^^^^^^
  File "/usr/lib/python3.13/inspect.py", line 1240, in getsourcelines
    lines, lnum = findsource(object)
                  ~~~~~~~~~~^^^^^^^^
  File "/usr/lib/python3.13/inspect.py", line 1089, in findsource
    raise OSError('source code not available')
OSError: source code not available

jimreynold2nd commented on 2025-06-04 20:17 (UTC)

For CPU, this is missing certain dependencies (all of which can be found either in the official repo or AUR):

python-prometheus-fastapi-instrumentator python-python-multipart python-partial-json-parser python-watchfiles python-gguf python-triton python-cloudpickle

Would be great if those can be added to the dependency list.

envolution commented on 2025-03-28 16:30 (UTC)

@amstan thanks for the report, there were a handful of dependencies not caught by namcap. FYI this package targets CPU, it's possible it detects and links for rocm correctly, but ideally a separate package like python-vllm-rocm (similarly to python-vllm-cuda i used to maintain) should be created due to lack of flexibility in the pacman subsystems (we can't elegantly detect and modify dependencies based on GPU architectures/packages installed on the host system).

If this new pkgrel works on rocm, please add a comment for others to become aware.

amstan commented on 2025-03-28 06:07 (UTC)

Compiles to the end, but then:

% python -c "import vllm"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
    import vllm
  File "/usr/lib/python3.13/site-packages/vllm/__init__.py", line 11, in <module>
    from vllm.engine.arg_utils import AsyncEngineArgs, EngineArgs
  File "/usr/lib/python3.13/site-packages/vllm/engine/arg_utils.py", line 15, in <module>
    from vllm.config import (CacheConfig, CompilationConfig, ConfigFormat,
    ...<5 lines>...
                             TokenizerPoolConfig, VllmConfig)
  File "/usr/lib/python3.13/site-packages/vllm/config.py", line 24, in <module>
    from transformers import PretrainedConfig
ModuleNotFoundError: No module named 'transformers'

amstan commented on 2025-03-28 05:42 (UTC) (edited on 2025-03-28 06:02 (UTC) by amstan)

If you get some weird compilation error(/lib/cmake/hip/hip-targets.cmake not found) and you want to do rocm, make sure you don't have hip-runtime-nvidia installed.

1 2 Next › Last »