Package Details: powerpill 2016.5-1

Git Clone URL: https://aur.archlinux.org/powerpill.git (read-only)
Package Base: powerpill
Description: Pacman wrapper for parallel and segmented downloads.
Upstream URL: http://xyne.archlinux.ca/projects/powerpill
Keywords: arch_linux pacman system
Licenses: GPL
Submitter: Xyne
Maintainer: Xyne
Last Packager: Xyne
Votes: 167
Popularity: 4.666021
First Submitted: 2012-11-29 03:58
Last Updated: 2016-05-24 20:53

Dependencies (9)

Required by (3)

Sources (2)

Latest Comments

zerophase commented on 2016-06-15 20:16

I fixed the issue I was having by updating my rsync servers. I just wrote a script for automating that weekly.

lmat commented on 2016-06-15 20:14

The git update URL gives 404, I guess it's out of date.

lmat commented on 2016-06-15 20:14

Not having any problems, zerophase

zerophase commented on 2016-06-12 00:05

Anyone else having issues with powerpill not updating packages? Pacman and Yaourt still work, but not powerpill.

Xyne commented on 2016-05-24 20:44

@pszynk
Fixed, thanks. Sorry for the delay. I somehow missed the comment notifications.

@computerfr33k
powerpill depends on python3-xcpf. python3-xcpy depends on python3-memoizedb. If you need to install it manually, then something else is broken because it should be picked up during dependency resolution.

computerfr33k commented on 2016-05-02 22:39

Missing dependency: python3-memoizedb

Had to manually install this as pointed out in the previous comments, not sure why this isn't listed as a dependency since it doesn't run without it.

pszynk commented on 2016-04-28 17:04

Hey, there is a bug in clean() method (when calling powerpill like: $ powerpill --powerpill-clean. Lines 910 and 913 with logging. No variable path, only dpath :).

mattlyons commented on 2016-04-21 01:23

@Xyne Using the AUR package, I used yaourt which was configured to download using powerpill. It uses powerpill as a wrapper for pacman, so I assume it made the call to upgrade one package, the powerpill call ended, then it called powerpill to upgrade the next dep, but the dep it just installed broke powerpill until the other dep and powerpill was updated. Either way, got it working by just upgrading configuring yaourt to use pacman temporarily.

lmat commented on 2016-04-21 00:04

powerpill --version shows "Pacman v5.0.1 - libalpm v10.0.1", so I think it's up to date. I installed using your repo. pacman allows me to continue to use your repo, but powerpill seems confused about the keys. Oh yeah, cute picture in pacman --version!

Xyne commented on 2016-04-20 20:15

@lmat
Does that error still occur with the latest version of powerpill?

@mattlyons
All imports are handled when it starts up. After that you can remove or change the modules on disk without affecting the running application because everything is already in RAM.

How did you upgrade powerpill and its deps? Are you using my repo or are you installing them from the AUR? If the former, did you do a partial upgrade or abort an upgrade midway? If the latter, did you upgrade all of the deps before invoking powerpill?

mattlyons commented on 2016-04-20 15:33

Edit: I'm pretty sure this is because I used powerpill to upgrade and updating the deps broke powerpill, which was being used to upgrade. Upgrading with pacman fixes it.

I get this error upgrading to the most recent version:

Traceback (most recent call last):
File "/usr/lib/python3.5/runpy.py", line 170, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/lib/python3.5/site-packages/Powerpill.py", line 36, in <module>
import pm2ml
File "/usr/lib/python3.5/site-packages/pm2ml.py", line 34, in <module>
import XCPF.ArchPkg
ImportError: No module named 'XCPF.ArchPkg'; 'XCPF' is not a package

lmat commented on 2016-04-19 13:58

When I powerpill -Syu, I get the error:

error: xyne-x86_64: signature from "Xyne. (key #3) <xyne@archlinux.ca>" is invalid
error: database 'xyne-x86_64' is not valid (invalid or corrupted database (PGP signature))


But when I run pacman -Syu, no such error is presented.

Xyne commented on 2016-03-29 00:11

@BrianAllred
As Lone_Wolf has already pointed out, the dependency is indirect. python3-xcpf depends on python3-memoizedb and powerpill depends on python3-xcpf. PKGBUILD dependencies are only for direct dependencies. Pacman deals with the dependency resolution. You must have disabled dependency resolution when you installed powerpill, which is a bad idea.

Lone_Wolf commented on 2016-03-27 11:44

BrianAllred , that's incorrect.

powerpill depends on python3-xcpf , and that depends on python3-memoizedb .

File "/usr/lib/python3.5/site-packages/XCPF.py", line 24, in <module>
import MemoizeDB
ImportError: No module named 'MemoizeDB'

The last line of the traceback clearly shows that xcpf calls memoizedb.

BrianAllred commented on 2016-03-27 04:33

@Xyne

Yes, Powerpill does, in fact, directly depend on python3-memoizedb.

sudo powerpill -Su
[sudo] password for brian:
Traceback (most recent call last):
File "/usr/lib/python3.5/runpy.py", line 170, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/lib/python3.5/site-packages/Powerpill.py", line 36, in <module>
import pm2ml
File "/usr/lib/python3.5/site-packages/pm2ml.py", line 33, in <module>
import XCPF
File "/usr/lib/python3.5/site-packages/XCPF.py", line 24, in <module>
import MemoizeDB
ImportError: No module named 'MemoizeDB'

After running 'pacaur -S python3-memoizedb', everything is fine.

Xyne commented on 2016-03-12 12:20

Line 144 in the current version of XCPF.py is "mdb.db_initialize()". Please update to the latest versions before reporting errors. Sometimes they have already been fixed, and even if not it makes it easier to find the source of the error in the code if we're looking at the same lines.

XCGF (where that line is currently located) implements a class to do the equivalent of a "pushd" followed by a "popd" using the "with" keyword to ensure that the current working directory is left as-is for subsequent commands if necessary, instead of a simple "chdir". Powerpill itself isn't trying to do anything in that directory, but the Powerpill.py module is used by other code that expects the working directory to remain unchanged after the download.

When I have the time I will likely restructure the code where it's not necessary to return to the previous working directory. In the meantime just run it in a different directory. In general running things in non-existing directories tends to generate errors due to various sanity checks even if there are no side effects.

lmat commented on 2016-03-11 13:53

When I run sudo powerpill -Syu; in a directory that root can't touch (or a directory that doesn't exist, etc.), I get the following output:
Traceback (most recent call last):

Traceback (most recent call last):
File "/usr/bin/powerpill", line 5, in <module>
sys.exit(run_main())
File "/usr/lib/python3.5/site-packages/Powerpill.py", line 865, in run_main
return main(args)
File "/usr/lib/python3.5/site-packages/Powerpill.py", line 852, in main
refresh_databases(powerpill)
File "/usr/lib/python3.5/site-packages/Powerpill.py", line 810, in refresh_databases
powerpill.download(pm2ml_args, dbs=True, force=(powerpill.pargs['y'] > 1))
File "/usr/lib/python3.5/site-packages/Powerpill.py", line 703, in download
aria2c_p.communicate(input=metalink)
File "/usr/lib/python3.5/site-packages/XCPF.py", line 144, in __exit__
os.chdir(self.orig)
PermissionError: [Errno 13] Permission denied: '/home/lawsa/reformable/Budget/repo'


and it fails. What is it trying to do in the current directory? Actually, I don't care what it's trying to do; I would think it should be unconcerned with the current directory, right?

Xyne commented on 2016-03-10 00:13

@afrenchguy
Powerpill does not (directly) depend on python3-memoizedb.

afrenchguy commented on 2016-03-09 13:54

please add python3-memoizedb to the depencies

Xyne commented on 2016-02-14 19:29

Powerpill didn't recognize the new(?) "Never" SigLevel. This has been fixed in the last update. For for information, see https://bbs.archlinux.org/viewtopic.php?pid=1604371#p1604371

zerophase commented on 2016-02-09 06:33

Since Pacman 5.0 was released whenever I run powerpill -Syu I receive download errors for extra.db.sig, core.db.sig, multilib.db.sig, and community.db.sig. Is this just since those sig files still need to be updated on the download server?

Xyne commented on 2016-01-29 19:26

I have replied about the download speed issue on the forum: https://bbs.archlinux.org/viewtopic.php?pid=1599874#p1599874

j1simon commented on 2016-01-29 10:32

Impossible to download: 'curl: (56) Recv failure: Connection reset by peer'
I've tried several times but it doesn't download.

Xyne commented on 2016-01-14 01:36

@gusdunn
Please double-check that you have the latest version of all deps. pm2ml does recognize --select. If the error persists, please pastebin or email me the full output of the error message.

The easiest way to check that you have everything up-to-date is to add my repo to pacman.conf.

gusdunn commented on 2016-01-13 16:55

Similar to @lmat I recently updated the powerpill stack but now get the complaint that pm2ml doesnt recognized '--select' option. Altering the powerpill.json file to omit the powerpill option ("select": true) does not correct the problem. But setting the option to ("select": false) does allow execution.

lmat commented on 2015-12-22 21:42

Thank you, my friend! And I will perhaps take a look at bauerbill some day, too.

Xyne commented on 2015-12-06 15:58

Bauerbill is back btw: https://aur.archlinux.org/packages/bauerbill/

Xyne commented on 2015-12-06 01:19

It should be fixed now.

lmat commented on 2015-12-02 20:46

just downloaded and installed python3-xcpf, pm2ml, and powerpill today, and running powerpill -Syu; returns "pm2ml: error: unrecognized arguments: --ask". I see pm2ml was updated a couple weeks ago; perhaps its parameters changed?

avb commented on 2015-11-30 19:41

@Xyne: It's working for me now, thank you.

Xyne commented on 2015-11-30 16:39

@avb, TwoLeaves
It should be fixed now.

TwoLeaves commented on 2015-11-23 11:57

Getting the same behaviour as avb.

avb commented on 2015-11-22 19:33

powerpill's having trouble syncing when there's nothing to update. I just ran powerpill -Syyu and it worked fine. When I run it again, I get:
$ sudo powerpill -Sy
receiving incremental file list
error: no targets specified (use -h for help)

pacman is happy:
sudo pacman -Sy
:: Synchronizing package databases...
core is up to date
extra is up to date
community is up to date
multilib is up to date

despairblue commented on 2015-10-16 07:09

same thing

disneyvillain commented on 2015-10-07 10:51

Broken with latest update - exact same error as txtsd.

Edit: Issue fixed after a pm2ml was updated. All is well now.

txtsd commented on 2015-10-07 06:20

Updated powerpill and this happens when I try to run it now.

https://bpaste.net/show/5316f9ec608a

jyantis commented on 2015-09-16 04:02

To install this you need to add to add Xyne's key.
gpg --keyserver pgp.mit.edu --recv-keys 1D1F0DC78F173680

Xyne commented on 2015-08-10 23:09

You need to import public keys for any source file signatures that you want to verify. See the comments on the python3-aur page for details. The AUR is full of such comments. Please search next time.

https://aur.archlinux.org/packages/python3-aur/

step-2 commented on 2015-08-08 12:39

==> Verifying source file signatures with gpg...
powerpill-2015.tar.xz ... FAILED (unknown public key 1D1F0DC78F173680)

couldn't get the update ??

SupBrah commented on 2015-08-02 02:38

Oops, flagged out of date by accident.

Xyne commented on 2015-04-29 22:58

@cyisfor
The official databases are not signed. I agree that they should be but doing that is not trivial due to the chain of custody and the way they are released.

You can avoid the redundant checks by correctly configuring your pacman.conf file. See the powerpill forum thread for details.

cyisfor commented on 2015-04-29 17:32

Oh, no when it DOES find the *.db.sig files, it's that just one of the mirrors is a bad webserver. It returns a 200 OK HTML document saying "File not found" in german. (I think). Then the signatures don't verify, since they're HTML files, and you can't upgrade.

cyisfor commented on 2015-04-29 17:29

Download aborted, *.db.sig, Resource not found (10 min later, same thing...)

Don't any of the pacman mirrors have the signatures for the current databases? It seems like a thing they ought to have. Are all the mirrors just trying to shave off that 2K by omitting the files, or are perhaps the signatures are stored under a different filename?

It finds them... eventually... perhaps the signatures are ONLY stored on the main mirror, and checking the others is an exercise in futility?

At any rate, it makes powerpill mostly unusable for upgrades, if it's going to spend a half hour checking every single mirror for signature files that they just don't have.

Xyne commented on 2014-08-17 17:30

Please read the following post concerning recent tweaks to the code and required user intervention:
https://bbs.archlinux.org/viewtopic.php?pid=1447763#p1447763

kozaki commented on 2014-04-17 07:27

All is back to normal after rebuilt pm2ml and powerpill.
Thank you for the answre Xyne.

adambot commented on 2014-04-08 21:13

nevermind... i confused --log-level with --console-log-level

adambot commented on 2014-04-08 20:59

powerpill seems to be ignoring this part of the aria2c config:
"--log-level=error",

I am constantly getting messages this on the console:
04/08 15:49:59 [NOTICE] Verification finished successfully. file=/var/cache/pacman/pkg/bash-4.3.008-2-x86_64.pkg.tar.xz

04/08 15:49:59 [NOTICE] Download complete: /var/cache/pacman/pkg/bash-4.3.008-2-x86_64.pkg.tar.xz

04/08 15:49:59 [NOTICE] Verification finished successfully. file=/var/cache/pacman/pkg/readline-6.3.003-2-x86_64.pkg.tar.xz

04/08 15:49:59 [NOTICE] Download complete: /var/cache/pacman/pkg/readline-6.3.003-2-x86_64.pkg.tar.xz

matersci commented on 2014-04-06 21:42

Maybe next time try to make it clear by just saying it. Not everybody knows in which function exactly is every dependency needed, thats why a said about -Ss.

Anyway, fixed

Xyne commented on 2014-04-06 12:59

You need to rebuild the entire dependency hierarchy (pm2ml, etc.) against the new version of Python, not just powerpill. I don't know how to make this any clearer.

matersci commented on 2014-04-05 13:42

Same problem here, can't even -Ss. I have already rebuild powerpill

Xyne commented on 2014-04-05 13:19

@kozaki
You need to rebuild all of your Python 3 AUR packages against the new version of Python in the official repos.

kozaki commented on 2014-04-05 09:20

Hi, with pacman it's fine, while with yaourt -{S,Q} I run into the same error described by kmicu.
--
pacman 4.1.2-5
powerpill 2013.9.19-2
python 3.4.0-2
yaourt 1.3-1

butler360 commented on 2014-03-27 19:39

I am running into these errors now without using testing, so it seems python has been updated.

Xyne commented on 2014-03-24 20:54

@kmicu
Sorry, your intention wasn't clear from your post. It seemed that you were reporting what you thought was an error with the current package.

Thanks for the warning about the upcoming error. I'll try to deal with it quickly when Python 3.4 is officially released.

kmicu commented on 2014-03-24 14:07

"Until then, if you wish to use [testing] then it is your responsibility to rebuild the necessary packages yourself." Yes, I am aware of that.
The point of [testing] repo and therefore my "reposnsibility" is to inform about possible issues and that was a reason behind my comment.
I did not flag your package as outdated, but only inform you and users about possible future problems. Nothing more.

Xyne commented on 2014-03-23 16:43

AUR packages (and those in my repos) work with packages in the official repos. When Python 3.4 is officially released by Arch then I will update my Python 3 packages. Until then, if you wish to use [testing] then it is your responsibility to rebuild the necessary packages yourself. All of the PKGBUILDs are available and there is no compilation required so this is relatively quick and easy to do.

kmicu commented on 2014-03-22 18:29

After upgrade to python 3.4 from [testing] repo we have:

> powerpill
Traceback (most recent call last):
File "/usr/bin/powerpill", line 41, in <module>
import pm2ml
ImportError: No module named 'pm2ml'

After downgrading to 3.3.5 everything is ok:
warning: downgrading package pyalpm (0.6.2-2 => 0.6.2-1)
:: pyalpm: requires python>=3.4
warning: downgrading package python (3.4.0-2 => 3.3.5-1)

Additional info:
- Python 3.4.0 upgrade requires rebuild for any packages that linking to libpython3.3m.so or put files in usr/lib/python3.3"
- Please also take the time to check if your package still mention "python-distribute" anywhere, and change it to "python-setuptools".
https://www.archlinux.org/todo/python-34-rebuild/

jtaylor991 commented on 2013-10-27 04:22

Why is this relying on an outdated version of pm2ml? I don't know how to get the old version. Update please??

frederik commented on 2013-10-02 14:18

made a skript that automatically updates the rsync servers in the powerpill json file and the mirrorlist:

#!/bin/sh

reflector --verbose --country 'Germany' --sort rate --save /etc/pacman.d/mirrorlist

# allow global read access (required for non-root yaourt execution)
chmod +r /etc/pacman.d/mirrorlist

cp /etc/powerpill/powerpill.json.sed /etc/powerpill/powerpill.json

reflector -p rsync --country 'Germany' --sort rate | sed 's/Server = //' | sed '1,7d' | sed '/^$/d' | awk '{printf("\"%s\",\n\t\t", $0);}' | sed '$d' | sed '$s/,/ /' | sed '1s/\t\t//' | sed '1s/^/ "servers": [/' >> /etc/powerpill/powerpill.json
echo " ]" >> /etc/powerpill/powerpill.json
echo " }" >> /etc/powerpill/powerpill.json
echo "}" >> /etc/powerpill/powerpill.json

chmod 644 /etc/powerpill/powerpill.json

# reload DB
yaourt -Syy

###########################################################################
You'll need a powerpill.json.sed that misses the last lines:
###########################################################################
{
"aria2": {
"args": [
"--allow-overwrite=true",
"--always-resume=false",
"--auto-file-renaming=false",
"--check-integrity=true",
"--conditional-get=true",
"--continue=true",
"--file-allocation=none",
"--log-level=error",
"--max-concurrent-downloads=100",
"--max-connection-per-server=5",
"--min-split-size=5M",
"--show-console-readout=true",
"--split=10"
],
"path": "/usr/bin/aria2c"
},
"pacman": {
"config": "/etc/pacman.conf",
"path": "/usr/bin/pacman"
},
"pacserve": {
"server": "http://localhost:15678"
},
"powerpill": {
"ask": true,
"reflect databases": false
},
"reflector": {
"args.unused": [
"--protocol",
"http",
"--latest",
"50"
]
},
"rsync": {
"args": [
"--no-motd",
"--progress"
],
"db only": true,
"path": "/usr/bin/rsync",

frederik commented on 2013-10-02 14:18

made a skript that automatically updates the rsync servers in the powerpill json file and the mirrorlist:

#!/bin/sh

reflector --verbose --country 'Germany' --sort rate --save /etc/pacman.d/mirrorlist

# allow global read access (required for non-root yaourt execution)
chmod +r /etc/pacman.d/mirrorlist

cp /etc/powerpill/powerpill.json.sed /etc/powerpill/powerpill.json

reflector -p rsync --country 'Germany' --sort rate | sed 's/Server = //' | sed '1,7d' | sed '/^$/d' | awk '{printf("\"%s\",\n\t\t", $0);}' | sed '$d' | sed '$s/,/ /' | sed '1s/\t\t//' | sed '1s/^/ "servers": [/' >> /etc/powerpill/powerpill.json
echo " ]" >> /etc/powerpill/powerpill.json
echo " }" >> /etc/powerpill/powerpill.json
echo "}" >> /etc/powerpill/powerpill.json

chmod 644 /etc/powerpill/powerpill.json

# reload DB
yaourt -Syy


You'll need a powerpill.json.sed that misses the last lines:

{
"aria2": {
"args": [
"--allow-overwrite=true",
"--always-resume=false",
"--auto-file-renaming=false",
"--check-integrity=true",
"--conditional-get=true",
"--continue=true",
"--file-allocation=none",
"--log-level=error",
"--max-concurrent-downloads=100",
"--max-connection-per-server=5",
"--min-split-size=5M",
"--show-console-readout=true",
"--split=10"
],
"path": "/usr/bin/aria2c"
},
"pacman": {
"config": "/etc/pacman.conf",
"path": "/usr/bin/pacman"
},
"pacserve": {
"server": "http://localhost:15678"
},
"powerpill": {
"ask": true,
"reflect databases": false
},
"reflector": {
"args.unused": [
"--protocol",
"http",
"--latest",
"50"
]
},
"rsync": {
"args": [
"--no-motd",
"--progress"
],
"db only": true,
"path": "/usr/bin/rsync",

Xyne commented on 2013-07-06 11:54

Some servers do not correctly handle HEAD requests and/or conditional GET requests. Without knowing if "powerpill -Syy" would have worked or if "pacman -Sy" would have failed, I can't tell if it is a powerpill error or a server error, but I expect that it is the latter. If it happens again, try "powerpill -Syy", then switch to a different mirror.

If it happens with other mirrors, pastebin your pacman and powerpill configuration files (including the mirrorlist) so that I can (try to) :determine the cause of the problem.

idupree commented on 2013-07-05 23:30

Powerpill corrupted my sync DBs somehow (fixable by `sudo pacman -Syy`). I installed powerpill today and did `sudo powerpill -Syuw`, then I tried to double-check with pacman:

% sudo pacman -Syuw
error: GPGME error: No data
error: GPGME error: No data
error: GPGME error: No data
error: GPGME error: No data
:: Synchronizing package databases...
core is up to date
extra is up to date
community is up to date
multilib is up to date
error: database 'core' is not valid (invalid or corrupted database (PGP signature))
error: database 'extra' is not valid (invalid or corrupted database (PGP signature))
error: database 'community' is not valid (invalid or corrupted database (PGP signature))
error: database 'multilib' is not valid (invalid or corrupted database (PGP signature))

Did I do something wrong?

Xyne commented on 2013-05-10 20:18

2013.5.10 supports the new Python 3 version of Pacserve and should be noticeably faster now due to both more efficient pacserve querying and better pacserve peer detection.

Xyne commented on 2013-04-29 21:24

It looks like a pyalpm error. Try re-installing pyalpm (and maybe pacman).

beanaroo commented on 2013-04-27 10:40

ImportError: libalpm.so.7: cannot open shared object file: No such file or directory

I assume this is related to the release of Pacman 4.1? Not sure.

Xyne commented on 2013-01-29 23:50

@atyoung
pacman -Si python
Name : python
...
Provides : python3

I prefer explicitly versioned Python dependencies. It's future-proof. For the same reason, all of my Python 3 packages are named python3-* instead of python-*.

Using a naming and dependency scheme that requires massive renaming when upstream changes is silly when there is a perfectly logical way to avoid it without any issues whatsoever.

Anonymous comment on 2013-01-29 16:05

Dependency needs to be python not python3. python2 has the number appended to the package name, python 3 is just python.

Xyne commented on 2012-12-12 14:54

@ranger
Thanks, I have added rsync as an optional dependency.

@oboedad55
Please try again with the current PKGBUILD.

ranger commented on 2012-12-12 12:51

rsync shouldn't be a dependency?

oboedad55 commented on 2012-12-12 03:48

powerpill-2012.12.12.1.tar.xz 404 Not Found

Xyne commented on 2012-11-29 21:55

*cof* fixed *cof*

Jristz commented on 2012-11-29 07:36

md5sums=('2ba9dff6dfdcc090e6606ad09275c9ea'
'142942954194f508991e25720c87a694')

*cof* wrong md5s and sha256 *cof*

Jristz commented on 2012-11-29 04:39

this work again, work, THANKS