Package Details: apache-spark 3.5.1-1

Git Clone URL: https://aur.archlinux.org/apache-spark.git (read-only, click to copy)
Package Base: apache-spark
Description: A unified analytics engine for large-scale data processing
Upstream URL: http://spark.apache.org
Keywords: spark
Licenses: Apache
Submitter: huitseeker
Maintainer: aakashhemadri
Last Packager: aakashhemadri
Votes: 57
Popularity: 0.014328
First Submitted: 2015-10-04 09:31 (UTC)
Last Updated: 2024-05-07 17:40 (UTC)

Dependencies (2)

Required by (2)

Sources (4)

Latest Comments

« First ‹ Previous 1 2 3 4 5 6 7 Next › Last »

Aetf commented on 2016-02-23 04:43 (UTC)

The the package function should be @@ -55,7 +55,7 @@ package() { mkdir -p $pkgdir/etc/profile.d echo '#!/bin/sh' > $pkgdir/etc/profile.d/apache-spark.sh - echo 'SPARK_HOME=$pkgdir/usr/share/apache-spark' >> $pkgdir/etc/profile.d/apache-spark.sh + echo 'SPARK_HOME=/usr/share/apache-spark' >> $pkgdir/etc/profile.d/apache-spark.sh echo 'export SPARK_HOME' >> $pkgdir/etc/profile.d/apache-spark.sh chmod 755 $pkgdir/etc/profile.d/apache-spark.sh

aftabnack commented on 2016-02-17 07:35 (UTC) (edited on 2016-02-17 07:38 (UTC) by aftabnack)

My compiling is failed at the point of building MQTT, every time its running out of memory. I'm running arch 32-bit in virtual box with 5GB RAM. It failed with outOfMemory error with message saying "malloc failed to allocate 171128 bytes for chunk::new" I have run it about 8times. I tried modifying PKGBUILD and editing MAVEN_OPTS with different values for -Xmx MaxPermSize. I can share the hs_err/replay file.

oneeyed commented on 2016-01-07 18:38 (UTC)

Adding two files in /etc/profile.d (sh and csh) to set SPARK_HOME would definitely help users of this package.

StefanK2 commented on 2016-01-05 14:46 (UTC) (edited on 2016-01-05 15:09 (UTC) by StefanK2)

LOL I found the problem, SPARK_HOME wasn't set ;-) Everything is working now. --- I just updated the package, but now I get this error-message when I call spark-submit or spark-shell: ls: cannot access /usr/assembly/target/scala-2.10: No such file or directory Failed to find Spark assembly in /usr/assembly/target/scala-2.10. You need to build Spark before running this program. According to pacman scala 2.11 is installed on my machine. Also when I checked in apache-spark/src/spark-1.6.0/assembly/target there is only a folder scala-2.11. Did I miss some configuration-step or something? I changed the line 36 in PKGBUILD from dev/change-version-to-2.11.sh to dev/change-scala-version.sh 2.11 to get rid of the deprecation warning (I assumed that my problem is related to the warning, but it didn't change it). Also I just found this: https://issues.apache.org/jira/browse/SPARK-7074 seems to be a spark problem.

huitseeker commented on 2015-12-21 14:44 (UTC)

@fosskers I really can't reproduce your issue. Do you have more info on your config ? Have you thought of clearing your maven cache before rebuilding ?

flying-sheep commented on 2015-11-23 14:16 (UTC)

yes, do this please. 1. it’s compiled from git master, so it should have a -git suffix after the name 3. the version is wrong (1.6.0 isn’t released yet, you’ll have to use some version generated by a pkgver() function) 2. the underscore is against convention (except if the name literally has an underscore in it)

fosskers commented on 2015-11-19 20:59 (UTC)

Have spent the last few hours trying to compile this. At the point of building `spark-streaming-mqtt`I always either run out of RAM (I have 4G on my laptop) or I see this: [INFO] --- maven-assembly-plugin:2.5.5:single (test-jar-with-dependencies) @ spark-streaming-mqtt_2.11 --- [INFO] Reading assembly descriptor: src/main/assembly/assembly.xml [WARNING] The assembly descriptor contains a filesystem-root relative reference, which is not cross platform compatible / [INFO] Building jar: /home/colin/building/apache-spark/src/spark-1.5.2/external/mqtt/target/scala-2.11/spark-streaming-mqtt-test-1.5.2.jar /home/colin/building/apache-spark/src/spark-1.5.2/build/mvn: line 152: 28401 Killed ${MVN_BIN} -DzincPort=${ZINC_PORT} "$@" Any ideas?

mister_karim commented on 2015-11-15 10:28 (UTC)

Thanks for the package, please change the name to apache-spark-git, like that people can understand that you are using always the last git version.

hanzeil commented on 2015-10-29 01:58 (UTC) (edited on 2015-10-29 01:59 (UTC) by hanzeil)

this is not working well for me eroor like this : ++ grep -v INFO ++ tail -n 1 VERSION='[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException' why? Thanks

« First ‹ Previous 1 2 3 4 5 6 7 Next › Last »