Arch Linux User Repository

PKGBUILD diff for @envolution

@@ -7,7 +7,7 @@
 # shellcheck shell=bash disable=SC2034,SC2154

 pkgname=apache-spark
-pkgver=4.0.1
+pkgver=4.1.1
 pkgrel=1
 pkgdesc="A unified analytics engine for large-scale data processing"
 arch=('any')
@@ -19,7 +19,7 @@ source=("https://dlcdn.apache.org/spark/spark-$pkgver/spark-$pkgver-bin-hadoop3.
   'apache-spark.sh'
   'apache-spark-master.service'
   'apache-spark-worker@.service')
-sha256sums=('bd5315fa89db737f005971835b94e093c3d2b8581d2411737d281627d6803cc3'
+sha256sums=('f5874d9d6a4935f26bc9f761467971d26812a17d94c6493b8eb4b810e7db31ae'
             '0cc82baad4d878d4e2bc5864a00b99d38f2906781ea47ee6282546788e797049'
             'de54c025ca8ce34a7b4fd95ec7b8d5dec44582787a0bd8da09232f26e2182c9a'
             '47e6c154daecf7631ac9a33fe53a76888070c823c4381fcbde8d98377e586505')

Following The other comments, spark-3.5.4 is not in the https://dlcdn.apache.org/spark/ anymore either.

Oldest version is now spark-3.5.8-bin-hadoop3.tgz you can grab your own sha256sum of this it should be 24c24fec87ab3e1187bba320844f283ceff6036efd8108d3ae17d6911e3f6895.

Someone should update this to be the latest version though, 4.1.1

Following Appuchia's comment, the first sha256sum should be : d8e08877ed428bf9fcd44dbbec8cecadad34785a05513e5020ae74ffdabcbc83

If needed, you can type the following command to update to the right sha256 (if I mistyped the key) : sha256sum spark-3.5.4-bin-hadoop3.tgz

Also change the first sha256sum to 8e08877ed428bf9fcd44dbbec8cecadad34785a05513e5020ae74ffdabcbc83. The sha512 of the file matches the one on the cdn.

This makes the install work

The file version 3.5.1 is no longer available in https://dlcdn.apache.org/spark/ Changing the version in the PKGBUILD to 3.5.4 makes the download work

A proper url is required I assume

==> Making package: apache-spark 3.5.1-1 (Tue 22 Oct 2024 04:52:06 PM EEST)
==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Retrieving sources...
  -> Downloading spark-3.5.1-bin-hadoop3.tgz...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 404
==> ERROR: Failure while downloading https://dlcdn.apache.org/spark/spark-3.5.1/spark-3.5.1-bin-hadoop3.tgz
    Aborting...

diff of the update to the latest + adding python back in

--- PKGBUILD.orig   2023-09-29 07:49:21.000000000 -0600
+++ PKGBUILD    2024-02-29 18:22:53.703306712 -0700
@@ -5,7 +5,7 @@
 # Contributor: Emanuel Fontelles ("emanuelfontelles") <emanuelfontelles@hotmail.com>

 pkgname=apache-spark
-pkgver=3.5.0
+pkgver=3.5.1
 pkgrel=1
 pkgdesc="A unified analytics engine for large-scale data processing"
 arch=('any')
@@ -17,7 +17,7 @@
         'apache-spark.sh'
         'apache-spark-master.service'
         'apache-spark-worker@.service')
-sha256sums=('ac064ec620129b4b9630e180e25ac2b71a8ccd4c6912bf5b5631943b742f0379'
+sha256sums=('5df15f8027067c060fe47ebd351a1431a61dbecc9c28b8dd29e2c6e1935c23eb'
             '0cc82baad4d878d4e2bc5864a00b99d38f2906781ea47ee6282546788e797049'
             'de54c025ca8ce34a7b4fd95ec7b8d5dec44582787a0bd8da09232f26e2182c9a'
             '47e6c154daecf7631ac9a33fe53a76888070c823c4381fcbde8d98377e586505')
@@ -28,7 +28,7 @@
     cd spark-${pkgver}-bin-hadoop3

     # Remove Python and R files
-    rm -rf python R
+    #rm -rf python R

     # Remove windows batch files
     rm bin/*.cmd

Pyspark is also broken for me. Seems this package is supposed to be only for cluster installation which is inconvenient, when one want to work also locally...

Why are the Python and R files removed? This breaks the pyspark command.

prepare() {
    ...
    # Remove Python and R files
    rm -rf python R
}

$ pyspark
Python 3.11.3 (main, Jun  5 2023, 09:32:32) [GCC 13.1.1 20230429] on linux
Type "help", "copyright", "credits" or "license" for more information.
Could not open PYTHONSTARTUP
FileNotFoundError: [Errno 2] No such file or directory: '/opt/apache-spark/python/pyspark/shell.py'
>>> 

@agaskell Updated