Search Criteria
Package Details: hadoop 3.3.6-1
Package Actions
Git Clone URL: | https://aur.archlinux.org/hadoop.git (read-only, click to copy) |
---|---|
Package Base: | hadoop |
Description: | An open-source software for reliable, scalable, distributed computing |
Upstream URL: | https://hadoop.apache.org/ |
Keywords: | apache big-data java map-reduce |
Licenses: | Apache |
Conflicts: | yarn |
Submitter: | sjakub |
Maintainer: | matthewq337 |
Last Packager: | matthewq337 |
Votes: | 85 |
Popularity: | 0.60 |
First Submitted: | 2009-04-07 16:39 (UTC) |
Last Updated: | 2024-09-11 04:45 (UTC) |
Dependencies (10)
- inetutils (inetutils-gitAUR, busybox-coreutilsAUR)
- java-runtime-headless (jre10AUR, jre12AUR, server-jreAUR, jdk10AUR, jdk10-openj9-binAUR, jdk7AUR, jre7AUR, amazon-corretto-16AUR, jdk16-adoptopenjdkAUR, jdk8-armAUR, liberica-jre-11-binAUR, jdk11-j9-binAUR, jre11-jbr-xdg-headlessAUR, jre16-openjdk-headlessAUR, jre14-openjdk-headlessAUR, amazon-corretto-18AUR, jre15AUR, jre14AUR, jre13AUR, jre16AUR, jre18-openjdk-headlessAUR, amazon-corretto-19-binAUR, liberica-jre-11-full-binAUR, jdk13-openjdk-binAUR, liberica-jre-8-full-binAUR, jre-openj9-headlessAUR, jre12-openjdk-headlessAUR, jdk11-dragonwell-standard-binAUR, jdk11-jetbrains-binAUR, amazon-corretto-20-binAUR, zulu-15-binAUR, jdk20-openj9-binAUR, zulu-13-binAUR, jdk8-dragonwell-extended-binAUR, jdk8-dragonwell-standard-binAUR, jdk11-dragonwell-extended-binAUR, jdk17-dragonwell-standard-binAUR, jre11AUR, jdk8-j9-binAUR, jdk7-j9-binAUR, jdk7r1-j9-binAUR, jre13-openjdk-headlessAUR, jre15-openjdk-headlessAUR, jdk8AUR, jre8AUR, jdk-openj9-binAUR, jdk11-openj9-binAUR, jdk8-openj9-binAUR, jre-ltsAUR, jre17AUR, microsoft-openjdk-11-binAUR, microsoft-openjdk-17-binAUR, microsoft-openjdk-21-binAUR, jdk17-openj9-binAUR, liberica-nik-24-full-binAUR, zulu-jre-fx-binAUR, amazon-corretto-17AUR, amazon-corretto-21-binAUR, amazon-corretto-22-binAUR, amazon-corretto-8AUR, amazon-corretto-11AUR, jdk-temurinAUR, jdk21-temurinAUR, jdk17-temurinAUR, jdk11-temurinAUR, zulu-21-binAUR, zulu-17-binAUR, zulu-11-binAUR, zulu-8-binAUR, liberica-jdk-8-full-binAUR, liberica-jdk-21-full-binAUR, zulu-17-fx-binAUR, liberica-jdk-17-full-binAUR, liberica-jdk-11-lite-binAUR, liberica-jdk-11-full-binAUR, liberica-jdk-11-binAUR, jdk8-perfAUR, jre19-openjdk-headlessAUR, zulu8-fx-binAUR, zulu11-fx-binAUR, zulu17-fx-binAUR, zulu21-fx-binAUR, zing-21-binAUR, zing-8-binAUR, jdk21-jetbrains-binAUR, jre-jetbrainsAUR, jre-openjdk-wakefield-headlessAUR, jre-openjdk-wakefieldAUR, jdk-openjdk-wakefieldAUR, jdk17-jetbrains-binAUR, zulu-jdk-fx-binAUR, java-openjdk-binAUR, liberica-jdk-full-binAUR, jdk21-openj9-binAUR, jre17-zulu-binAUR, jreAUR, jdkAUR, jre21-zulu-binAUR, java-openjdk-ea-binAUR, jdk-openjdk, jdk11-openjdk, jdk17-openjdk, jdk21-openjdk, jre-openjdk, jre-openjdk-headless, jre11-openjdk, jre11-openjdk-headless, jre17-openjdk, jre17-openjdk-headless, jre21-openjdk, jre21-openjdk-headless, jre8-openjdk-headless)
- openssh (openssh-gitAUR, openssh-gssapiAUR, openssh-dotconfigAUR, openssh-dotconfig-binAUR, openssh-selinuxAUR, openssh-hpn-shimAUR)
- protobuf (protobuf-gitAUR)
- cmake (cmake-gitAUR) (make)
- gcc12AUR (make)
- java-environment (jdk12AUR, jdk10AUR, jdk10-openj9-binAUR, jdk7AUR, amazon-corretto-16AUR, jdk8-graalvm-binAUR, jdk16-graalvm-binAUR, jdk16-adoptopenjdkAUR, jdk8-armAUR, liberica-jre-11-binAUR, jdk11-j9-binAUR, jdk11-jbr-xdgAUR, jdk16-openjdkAUR, jdk14-openjdkAUR, amazon-corretto-18AUR, jdk18-openjdkAUR, jdk-android-studioAUR, amazon-corretto-19-binAUR, jdk19-graalvm-binAUR, liberica-jre-11-full-binAUR, jdk19-graalvm-ee-binAUR, jdk13-openjdk-binAUR, liberica-jre-8-full-binAUR, jdk11-graalvm-binAUR, jdk-openj9AUR, jdk11-graalvm-ee-binAUR, jdk12-openjdkAUR, jdk11-dragonwell-standard-binAUR, jdk11-jetbrains-binAUR, jdk20-graalvm-binAUR, jdk17-graalvm-binAUR, jdk8-graalvm-ee-binAUR, amazon-corretto-20-binAUR, zulu-15-binAUR, jdk20-openj9-binAUR, zulu-13-binAUR, jdk8-dragonwell-extended-binAUR, jdk8-dragonwell-standard-binAUR, jdk11-dragonwell-extended-binAUR, jdk17-dragonwell-standard-binAUR, jdk11AUR, jdk8-j9-binAUR, jdk7-j9-binAUR, jdk7r1-j9-binAUR, jdk8-dragonwell-extendedAUR, jdk13-openjdkAUR, jdk15-openjdkAUR, jdk21-graalvm-binAUR, jdk8AUR, jdk-openj9-binAUR, jdk11-openj9-binAUR, jdk17-jetbrainsAUR, jdk8-openj9-binAUR, jdk-ltsAUR, microsoft-openjdk-11-binAUR, microsoft-openjdk-17-binAUR, microsoft-openjdk-21-binAUR, jdk17-openj9-binAUR, liberica-nik-24-full-binAUR, jdk21-jetbrains-gitAUR, amazon-corretto-17AUR, amazon-corretto-21-binAUR, amazon-corretto-22-binAUR, amazon-corretto-8AUR, amazon-corretto-11AUR, jdk-temurinAUR, jdk21-temurinAUR, jdk17-temurinAUR, jdk11-temurinAUR, zulu-21-binAUR, zulu-17-binAUR, zulu-11-binAUR, zulu-8-binAUR, liberica-jdk-8-full-binAUR, liberica-jdk-21-full-binAUR, jdk8-temurinAUR, mandrel-binAUR, mandrel24-binAUR, zulu-17-fx-binAUR, liberica-jdk-17-full-binAUR, liberica-jdk-11-lite-binAUR, liberica-jdk-11-full-binAUR, liberica-jdk-11-binAUR, jdk17-graalvm-ee-binAUR, jdk21-graalvm-ee-binAUR, jdk22-graalvm-ee-binAUR, jdk20-graalvm-ee-binAUR, jdk8-perfAUR, jdk22-graalvm-binAUR, jdk19-openjdkAUR, zulu8-fx-binAUR, zulu11-fx-binAUR, zulu17-fx-binAUR, zulu21-fx-binAUR, zing-21-binAUR, zing-8-binAUR, jdk21-jetbrains-binAUR, jre-jetbrainsAUR, jdk-openjdk-wakefieldAUR, jdk17-jetbrains-binAUR, zulu-jdk-fx-binAUR, java-openjdk-binAUR, liberica-jdk-full-binAUR, jabba-binAUR, jdk21-jetbrainsAUR, jdk17-zulu-prime-binAUR, jdk21-openj9-binAUR, jdkAUR, java-openjdk-ea-binAUR, jdk-openjdk, jdk11-openjdk, jdk17-openjdk, jdk21-openjdk, jdk8-openjdk) (make)
- make (make-gitAUR) (make)
- maven (mvnvmAUR) (make)
- pkgconfig (pkgconf-gitAUR, pkg-config-gitAUR, pkgconf) (make)
Required by (3)
- apache-flink (optional)
- swcdb-lib-fs-hadoop-jvm (make)
- swcdb-lib-fs-hadoop-jvm (optional)
Latest Comments
1 2 3 4 5 6 .. 8 Next › Last »
sourcandy commented on 2023-10-11 15:15 (UTC)
@ttc0419 I see that you maintained both Hadoop and Spark packages, but both have different directory structures that makes me go insane for the first few days. Is it possible to move everything centralized to
/opt/apache-hadoop
instead (like most AUR packages)? Is it for historical purpose? Because it doesn't even follow the default Hadoop directory structures (which search for log and config file in/usr
directory) if you run it directly.I also spotted a little inconsistencies between
/etc/profile.d/hadoop.sh
and/etc/conf.d/hadoop
whereJAVA_HOME
is not set on one of them.Other than the Java error, I think
/etc/profile.d/hadoop.sh
should be modified to something like this to make it consistent when running it manually (e.g. as root) vs when running it via hadoop daemons:Also I created this guide in case some of you poor souls have the same problems as me when setting up Hadoop on Arch (since I can't find a good one and the Arch wiki for Hadoop is kinda outdated).
lllf commented on 2023-10-09 10:26 (UTC) (edited on 2023-10-09 10:26 (UTC) by lllf)
Had to put in this grotesque hack into
build()
to get it to build due to multiple errors: 1. C++17 not being set 2. Linker errors 3. Test not compiling 4. Yarn dependency requiring node 14 when node 12 is baked inlllf commented on 2023-10-09 06:41 (UTC)
No longer building due to:
Adding -std=c++14 to CXXFLAGS doesn't seem to help. Please advise.
gnaggnoyil commented on 2023-06-18 18:29 (UTC)
I'm facing the same issue @dom encountered. I think it's exactly the issue described in https://issues.apache.org/jira/browse/MINIFICPP-2116 . And as the upstream issue mentioned, adding
-include cstdint
toCXXFLAGS
should be a working workaround.dom commented on 2023-05-16 03:04 (UTC)
Getting the following failure. Unsure of how to fix.
Full log here
lllf commented on 2022-10-19 03:00 (UTC)
Can line 31 be changed to
It's currently building with java 11 which creates binaries that are incompatible with java 8. Some apache utilities that use hadoop, such as hive, do not work with java 11.
gnaggnoyil commented on 2022-08-27 14:15 (UTC)
@galaxy0419 Confirmed that package can be successfully built after your update. Thanks for your work.
ttc0419 commented on 2022-08-25 07:13 (UTC)
@petronny Thanks for the log, the issue should be resolved
petronny commented on 2022-08-24 18:36 (UTC) (edited on 2022-08-24 18:37 (UTC) by petronny)
The new log with
-X -e
enabled can be downloaded from https://github.com/arch4edu/cactus/actions/runs/2921411343Also, you can reproduce the error yourself by build with
extra-x86_64-build
/ttc0419 commented on 2022-08-24 12:34 (UTC) (edited on 2022-08-24 12:35 (UTC) by ttc0419)
@petronny As the error message said, please use
-X
and-e
options to generate a useful debug log. The log is not very helpful.1 2 3 4 5 6 .. 8 Next › Last »