Arch Linux User Repository

There has been a recent influx of users reporting that they are encountering an error related to missing libcrypt.so.1:

error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory

If you encounter this error, the issue is that your shell has not imported /etc/profile.d/google-cloud-sdk.sh, causing the CLOUDSDK_PYTHON environment variable (among others) to not be set. This causes the tool to use a bundled python executable with its own library searching logic which fails to find the correct library.

libcrypt is provided by core/libxcrypt, which is required by core/python, which is required by aur/google-cloud-sdk. If you've installed this package, you have the correct library.

This is not a bug with the package; it is a bug with your shell. At the moment, this appears to be limited to users of fish. Please check the wiki for information about the tasks you should perform when setting fish as your default shell to avoid errors like this.

PLEASE NOTE

This PKGBUILD is maintained on GitHub. Please use the following repository to submit patches, ask questions, and raise issues:

https://github.com/sudoforge/pkgbuilds

IMPORTANT NOTE!

Commit e01da1e449882ad8e4fe590b01cd09624b58143a removes the appengine components from this package. These components, along with all of the others, will be made available as independent packages in the future.

If you need any of the additional components for your day-to-day workflow, set "disable_updater": false in /opt/google-cloud-sdk/lib/googlecloudsdk/core/config.json and manage components using gcloud components <command>.

Be sure to install any dependencies for the additional components that you may need.

385.0.0-2 works, thanks for update.

385.0.0-2 resolved the issue. If you are still experiencing a build failure, you are not building 385.0.0-2.

Same error here for the latest version:

==> Starting prepare()...
failed to apply patch: 0004-collections-abc.patch
==> ERROR: A failure occurred in prepare().
    Aborting...
 -> error making: google-cloud-sdk

Latest version fails to apply the collections patch.

failed to apply patch: 0004-collections-abc.patch
==> ERROR: A failure occurred in prepare().

There has been a recent influx of users reporting that they are encountering an error related to missing libcrypt.so.1:

error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory

If you encounter this error, the issue is that your shell has not imported /etc/profile.d/google-cloud-sdk.sh, causing the CLOUDSDK_PYTHON environment variable (among others) to not be set. This causes the tool to use a bundled python executable with its own library searching logic which fails to find the correct library.

libcrypt is provided by core/libxcrypt, which is required by core/python, which is required by aur/google-cloud-sdk. If you've installed this package, you have the correct library.

This is not a bug with the package; it is a bug with your shell. At the moment, this appears to be limited to users of fish. Please check the wiki for information about the tasks you should perform when setting fish as your default shell to avoid errors like this.

Note: I overhauled my internal CI process that updates packages and syncs them out to external destinations (like the AUR). Please open an issue using the issue tracker if you encounter any issues.

@sudoforge @ginjiruu I received the same error in bash and fixed by installing https://archlinux.org/packages/core/x86_64/libxcrypt-compat/ as described here https://bbs.archlinux.org/viewtopic.php?pid=2022246#p2022246 (but now in community rather than aur) due to dropping libcrypt.so.1 from glibc.

@sudoforge export CLOUDSDK_PYTHON=$(which python3) also fixed it for me on zsh. Thanks!

To those curious: ginjiruu was using fish, which apparently doesn't source /etc/profile, causing the CLOUDSDK_PYTHON variable to be unset (among other important variables). Specifically, when CLOUDSDK_PYTHON is unset, gcloud uses a vendored python3 executable, which wasn't linked appropriately.

I will likely remove this bundled python in future releases.

@ginjiruu Please open an issue using the issue tracker, and I'll follow up with you today to debug this issue specific to your environments.

@sudoforge I don't believe so. Was able to build version 371 just fine with the same shell config. First system tried was a local arch installation and the other was a wsl2 install that also succeeded on version 371 and is also having issues now

@ginjiruu This does not occur in a clean chroot, nor on my local system (outside of a chroot). Are you building this in a python virtual environment, or do you otherwise have a python virtual environment active in the shell session you are attempting to build it in?

Edit: And to be clear, you are encountering this error when building the package on your systems (as opposed to encountering it when running commands), correct?

Version 372.0.0-1 is giving me the error

/opt/google-cloud-sdk/platform/bundledpythonunix/bin/python3: error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory

despite having the various ssl packages installed. Repeated on 2 separate computers.

@rsaxena correct, 371.0.0-4 updates the patch to cover a new use of collections.Mapping that was introduced in 370.0.0 and missed (because my pipeline does not test every possible command; although this is something I'm aiming to resolve in the upcoming weeks).

Somewhat hilariously, 370.0.0 actually introduced the removal of a workaround to support the collections -> collections.abc change for newer versions of Python.

➜ diff {369,370}/platform/bq/bigquery_client.py | head -10
--- 369/platform/bq/bigquery_client.py    1980-01-01 01:00:00.000000000 -0700
+++ 370/platform/bq/bigquery_client.py    1980-01-01 01:00:00.000000000 -0700
@@ -58,9 +58,6 @@

 _GCS_SCHEME_PREFIX = 'gs://'

-collections_abc = collections
-if sys.version_info > (3, 8):
-  collections_abc = collections.abc

With the new patch (v. 371.0.0-4) it is working for me, Python 3.10.2

The current version is failing with Python 3.10.1:

└─[$] bq
Traceback (most recent call last):
  File "/opt/google-cloud-sdk/platform/bq/bq.py", line 63, in <module>
    import bigquery_client
  File "/opt/google-cloud-sdk/platform/bq/bigquery_client.py", line 6755, in <module>
    class ApiClientHelper(object):
  File "/opt/google-cloud-sdk/platform/bq/bigquery_client.py", line 6761, in ApiClientHelper
    class Reference(collections.Mapping):
AttributeError: module 'collections' has no attribute 'Mapping'
└─[$] python -V
Python 3.10.1

I will investigate more later and provide details if I find the reason.

@Jont828, please see the pinned comments.

I'm in the process of finishing the PKGBUILDs for each external component. I'd suggest following the instructions in the pinned comments until they are finalized and published.

Is it possible to get gcloud's cloud-build-local package on the AUR? I installed the Google Cloud SDK from here, but this tutorial requires installing cloud-build-local, which I could not find on the AUR. It can be installed through gcloud using gcloud components install cloud-build-local, but that command doesn't work when it's installed through the AUR since gcloud sees a separate package manager.

@akhil: I have no idea what Yay does, but the package most definitely installs the shell inclusion files under /opt. Please file issues related to Yay at its project page.

PLEASE NOTE

This PKGBUILD is maintained on GitHub. Please use the following repository to submit patches, ask questions, and raise issues:

https://github.com/sudoforge/pkgbuilds

@rachejazz: comments from 2022-01-13 are obsolete. The issue was fixed later that day.

PLEASE NOTE

This PKGBUILD is maintained on GitHub. Please use the following repository to submit patches, ask questions, and raise issues:

https://github.com/sudoforge/pkgbuilds

@myyc do you mean to say dev-app-server and endpoints-cfg patches needn't be patched since it was already patched?

edit: i was lazy and only checked the first two files. as per the comment below this one actually only the first two files had been patched already. deleting the first two files from the diff and changing the checksum works.

original comment:

it seems like the patch that's failing has been applied already, i've checked a couple of the affected files and it seems to be the case.

i am not the maintainer of the package so i don't know the risk in removing it, but doing so installs the sdk.

I've removed the >/dev/null and get the following conrete error messages:

==> Starting prepare()...
patching file bin/dev_appserver.py
patching file bin/endpointscfg.py
patching file completion.zsh.inc
patching file lib/googlecloudsdk/api_lib/run/k8s_object.py
Reversed (or previously applied) patch detected!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file lib/googlecloudsdk/api_lib/run/k8s_object.py.rej
patching file lib/googlecloudsdk/api_lib/run/traffic.py
Reversed (or previously applied) patch detected!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file lib/googlecloudsdk /api_lib/run/traffic.py.rej
patching file lib/third_party/dns/namedict.py
patching file lib/third_party/frozendict/__init__.py
patching file lib/third_party/functools32/functools32.py
patching file lib/third_party/ml_sdk/cloud/ml/prediction/prediction_utils.py
patching file lib/third_party/prompt_toolkit/styles/from_dict.py
patching file platform/gsutil/gslib/vendored/boto/boto/dynamodb/types.py
patching file platform/gsutil/gslib/vendored/boto/boto/mws/connection.py
failed to apply patch: 0004-collections-abc.patch
==> ERROR: A failure occurred in prepare().
    Aborting...

0004-collections-abc.patch fails to be applied when installing the package.

failed to apply patch: 0004-collections-abc.patch

failed to apply patch

@fabian-ang same here

failed to apply patch: 0004-collections-abc.patch

With the new update it fails building on my machine, it can't apply 0004-collections-abc.patch. Cleaning all build artifatcs etc. did not help

looks like its removing the extracted files after installation

no such file or directory: /home/akhil/.cache/yay/google-cloud-sdk/pkg/google-cloud-sdk/opt/google-cloud-sdk/completion.zsh.inc
no such file or directory: /home/akhil/.cache/yay/google-cloud-sdk/pkg/google-cloud-sdk/opt/google-cloud-sdk/path.zsh.inc

EDIT: Should be fixed now

Hey folks, if anyone is getting this error when building the package:

PermissionError: [Errno 13] Permission denied: '/usr/lib/python3.10/site-packages/google_api_core-2.3.2-py3.10.egg-info/PKG-INFO'

Please be aware that the problem is not caused by this package itself, but is related to python-google-api-core and is being tracked in FS#73074

To anyone experiencing an issue with version 367.0.0-3 of this package, please rebuild with 367.0.0-4.

A reminder: Please use sudoforge/pkgbuilds for reporting issues in the future.

Issue tracking via AUR comments is not preferred for various reasons, the least of which is that this repository is essentially just a read-only mirror; no development occurs here.

Same here

failed to apply patch: 0004-collections-abc.patch
==> ERROR: A failure occurred in prepare().
Aborting...

Same here, failed to apply patch: 0004-collections-abc.patch System python is 3.10.1-1

There are changes related to collections.abc.Callable in 3.10 https://docs.python.org/3/whatsnew/3.10.html#changes-in-the-python-api

fails to apply patch

-> Extracting google-cloud-sdk_367.0.0.orig.tar.gz with bsdtar
==> Starting prepare()...
failed to apply patch: 0004-collections-abc.patch
==> ERROR: A failure occurred in prepare().
Aborting...
 -> error making: google-cloud-sdk

@frabjous The package is tested before it is uploaded. The URL is correct. Check your internet connection, firewall, and any host-blocking tools you may use (e.g. PiHole).

I'm getting a 404 error when attempting to download https://dl.google.com/dl/cloudsdk/release/downloads/for_packagers/linux/google-cloud-sdk_355.0.0.orig.tar.gz. Anyone else?

Would it be possible to package with the cloud-datastore-emulator component?

EDIT: Nevermind I read the pinned comments.

@allsyed component support in individual packages is still under way; only a few packages are currently maintained (check my profile for a complete list). Read the pinned comment(s) for instructions on how to receive support for this package, and how to modify your installation to allow for component management via gcloud components <command>.

How do I go about installing other components?

Like when I try to install cloud_sql_proxy. I get this message.

ERROR: (gcloud.components.install) You cannot perform this action because this Cloud SDK installation is managed by an external package manager. Please consider using a separate installation of the Cloud SDK created through the default mechanism described at: https://cloud.google.com/sdk/

@brody great catch, that was patched in. As a reminder, please use Github to submit patches, ask questions, and raise issues.

Please can you adjust the permission of the bash-completion file to 644 (from the least privilege perspective)?

diff --git a/PKGBUILD b/PKGBUILD
index ce029a3..6260b9d 100644
--- a/PKGBUILD
+++ b/PKGBUILD
@@ -66,7 +66,7 @@ package() {
   install -Dm755 "${srcdir}/${source[1]}" \
     "${pkgdir}/etc/profile.d/google-cloud-sdk.sh"

-  install -Dm755 "${pkgdir}/opt/${pkgname}/completion.bash.inc" \
+  install -Dm644 "${pkgdir}/opt/${pkgname}/completion.bash.inc" \
     "${pkgdir}/etc/bash_completion.d/google-cloud-sdk"

   mkdir -p "${pkgdir}/usr/share"

Will do, thank you for help and patience! :)

Haha, cannot patch PKGUILD because 'patch not found'. reinstalled base-devel group and all good now! :)

I guessed that was going to be the issue, but wanted to make sure we debugged it properly. Glad it's sorted out! As a note, please use the GitHub parent project for reporting issues in the future.

Haha, cannot patch PKGUILD because 'patch not found'. reinstalled base-devel group and all good now! :)

Cheers

Super weird. I still encounter this on one computer (a virtual server), however I am able to install the package on my laptop without any problems. Both are set up in pretty much the same way, but differ in the list of installed packages (e.g. laptop has desktop stuff installed). Using yay as the aur-helper on both machines. Tried manually with makepkg, same result. I cannot figure out an easy way to see why patching fails (no verbose option).

After cloning this repository on the failing machine, apply the following patch to the PKGBUILD before running makepkg; this should give us the information we need:

diff -urN a/PKGBUILD b/PKGBUILD
--- a/PKGBUILD  2020-02-18 13:33:09.701671909 -0800
+++ b/PKGBUILD  2020-02-18 13:35:33.254889896 -0800
@@ -36,7 +36,7 @@
   cd "$pkgname"

   for f in ./../*.patch; do
-    patch -p1 -i $f > /dev/null 2>&1 || ( echo "failed to apply patch: $(basename $f)" && exit 1 )
+    patch -p1 --verbose -i $f
   done
 }

I've uploaded the above patch to ix.io. To easily apply this from your virtual server:

$ git clone https://aur.archlinux.org/google-cloud-sdk.git
$ cd google-cloud-sdk
$ patch -p1 < <(wget -qO- http://ix.io/2c5o)
$ makepkg -sr

Edit: I initially made this comment with an invalid patch containing an erroneous s I had added to verify that the patch would print out the error.

Super weird. I still encounter this on one computer (a virtual server), however I am able to install the package on my laptop without any problems. Both are set up in pretty much the same way, but differ in the list of installed packages (e.g. laptop has desktop stuff installed). Using yay as the aur-helper on both machines. Tried manually with makepkg, same result. I cannot figure out an easy way to see why patching fails (no verbose option).

==> Extracting sources... -> Extracting google-cloud-sdk_279.0.0.orig.tar.gz with bsdtar ==> Starting prepare()... failed to apply patch: 0001-fix-console-io-syntax-warning.patch ==> ERROR: A failure occurred in prepare().

Since a couple of days now. Any news on this?

I apologize for the delay in responding to you; I'm just now seeing this. This would seem to indicate that there is an error with applying the patch file at commit 3540d93b5a51f9a0cd3a6b54c9491363efcfb4f3 in this repository (d60f3d6bb2808253fd4b51d975b78c47ae6e3080 in the parent).

I'm not able to recreate this at that revision, nor at the latest. For context, commits are only sent to this repository (and the parent) if the google-cloud-sdk package is built successfully in a chroot. I have also taken to testing that various subcommands return expected results without errors or erroneous content spewed out to the console due to recent upstream issues (which I have opened bugs for). These are not automated yet; I build and install the package, and then perform some manual "E2E tests".

That said, I'm surprised to hear that you encountered issues. Are you still experiencing this, @mindrunner? Is anyone else?

==> Extracting sources... -> Extracting google-cloud-sdk_279.0.0.orig.tar.gz with bsdtar ==> Starting prepare()... failed to apply patch: 0001-fix-console-io-syntax-warning.patch ==> ERROR: A failure occurred in prepare().

Since a couple of days now. Any news on this?

Hey folks,

277.0.0-2 has been released. This includes 7 new commits, bringing quite a few changes in. Namely:

• The PKGBUILD now downloads and, well, packages, version 277.0.0
• The SDK should now run with python 3 instead of python 2
• The SyntaxWarning brought about by an upstream commit is patched locally
• Building + packaging should now be a lot more quiet; interpreter-compiled bytecode are removed at packaging time (which will suppress a makepkg warning), and the various echo statements have been removed

As always, please let me know if you run into issues, or have a usability concern. While I do still monitor AUR comments, the proper place to report issues is at the Github repository.

Why is python in both depends a optdepends?

Whoops - good catch, thanks!

Also, CLOUDSDK_PYTHON still defaults to python2, so why exactly does this packages depends on python? Since Python 2 is dead, shouldn't this package change CLOUDSDK_PYTHON to python3 and move python2 from depends to optdepends?

While the core SDK supports Python3, some of the internal tools (dev_appserver and endpointscfg) that are bundled with the SDK do not yet support Python3. Therefore, both Python3 and Python2 are listed as dependencies. I'm still experimenting with setting CLOUDSDK_PYTHON to 3; I have not had time to determine whether or not this breaks the internal tools which depend on 2. I suppose that line could be removed from the script and we could rely on the internal resolution; but note that this would also default to Python2, as seen in the load order described in gcloud topic startup or online at: https://cloud.google.com/sdk/gcloud/reference/topic/startup

Why is python in both depends a optdepends? Also, CLOUDSDK_PYTHON still defaults to python2, so why exactly does this packages depends on python? Since Python 2 is dead, shouldn't this package change CLOUDSDK_PYTHON to python3 and move python2 from depends to optdepends?

Version 274.0.0 has been released and it features gcloud GA support for Python 3. See https://cloud.google.com/sdk/docs/quickstart-linux#before-you-begin.

I reverted the package to 272 after the build error was discovered.

The issue preventing me from building 273.0.0 is being tracked here:

https://issuetracker.google.com/issues/146012762

To summarize, there's a missing module: google_type_annotations. This is (supposedly) provided by the python package pytype, however, installing pytype doesn't seem to resolve this error, and I don't really have the time to dedicate digging into this further.

I could remove the step that is compiling the python source files to bytecode and bypass this error to build the package for 273.0.0, however, this would result in a SyntaxError at runtime whenever the particular file (active_peering_zones.py) is invoked, so I'm making the decision to halt upgrades until the issue referenced above is resolved, or until a future version of the SDK builds successfully.

error in building script after today's update

Installing bash completion script Fixing python references for python2 and compiling *.pyc Compiling /home/pio/.cache/yay/google-cloud-sdk/pkg/google-cloud-sdk/opt/google-cloud-sdk/lib/googlecloudsdk/api_lib/dns/active_peering_zones.py ... SyntaxError: future feature google_type_annotations is not defined (active_peering_zones.py, line 19)

Not building after today's system update:

Installing bash completion script Fixing python references for python2 and compiling *.pyc Compiling /home/ewaller/devel/build/google-cloud-sdk/pkg/google-cloud-sdk/opt/google-cloud-sdk/lib/googlecloudsdk/api_lib/dns/active_peering_zones.py ... SyntaxError: future feature google_type_annotations is not defined (active_peering_zones.py, line 19)

==> ERROR: A failure occurred in package().

@chibby0ne, this is very likely an upstream issue, but I've filed it and will be following up on Github:

https://github.com/sudoforge/pkgbuilds/issues/12

I've read the pinned comment from @sudoforge, and the comments from @oskar and @esbdb, but these assume that kubectl is installed.

I've just installed this package which is now in version 270.0.0-3, and it looks like it doesn't bring kubectl anymore but it still shows up as installed in the output of gcloud components list:

Your current Cloud SDK version is: 270.0.0
The latest available version is: 270.0.0

+------------------------------------------------------------------------------------------------------------+
|                                                 Components                                                 |
+---------------+------------------------------------------------------+--------------------------+----------+
|     Status    |                         Name                         |            ID            |   Size   |
+---------------+------------------------------------------------------+--------------------------+----------+
| Not Installed | App Engine Go Extensions                             | app-engine-go            |  4.9 MiB |
| Not Installed | Cloud Bigtable Command Line Tool                     | cbt                      |  7.5 MiB |
| Not Installed | Cloud Bigtable Emulator                              | bigtable                 |  6.6 MiB |
| Not Installed | Cloud Datalab Command Line Tool                      | datalab                  |  < 1 MiB |
| Not Installed | Cloud Datastore Emulator                             | cloud-datastore-emulator | 18.4 MiB |
| Not Installed | Cloud Firestore Emulator                             | cloud-firestore-emulator | 40.0 MiB |
| Not Installed | Cloud Pub/Sub Emulator                               | pubsub-emulator          | 34.9 MiB |
| Not Installed | Cloud SQL Proxy                                      | cloud_sql_proxy          |  3.8 MiB |
| Not Installed | Emulator Reverse Proxy                               | emulator-reverse-proxy   | 14.5 MiB |
| Not Installed | Google Cloud Build Local Builder                     | cloud-build-local        |  6.0 MiB |
| Not Installed | Google Container Registry's Docker credential helper | docker-credential-gcr    |  1.8 MiB |
| Not Installed | Skaffold                                             | skaffold                 | 22.1 MiB |
| Not Installed | gcloud app Java Extensions                           | app-engine-java          | 85.9 MiB |
| Not Installed | gcloud app PHP Extensions                            | app-engine-php           |          |
| Not Installed | gcloud app Python Extensions                         | app-engine-python        |  6.0 MiB |
| Not Installed | gcloud app Python Extensions (Extra Libraries)       | app-engine-python-extras | 27.1 MiB |
| Installed     | BigQuery Command Line Tool                           | bq                       |  < 1 MiB |
| Installed     | Cloud SDK Core Libraries                             | core                     | 12.3 MiB |
| Installed     | Cloud Storage Command Line Tool                      | gsutil                   |  3.6 MiB |
| Installed     | gcloud Alpha Commands                                | alpha                    |  < 1 MiB |
| Installed     | gcloud Beta Commands                                 | beta                     |  < 1 MiB |
| Installed     | kubectl                                              | kubectl                  |  < 1 MiB |
+---------------+------------------------------------------------------+--------------------------+----------+
To install or remove components at your current SDK version [270.0.0], run:
  $ gcloud components install COMPONENT_ID
  $ gcloud components remove COMPONENT_ID

To update your SDK installation to the latest version [270.0.0], run:
  $ gcloud components update

@bswartz thanks for catching that; python3 should have been an optdep.

This package no longer builds. The error message is:

==> Starting package()...
Copying core SDK components
Running bootstrapping script
/home/arch/google-cloud-sdk/PKGBUILD: line 38: python2: command not found
==> ERROR: A failure occurred in package().
    Aborting...

Hey folks,

The PKGBUILD has been updated from 261.0.0 to 263.0.0. I apologize for skipping 262.0.0's release; I was out of town.

I'm in the process of working out how I want to automate these updates, since Google is a trusted source (at least in terms of package integrity). I currently use a mashed up shell script to do this for me, but it still needs to be manually invoked -- after it's completion, I create the commit and handle pushing to Github (the source of truth for the PKGBUILD) and then subtree the google-cloud-sdk directory out to the AUR.

I'll almost certainly have to do two things:

1) Stop signing the Git commits with my PGP key

2) Use a passwordless key to push to the AUR

1 is more or less unimportant, as I doubt anyone has workflows that rely on my key. 2 is more concerning, as it would mean that the machine I'm using to update the AUR package is able to hit the git server with a passwordless RSA key. To limit the attack vector to packages which have a "trusted upstream source" (like this one), I'll be adding a bot user as a co-maintainer. I may eventually remove my personal account from this package if this proves successful.

Let me know if:

• Your workflow will break if my commits are no longer signed with my PGP key, or
• Your workflow will break if the package owner changes

@Brinox:

I have confirmed this regression and updated the pinned comment.

The described mechanism of installing additional components by modifying the _additional_components variable does not work. The SDK installer just won't install the given components, because "disable_updater" is set to true. This seems to affect the bootstrap install, too.

@esbdb and @oskar - There has been a pinned comment with instructions for managing components since August 2018.

I wasn't able to update kubectl component either due to the following error:

ERROR: (gcloud.components.remove) You cannot perform this action because this Cloud SDK installation is managed by an external package manager. Please consider using a separate installation of the Cloud SDK created through the default mechanism described at: <https://cloud.google.com/sdk/>

I circumvented this by installing kubectl package from AUR (which is up-to-date) and removing /opt/google-cloud-sdk/bin/kubectl.

How do I actually update installed components? I'm still running kubectl 1.7.2. Even when update gcloud, gcloud components update never gets run.

@mathieu.clabaut:

You can follow the very straightforward instructions in the pinned comment to add additional components. No additional components will be added to this PKGBUILD at any time now or in the future.

First, I must thank you for your useful package.

I'd like to ask if it Wouldn't be possible to keep the app engine component in this package until the other package is available ?

@tuxsavvy Upon reading the output you posted, that would seem to be an error from the bootstrap script itself - or rather, that yaml cannot be imported from ruamel. Something definitely got borked with your environment, as that's a bundled SDK dependency included in the tarball that this PKGBUILD downloads from Google.

PLEASE NOTE

This PKGBUILD is maintained on GitHub. Please use the following repository to submit patches, ask questions, and raise issues:

https://github.com/sudoforge/pkgbuilds

@tuxsavvy

Apart from git diff output as that returns nothing I figured you want me to compare the previous major commit

No, the intent was to make sure you didn't have a dirty working tree (changes from HEAD). I don't need to see the difference between two commits if you've not diverged from origin like you say...

You should try deleting the repository from disk and re-cloning it, and building from that. I don't use aurman, and don't know what it's trying to do under the hood, but I can tell you with absolute certainty that python-ruamel-yaml is not a build dependency.

@sudoforge Thanks for replying. Apart from git diff output as that returns nothing I figured you want me to compare the previous major commit. Below are requested output.

$ git rev-parse HEAD
929c850f3397f91040c3e23b78d3165d2a6cc3be

$ git diff 929c850f3397f91040c3e23b78d3165d2a6cc3be d7688acc171002029ee9ac1ccd6fa41ca104aee6
diff --git a/.SRCINFO b/.SRCINFO
index 922e193..92bc9e4 100644
--- a/.SRCINFO
+++ b/.SRCINFO
@@ -1,17 +1,18 @@
 pkgbase = google-cloud-sdk
        pkgdesc = A set of command-line tools for the Google Cloud Platform. Includes gcloud (with beta and alpha commands), gsutil, and bq.
-       pkgver = 214.0.0
+       pkgver = 213.0.0
        pkgrel = 1
        url = https://cloud.google.com/sdk/
        arch = x86_64
        license = Apache
        depends = python2
        optdepends = python2-crcmod: [gsutil] verify the integrity of GCS object contents
+       conflicts = google-cloud-sdk-minimal
        options = !strip
        options = staticlibs
-       source = https://dl.google.com/dl/cloudsdk/release/downloads/for_packagers/linux/google-cloud-sdk_214.0.0.orig.tar.gz
+       source = https://dl.google.com/dl/cloudsdk/release/downloads/for_packagers/linux/google-cloud-sdk_213.0.0.orig.tar.gz
        source = google-cloud-sdk.sh
-       sha256sums = f7c2720792aa8f819d553746d5ecdcd5df4a317129908e55d9da9d6bf3839379
+       sha256sums = 604d13879812bfc79879d845714211f36611d38740e6621c27f390736b43b648
        sha256sums = 36ac88de630e49ea4b067b1f5f229142e4cf97561b98b3bd3d8115a356946692

 pkgname = google-cloud-sdk
diff --git a/PKGBUILD b/PKGBUILD
index c543826..1a0e78c 100644
--- a/PKGBUILD
+++ b/PKGBUILD
@@ -6,12 +6,13 @@
 # Contributor: Justin Dray <justin@dray.be>

 pkgname="google-cloud-sdk"
-pkgver=214.0.0
+pkgver=213.0.0
 pkgrel=1
 pkgdesc="A set of command-line tools for the Google Cloud Platform. Includes gcloud (with beta and alpha commands), gsutil, and bq."
 url="https://cloud.google.com/sdk/"
 license=("Apache")
 arch=('x86_64')
+conflicts=("google-cloud-sdk-minimal")
 depends=('python2')
 optdepends=('python2-crcmod: [gsutil] verify the integrity of GCS object contents')
 options=('!strip' 'staticlibs')
@@ -21,7 +22,7 @@ source=(
   "google-cloud-sdk.sh"
 )
 sha256sums=(
-  'f7c2720792aa8f819d553746d5ecdcd5df4a317129908e55d9da9d6bf3839379'
+  '604d13879812bfc79879d845714211f36611d38740e6621c27f390736b43b648'
   '36ac88de630e49ea4b067b1f5f229142e4cf97561b98b3bd3d8115a356946692'
 )

@@ -93,13 +94,13 @@ package() {

   msg2 "Creating symlinks for applications"
   mkdir -p "${pkgdir}/usr/bin"
-  for i in "${pkgdir}/opt/${pkgname}/bin"/*; do
-    ln -st "${pkgdir}/usr/bin/" "${i#${pkgdir}}"
-  done
+  find "${pkgdir}/opt/${pkgname}/bin" -maxdepth 1 -type f -printf \
+    "/opt/${pkgname}/bin/%f\n" | xargs ln -st "${pkgdir}/usr/bin"
   rm -f "${pkgdir}"/usr/bin/{bq,dev_appserver.py*,endpointscfg.py*,java_dev_appserver.sh}

   msg2 "Fixing file permissions"
   chmod -x "${pkgdir}"/usr/share/man/man1/*
-  find "${pkgdir}/opt/${pkgname}" -name "*.html" -o -name "*.json" -exec chmod -x {} \;
-  find "${pkgdir}/opt/${pkgname}" -name "*_test.py" -exec chmod +x {} \;
+  find "${pkgdir}/opt/${pkgname}" -name "*.html" -print0 | xargs -0 chmod -x
+  find "${pkgdir}/opt/${pkgname}" -name "*.json" -print0 | xargs -0 chmod -x
+  find "${pkgdir}/opt/${pkgname}" -name "*_test.py" -print0 | xargs -0 chmod +x
 }

$ makepkg -sf
==> Making package: google-cloud-sdk 214.0.0-1 (Tue 04 Sep 2018 19:52:51 AEST)
==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Retrieving sources...
  -> Found google-cloud-sdk_214.0.0.orig.tar.gz
  -> Found google-cloud-sdk.sh
==> Validating source files with sha256sums...
    google-cloud-sdk_214.0.0.orig.tar.gz ... Passed
    google-cloud-sdk.sh ... Passed
==> Extracting sources...
  -> Extracting google-cloud-sdk_214.0.0.orig.tar.gz with bsdtar
==> Starting prepare()...
  -> Checking for newer upstream release
  -> This AUR release: 214.0.0
  -> Latest upstream release: 214.0.0
==> Entering fakeroot environment...
==> Starting package()...
  -> Copying core SDK components
  -> Running bootstrapping script and adding additional components
Traceback (most recent call last):
  File "/home/user/.cache/aurman/google-cloud-sdk/pkg/google-cloud-sdk/opt/google-cloud-sdk/bin/bootstrapping/install.py", line 12, in <module>
    import bootstrapping
  File "/home/user/.cache/aurman/google-cloud-sdk/pkg/google-cloud-sdk/opt/google-cloud-sdk/bin/bootstrapping/bootstrapping.py", line 46, in <module>
    from googlecloudsdk.core.updater import update_manager
  File "/home/user/.cache/aurman/google-cloud-sdk/pkg/google-cloud-sdk/opt/google-cloud-sdk/lib/googlecloudsdk/core/updater/update_manager.py", line 35, in <module>
    from googlecloudsdk.core import yaml
  File "/home/user/.cache/aurman/google-cloud-sdk/pkg/google-cloud-sdk/opt/google-cloud-sdk/lib/googlecloudsdk/core/yaml.py", line 32, in <module>
    from ruamel import yaml
ImportError: cannot import name yaml
==> ERROR: A failure occurred in package().
    Aborting...

Do note that the last output was me forcing makepkg within aurman directory. This is not using any AUR helper/wrapper to compile it.

Thanks in advance

@sudoforge Yes, the package has never bundled the app-engine-java component (I always installed the component by modifying the PKGBUILD file). I just wanted to highlight that installing a JDK or JRE is (and never was) sufficient to use the Java environment. This was a bug in the old package because it simply listed as an optional dependency:

'java-environment: for Java version of App Engine'

@tuxsavvy python2-ruamel-yaml is a (bundled) dependency of the SDK, and is not required for building. I'm not sure why you're experiencing that error.

Please upload the following to ptpbw.pw (or a similar pastebin service):

• The current commit that your cloned repo is at (git rev-parse HEAD)
• The output of git diff in your cloned repo
• The output of makepkg -s

I believe python2-ruamel-yaml might be required because it would fail to complete otherwise. Here is my build log without the dependency:

==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Retrieving sources...
  -> Found google-cloud-sdk_214.0.0.orig.tar.gz
  -> Found google-cloud-sdk.sh
==> Validating source files with sha256sums...
    google-cloud-sdk_214.0.0.orig.tar.gz ... Passed
    google-cloud-sdk.sh ... Passed
==> Extracting sources...
  -> Extracting google-cloud-sdk_214.0.0.orig.tar.gz with bsdtar
==> Starting prepare()...
  -> Checking for newer upstream release
  -> This AUR release: 214.0.0
  -> Latest upstream release: 214.0.0
==> Removing existing $pkgdir/ directory...
==> Entering fakeroot environment...
==> Starting package()...
  -> Copying core SDK components
  -> Running bootstrapping script and adding additional components
Traceback (most recent call last):
  File "/home/user/.cache/aurman/google-cloud-sdk/pkg/google-cloud-sdk/opt/google-cloud-sdk/bin/bootstrapping/install.py", line 12, in <module>
    import bootstrapping
  File "/home/user/.cache/aurman/google-cloud-sdk/pkg/google-cloud-sdk/opt/google-cloud-sdk/bin/bootstrapping/bootstrapping.py", line 46, in <module>
    from googlecloudsdk.core.updater import update_manager
  File "/home/user/.cache/aurman/google-cloud-sdk/pkg/google-cloud-sdk/opt/google-cloud-sdk/lib/googlecloudsdk/core/updater/update_manager.py", line 35, in <module>
    from googlecloudsdk.core import yaml
  File "/home/user/.cache/aurman/google-cloud-sdk/pkg/google-cloud-sdk/opt/google-cloud-sdk/lib/googlecloudsdk/core/yaml.py", line 32, in <module>
    from ruamel import yaml
ImportError: cannot import name yaml
==> ERROR: A failure occurred in package().
    Aborting...

@allgaeuer.fabian This package has, as of my maintership, never bundled the app-engine-java component. You should read and follow the instructions in the currently pinned comment.

@sudoforge Regarding your commit e01da1e449882ad8e4fe590b01cd09624b58143a: the Java environment requires installation of an additional component app-engine-java. Installing just the JDK or JRE is not sufficient.

IMPORTANT NOTE!

Commit e01da1e449882ad8e4fe590b01cd09624b58143a removes the appengine components from this package. These components, along with all of the others, will be made available as independent packages in the future.

If you need any of the additional components for your day-to-day workflow, set "disable_updater": false in /opt/google-cloud-sdk/lib/googlecloudsdk/core/config.json and manage components using gcloud components <command>.

Be sure to install any dependencies for the additional components that you may need.

I've gone ahead and pushed the update - as @sudoforge mentions, please use the link in the upper right "Flag package out-of-date" for AUR packages.

@deace Flagging a package as out-of-date is the proper way to inform the author of an update.

update, please.

The newest pacman exposed that the use of python2 -m compileall embeds the source directory of the packaging process into the resulting *.pyc files as strings. I've added a -d /opt/google-cloud-sdk to the compileall PKGBUILD step for this new 205.0.0 release and it seems to do the trick (checked it with strings), please holler if this breaks something though. I tested a quick gcloud and gsutil and they're functional. Ref: https://docs.python.org/2/library/compileall.html

@troyengel ah, you mean:

https://dl.google.com/dl/cloudsdk/release/sha1.txt

and

https://dl.google.com/dl/cloudsdk/release/sha256.txt

Cool. Thanks! I should have noticed that in the prepare() step. It looks like the sha1.txt file was obsoleted as of v148.0.1 of the SDK.

If you have a moment, hit me up on IRC (freenode) as sudoforge or via email. I'm planning some big changes, namely:

• Slimming the google-cloud-sdk AUR package to only include alpha and beta components
• Adding AUR packages for each individual component

This will make the available AUR packages match Google's DEB and RPM releases. Most importantly, I think (once I complete the above refactoring) we should submit a request to merge this package into google-cloud-sdk. After that, I'd be happy to add you as a co-maintainer to it and whatever other packages you'd like to support. What are your thoughts on this?

@sudoforge I used to maintain the larger package you now do (google-cloud-sdk) for a few years (2015-07-18 to 2017-06-14 in git, no logs from AUR3 pre-git). I looked in the comments, on 2017-01-13 when I was preparing 139.0.0 I wrote 'Heya all, as I was preparing 139.0.0 for release just now I notice in the sha1.txt a number of new downloads appeared that has "for packagers" in the directory name' (back then it was sha1.txt).

So I'd have to say that between 138.0.0 and 139.0.0 I saw it first appear in the hash checksum file, not that I'd read about it on their docs or saw a blog post or anything like that... doesn't really help you with your goal, I know - sorry. But does give you a date when it seemed to first have happened, which might be a clue somehow to help. Here's the commit where I first "found" sha1.txt (I don't remember how) and implemented it in the PKGBUILD we both use today as sha256.txt: (edit: I can't get this comment to stop butchering the link, look in the Changes for 0.9.84 -> 0.9.85 dated 2015-11-06 )

Hope some of this helps! Good luck tracking those archives down...

@troyengel where did you find the "for packagers" archive? i'm trying to find archives for the different components in order to create packages for them, and i'm not able to find current archives for anything.

@sudoforge - thanks! Following your hint, I see various google hits and was able to locate a random package on Debian that shows it in use, so we can follow their pattern: https://www.apt-browse.org/browse/debian/jessie/main/amd64/xss-lock/0.3.0-1/ (edit: and confirm now the Arch package does the same - https://www.archlinux.org/packages/community/x86_64/xss-lock/ )

If y'all can work out the file changes needed, I can integrate here as a patch as well etc. Thanks for the heads up that just copying it won't work, will wait on a solution. @MaddyBoo has never returned to pick on this conversation...

@troyengel (RE: zsh completions):

The zsh package owns /usr/share/zsh/site-functions, which is a good location to put completion files. This is where the zsh-completions package puts its completion files, and is (from my experience) included in the default $FPATH.

It's true that there isn't a well defined convention for where completion files should live, but this seems like a pretty sane choice to me.

It should be noted, however, that the completion.zsh.inc file will not work in its current form. It cannot simply be copied to /usr/share/zsh/site-functions/_gcloud. Google intends for the user to source this file in their shell, and I suspect this has something to do with it.

I haven't spent time debugging this, but there will need to be modifications to the file in order for it to... well, work. I'm a co-maintainer of the google-cloud-sdk package, so it's on my list of things to check out, but isn't a high priority.

Thanks for the update!

Hi @oxplot I would also like to help maintaining this package. How can I help you?

@oxplot any chance this can be updated at some point in the near future? I'd be happy to co-maintain this and help with updates.

@troyengel I understand, and you should of course keep to your vision :) A minimal package is exactly what I want since I have to install additional components anyway. I'll keep using this and apply my "hack". Thanks!

@oyvindsk - you may be looking for the "full" package which I think would facilitate your need, it's based off a different tarball and includes other components. I created this version specifically to be a minimal footprint of just the basics, it's not really intended (on purpose) to be used in the way you want. See: https://aur.archlinux.org/packages/google-cloud-sdk/

If you try to install or update packages with 'gcloud components ..' you'll get a

"You cannot perform this action because this Cloud SDK installation is managed by an external package manager. Please consider using a separate installation of the Cloud SDK created through the default mechanism .." error.

I call bullshit on this :) You can "fix" it by setting: "disable_updater": false in /opt/google-cloud-sdk/lib/googlecloudsdk/core/config.json

Maybe consider setting this since it makes this package much more useful.

@MaddyBoo - would be happy to help, however I don't use zsh so I started looking into this... and it's horribly complicated. I can't find any wiki (including Arch's), blog, or man pages which indicate how you install system wide completion files. Everything talks about "must be explicitly enabled from your shell", and refer to triggering it from ~/.zshrc and it's $FPATH. (even man zshcompsys(1) has no clue)

Can you provide another 3rd party package here in Arch which provides zsh completion files systemwide? I looked into the zsh package itself and zsh-completions package, they don't own a system directory (like /etc/zsh.completions.d/ or /etc/zsh/completions or whatnot), so it's just not clear what you do with this file (completion.zsh.inc) provided. Guidance and instructions needed. :)

The PKGBUILD seems to install the bash completions but not the zsh completions, can we get this fixed?

@troyengel optional subpackage seems to be the ideal thing to do, but I'm not familiar with it and it'll take me sometime to get around to it. Meanwhile, I'll try to get patches in if someone else can work on it.

The inclusion of google-appengine-go in release 186.0.0 has more than doubled the size of this entire package to over 500MB, it appears that there's a lot of go source getting bundled. Perhaps that can be culled manually.

==== Packages (1) google-cloud-sdk-186.0.0-1

Total Installed Size: 562.75 MiB Net Upgrade Size: 359.64 MiB ====

That was rather unexpected, perhaps the go engine should be an optional subpackage.

File "/usr/lib/python2.7/ssl.py", line 286, in match_hostname % (hostname, dnsnames[0])) ssl.CertificateError: hostname 'metadata.google.internal' doesn't match 'streaming.asonix.dog'