How to setup gsutil to run from Anacron?

Question

As user, gsutil works nice. gsutil works nice when called from crontab (user).

As root, gsutil says:

Caught non-retryable exception while listing gs://....: ServiceException: 401 Anonymous users does not have storage.objects.list access to bucket ...."

gsutil does not work when called from Anacron (root). Other scripts called from Anacron run nice.

The ~/.boto file contains credentials, and is located in user HOME directory. So maybe that is causing the exception.

I tried setting BOTO_CONFIG, but it didn't change results:

$ gsutil -D ls 2>&1 | grep config_file_list 
config_file_list: ['/home/wolfv/.boto']
$ sudo gsutil -D ls 2>&1 | grep config_file_list 
config_file_list: []
$ BOTO_CONFIG="/root/.boto"
$ sudo gsutil -D ls 2>&1 | grep config_file_list 
config_file_list: []

How to setup gsutil to run from Anacron?

$ gsutil -D
gsutil version: 4.22
checksum: 2434a37a663d09ae21d1644f64ce60ca (OK)
boto version: 2.42.0
python version: 2.7.13 (default, Jan 12 2017, 17:59:37) [GCC 6.3.1 20161221 (Red Hat 6.3.1-1)]
OS: Linux 4.9.11-200.fc25.x86_64
multiprocessing available: True
using cloud sdk: True
config path: /home/wolfv/.boto
gsutil path: /home/wolfv/Downloads/google-cloud-sdk/platform/gsutil/gsutil
compiled crcmod: True
installed via package manager: False
editable install: False
Command being run: /home/wolfv/Downloads/google-cloud-sdk/platform/gsutil/gsutil -o GSUtil:default_project_id=redacted -D
config_file_list: ['/home/wolfv/.config/gcloud/legacy_credentials/redacted/.boto', '/home/wolfv/.boto']
config: [('debug', '0'), ('working_dir', '/mnt/pyami'), ('https_validate_certificates', 'True'), ('debug', '0'), ('working_dir', '/mnt/pyami'), ('content_language', 'en'), ('default_api_version', '2'), ('default_project_id', 'redacted')]

UPDATE_1

export BOTO_CONFIG worked for the terminal:

$ sudo -s
[root] # export BOTO_CONFIG=/home/wolfv/.boto
[root] # gsutil -D ls 2>&1 | grep config_file_list
config_file_list: ['/home/wolfv/.boto']

[root] # vi /root/.bashrc
add this line to end of .bashrc:
export BOTO_CONFIG=/home/wolfv/.boto
exit

open new terminal and test the new BOTO_CONFIG in bash.rc
$ sudo -s
[root] # gsutil -D ls 2>&1 | grep config_file_list
config_file_list: ['/home/wolfv/.boto']
exit

Unfortunately export BOTO_CONFIG in /root/.bashrc did not help Anacron call gsutil. The backup log shows that Anacron called the backup script, and the backup script call to gsutil failed.

Does it matter in which initialization script sets path BOTO_CONFIG? To make the path permanently accessible to Anacron (root), in which file should set BOTO_CONFIG?:

/etc/profile
/root/.bash_profile
/root/.bashrc

UPDATE_2

My credentials are now invlalid, probably from some change I made. Here is my attempt at houglum's suggestions for BOTO_CONFIG.

First authorize login to get that out of the way:

$ gcloud auth login
Your browser has been opened to visit:

    https://accounts.google.com/o/oauth2/auth?redirect_uri=http%3A%2F%2Flocalhost%3A8085%2F&prompt=select_account&response_type=code&client_id=redacted.apps.googleusercontent.com&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fappengine.admin+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcompute+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Faccounts.reauth&access_type=offline


Created new window in existing browser session.
WARNING: `gcloud auth login` no longer writes application default credentials.
If you need to use ADC, see:
  gcloud auth application-default --help

You are now logged in as [edacted].
Your current project is [redacted].  You can change this setting by running:
  $ gcloud config set project PROJECT_ID

Defining BOTO_CONFIG inline does not work:

$ BOTO_CONFIG=/home/wolfv/.boto gsutil ls
Your credentials are invalid. Please run
$ gcloud auth login

Exporting BOTO_CONFIG does not work:

$ export BOTO_CONFIG=/home/wolfv/.boto; gsutil ls
Your credentials are invalid. Please run
$ gcloud auth login

Sourcing bashrc does not work:

$ ls /home/wolfv/.bashrc
/home/wolfv/.bashrc
$ . /home/wolfv/.bashrc; gsutil ls
Your credentials are invalid. Please run
$ gcloud auth login

UPDATE_3

My credentials work if I remove my credentials from .boto, and use auth login instead (based on Your credentials are invalid. Please run $ gcloud auth login)

$ gcloud auth login [email protected]
WARNING: `gcloud auth login` no longer writes application default credentials.
If you need to use ADC, see:
  gcloud auth application-default --help

You are now logged in as [[email protected]].
Your current project is [redacted-123].  You can change this setting by running:
  $ gcloud config set project PROJECT_ID

After using auth login, gsutil works from the terminal:

$ gsutil ls
gs://redacted/
gs://redacted/
gs://redacted/

And the backup script that calls gsutil also works from the terminal:

$ ~/scripts/backup_to_gcs/backup_to_gcs.sh
backup_to_gcs.sh in progress ...
backup_to_gcs.sh completed successfully

However, backup_to_gcs.sh fails when called from crontab.

How to run gsutil from crontab?

UPDATE_4

This is in my anacron file:

1       10        anacron_test_id   BOTO_PATH=/home/wolfv/.config/gcloud/legacy_credentials/[email protected]/.boto:/home/wolfv/.boto /home/wolfv/scripts/backup_to_gcs/backup_to_gcs.sh

anacron runs the backup_to_gcs.sh script as expected, but the backup fails. When backup_to_gcs.sh script is called from command line, it works fine.

Probably because gsutil runs as user, but does not run as root:

$ gsutil ls
gs://wolfv/
gs://wolfv-test-log/
gs://wolfv2/
gs://wolfvtest/
$ BOTO_PATH=/home/wolfv/.config/gcloud/legacy_credentials/[email protected]/.boto:/home/wolfv/.boto gsutil ls
gs://wolfv/
gs://wolfv-test-log/
gs://wolfv2/
gs://wolfvtest/
$ sudo BOTO_PATH=/home/wolfv/.config/gcloud/legacy_credentials/[email protected]/.boto:/home/wolfv/.boto gsutil ls
sudo: gsutil: command not found
$ sudo gsutil ls
sudo: gsutil: command not found

Two days ago root was able to run gsutil. Since then I used dnf history rollback to uninstall a different software. Could that have effected gsutil authentication?

UPDATE_5

I followed the instructions on https://cloud.google.com/storage/docs/authentication#gsutilauth USING SERVICE ACCOUNT

$ gcloud auth activate-service-account --key-file=/home/wolfv/REDACTED.json

Activated service account credentials for: [[email protected]]

But still, root could not run gsutil:

$ sudo gsutil ls
sudo: gsutil: command not found

$ gsutil ls -la gs://wolfvtest/test_lifecycle/
CommandException: You have multiple types of configured credentials (['Oauth 2.0 User Account', 'OAuth 2.0 Service Account']), which is not supported. One common way this happens is if you run gsutil config to create credentials and later run gcloud auth, and create a second set of credentials. Your boto config path is: ['/home/wolfv/.boto', '/home/wolfv/.config/gcloud/legacy_credentials/[email protected]/.boto']. For more help, see "gsutil help creds".

The help referse to a page that no longer mentions "auth" https://developers.google.com/cloud/sdk/gcloud/#gcloud.auth

So I have one too many credentials:

$ gsutil -D
...
config_file_list: ['/home/wolfv/.boto', '/home/wolfv/.config/gcloud/legacy_credentials/[email protected]/.boto']

Are any of these credentials used by root (for anacron)? They are not in the root directory. Should credintals needed for anacron be in the root directory?

UPDATE_5 I tried again after installing Fedora 26 on How to authorize root to run gsutil?

Unknown Unknown · Accepted Answer · 2017-03-08T00:27:32

When you execute BOTO_CONFIG=<value> in the shell, you're not actually defining an environment variable, but rather a local shell variable (see this thread for more details). You want to either define the variable inline with the command:

BOTO_CONFIG=/path/to/config gsutil ls

or first export the BOTO_CONFIG environment variable, then run the gsutil command:

export BOTO_CONFIG=/path/to/config; gsutil ls

EDIT:

I just noticed that in addition to your own $HOME/.boto file, you're relying on gcloud's credentials that get set up from gcloud auth login. When you run this, gcloud creates another .boto file for you, and when you run gsutil from gcloud's wrapper script, it loads that .boto file first, followed by whatever .boto file(s) you specify with either the BOTO_CONFIG or BOTO_PATH environment variable.

If you want to run as root (which the cron job does) and use both those .boto files, you'll need to instead use the BOTO_PATH variable to list them, separated by colons, also making sure the BOTO_CONFIG environment variable is not set (BOTO_CONFIG takes precedence over BOTO_PATH... the gsutil docs mention this briefly):

BOTO_PATH=/home/wolfv/.config/gcloud/legacy_credentials/REDACTED/.boto:/home/wolfv/.boto gcloud ls

EDIT 2:

1) When you get the error "sudo: gsutil: command not found", it means that the root user cannot find the gsutil executable in its PATH. You should use the absolute path to the gsutil executable instead -- from your post, it looks like this is /home/wolfv/Downloads/google-cloud-sdk/platform/gsutil/gsutil.

2) When you activate service account credentials, the gcloud wrapper for gsutil will create a separate .boto file (with a path containing legacy_credentials/myproject@appspot[...]), and prefer to use this one if it's present. It contains the attribute gs_service_key_file, while your other .boto file probably contains gs_oauth2_refresh_token -- loading multiple .boto files with multiple credentials attributes like this will result in the error you're seeing.

If you want to use gcloud to manage your auth credentials, you generally shouldn't put anything under the [Credentials] section of your $HOME/.boto file.

How to setup gsutil to run from Anacron?

1 Answers