What's the proper way to log any errors or warnings when performing a quiet rsync?
This is what I currently run from my crontab:
gsutil -m -q rsync -r -C /mount1/share/folder gs://my-bucket-1/folder/ > /mount2/share/folder/gsutil.log
Since the log file is always completely empty and I'm uploading terabytes of data I'm starting to think that maybe even errors and warnings are being supressed.
After having realized that this is related to how you pipe stdout and/or stderr to files in general, the answer really lies within this existing thread: How to redirect both stdout and stderr to a file
So a simple solution to log as much as possible into one single log file could be something like:
gsutil -d rsync [src] [dst] &> [logfile]
...where -d enables debug output. I found this to be the only way to show files which were affected by an error such as CommandException: 3 files/objects could not be copied. Please note that -d exposes authentication credentials.
-qbut they should get sent to stderr instead of stdout. Can you redirect stderr as well? In general though, you shouldn't receive many errors, if any. - jterracecommand 2>&1 >> gsutil.log. Is that what you mean? It's hard to know if it is really working since there is absolutely nothing being logged to the file. - fredrikcommand >> gsutil.log 2>&1. Thank you. Still not getting anything in the log when using -q though... but I guess that's a good thing! :) - fredrik