multiple object notification message by google

Question

I was copying few files of < 10MB size individually to bucket located in EU location from a US physical location using gsutil cp command. I received multiple object notification message for the same file. These multiple object notification had the same size, md5sum but different x-goof-message-number. Not all files or any specific file had multiple object notification, but this happened randomly for any file.

To eliminate the possibility of gsutil resumable transfer as culprit, I had set resumable_threshold to 15MB in .boto file.

Let me know what may be cause of multiple object notification. Multiple object notification for the same object is erroneous event for my application.

This is strange as this is not seen while the same operation is done for bucket located in US location.

Answer 1

As mentioned in the Object Change Notification documentation:

Note: Because of the retry mechanism above, it is possible that notifications are delivered more than once. Ensure that your application is idempotent with respect to processing a unique notification.

However, the X-Goog-Message-Number header should be the same if you receive multiple notifications. Are you receiving multiple notifications for the same event with different X-Goog-Message-Number values? Are you sure you didn't create multiple notification channels? You can tell if two messages are part of the same notification channel by looking at the X-Goog-Channel-Id header.