I'm using the amazon sqs java extended client library to store all SQS messages in S3. When I delete the messages using the Amazon SQS web interface, the messages get deleted only from the SQS queue and not from S3. However, when I use the aws java sdk sqs client / the extended library client, the message gets deleted from both SQS and from S3. I'm looking to retain the actual message in S3, but just get rid of the SQS messages. Is there a way I can achieve this programmatically using the aws java sdk sqs client or the extended client?
1
votes
1 Answers
1
votes
You could create (and maintain) a custom version of the extended client library, but that is not a great solution in the long-run.
Another alternative is to use S3 event notifications which can automatically create an SQS message for each file uploaded to the S3 bucket. Your producer would need to change to use the S3 API to upload files (rather than the extended client library to queue a message). Your consumer would continue to use SQS to receive the message, but would use the normal rather than extended library. Your S3 bucket would be configured to send event notifications.
Perhaps another alternative is to try to mimic what you did manually through the SQS Console. I wouldn't be surprised if you could get the effect that you are looking for by using the extended client library to receive the SQS message (and message body that is stored in S3). Then, use the normal client library to delete the message. Since the normal client library is unaware of the S3 component of the message, it should not delete the S3 file.
