How to upload large archive to Amazon Glacier using ruby and aws-sdk?

5
votes

The amazon documentation for glacier doesn't seem to include any Ruby examples and the documentation itself is rather sparse.

I gather I need to instantiate a Glacier Client object and then use the upload_multipart_part method to access the Glacier API, but not sure how to massage the arguments to pass to upload_multipart_part. How do I calculate the checksums that AWS is looking for using ruby? And what is upload_id?

UPDATE

Figured out most of this by reading Amazon's documentation. They don't appear to have any code samples for ruby, the github repo of code examples does not cover Glacier. But by looking at the raw API documentation and some Java and PHP examples, it looks like I'd do this:

client = AWS::Glacier::Client.new(access_key_id: ACCESS_KEY_ID, secret_access_key: SECRET_ACCESS_KEY)
resp = client.initiate_multipart_upload(account_id: ACCOUNT_ID, vault_name: 'My Vault', archive_description: "Backup of some stuff", part_size: PART_SIZE_IN_BYTES)

And if all goes well, the Amazon API response should include a unique upload_id which I would then use in subsequent calls using client.upload_multipart_part().

I'm guessing the checksums can be calculated like this:

Digest::SHA256.file(file_to_upload).hexdigest

UPDATE 2

Seems like this has already been solved:

https://github.com/fog/fog

1
rubyamazon-web-servicesamazon-glacier
Please consider uploading to Amazon S3, then setting a life cycle rule to move your object to Glacier. This is much easier to do than working with Glacier directly. – user846969
@phil But doesn't that cost more money? – Ben

1 Answers

2
votes

For anyone else interested, this link is helpful, pretty much covers most of what you need:

http://www.spacevatican.org/2012/9/4/using-glacier-with-fog/

It doesn't really provide all details on how to instantiate a glacier object:

glacier = Fog::AWS::Glacier.new({
  :aws_access_key_id => YOUR_ACCESS_KEY_ID,
  :aws_secret_access_key => YOUR_SECRET_ACCESS_KEY
})