I store big zip files in a s3 bucket so i'd like to download them by chunks using the python boto3 library.
I first tried to use object.download_fileobj method but there's no way to get only parts of the object so I now use s3.get method which supports a PartNumber=X argument.
However i'm unable to find the number of parts before i start to get parts. I mean if i call get without given a part number in argument i get the following response:
s3.Object('mybucket', 'mytest.zip').get()
{'ETag': '"493458a23b7d2ed524e8f144aa9e91f4-27"', 'LastModified': datetime.datetime(2017, 2, 23, 9, 53, 48, tzinfo=tzutc()), 'ResponseMetadata': {'RetryAttempts': 0, 'RequestId': 'E67ABBE9682AE0CA', 'HTTPHeaders': {'accept-ranges': 'bytes', 'x-amz-id-2': 'Kymvd3rkQZFkjOdnNvbh1f0OG2zFzwJQEai++kvfnZSNtQ2cSQ7wmASMZxT17xg/WJJ29xBxyUQ=', 'date': 'Thu, 23 Feb 2017 10:55:02 GMT', 'last-modified': 'Thu, 23 Feb 2017 09:53:48 GMT', 'server': 'AmazonS3', 'content-type': 'application/zip', 'x-amz-request-id': 'E67ABBE9682AE0CA', 'content-length': '222492172', 'etag': '"493458a23b7d2ed524e8f144aa9e91f4-27"'}, 'HostId': 'Kymvd3rkQZFkjOdnNvbh1f0OG2zFzwJQEai++kvfnZSNtQ2cSQ7wmASMZxT17xg/WJJ29xBxyUQ=', 'HTTPStatusCode': 200}, 'ContentLength': 222492172, 'ContentType': 'application/zip', 'Body': , 'AcceptRanges': 'bytes', 'Metadata': {}}
We can see that there's no PartCount here
If I do the same but with PartNumber=1:
s3.Object('mybucket', 'mytest.zip').get(PartNumber=1)
{'ETag': '"493458a23b7d2ed524e8f144aa9e91f4-27"', 'LastModified': datetime.datetime(2017, 2, 23, 9, 53, 48, tzinfo=tzutc()), 'PartsCount': 27, 'ContentRange': 'bytes 0-8388607/222492172', 'ResponseMetadata': {'RetryAttempts': 0, 'RequestId': '2EE3109196C76834', 'HTTPHeaders': {'accept-ranges': 'bytes', 'x-amz-id-2': 'Pl4ybedoDA99xCH2fa5zuge9Az7rPxZET+EB2fAZ4BtTDo5dqw/fJZ8PNu3vM5/0mTUkj9/AqhY=', 'x-amz-mp-parts-count': '27', 'date': 'Thu, 23 Feb 2017 10:58:20 GMT', 'etag': '"493458a23b7d2ed524e8f144aa9e91f4-27"', 'content-range': 'bytes 0-8388607/222492172', 'content-type': 'application/zip', 'server': 'AmazonS3', 'x-amz-request-id': '2EE3109196C76834', 'content-length': '8388608', 'last-modified': 'Thu, 23 Feb 2017 09:53:48 GMT'}, 'HostId': 'Pl4ybedoDA99xCH2fa5zuge9Az7rPxZET+EB2fAZ4BtTDo5dqw/fJZ8PNu3vM5/0mTUkj9/AqhY=', 'HTTPStatusCode': 206}, 'ContentLength': 8388608, 'ContentType': 'application/zip', 'Body': , 'AcceptRanges': 'bytes', 'Metadata': {}}
I know can see the number of chunks
'PartsCount': 27
Is there a way to get the number of chunks before I start to download the file ?
