S3 presigned URL: Generate on server, use from client?

Question

Is it possible to generate an S3 presigned URL in a Lambda function and return that URL to a client, so the client can use it to do an unauthenticated HTTP PUT?

I'm finding that S3 is unexpectedly closing my HTTPS connection when I try to PUT to the URL I get from the lambda function, and I don't know if it's because the server and client are different entities.

Can I do what I want to do? Is there a secret step I'm missing here?

EDIT: per Anon Coward's request, the server-side code is:

presigned_upload_parts = []
        for part in range(num_parts):
            resp = s3.generate_presigned_url(
                ClientMethod = 'upload_part',
                Params = {
                    'Bucket': os.environ['USER_UPLOADS_BUCKET'],
                    'Key': asset_id,
                    'UploadId': s3_upload_id,
                    'PartNumber': part
                }
            )
            presigned_upload_parts.append({"part": part, "url": resp})
        return custom_http_response_wrapper(presigned_upload_parts)

The client-side code is:

 for idx, part in enumerate(urls):
                startByte = idx * bytes_per_part
                endByte = min(filesize, ((idx + 1) * bytes_per_part))
                f.seek(startByte, 0)
                bytesBuf = f.read(endByte - startByte)
                print(f"Buffer is type {type(bytesBuf)} with length {len(bytesBuf):,}")

                print(f"Part {str(idx)}: bytes {startByte:,} to {endByte:,} as {part['url']}")
                #resp = requests.post(part['url'], data = bytesBuf, headers = self.get_standard_headers())
                resp = requests.put(
                    url = part['url'],
                    data = bytesBuf
                )

The error I'm getting is:

ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

The presigned URL looks like: https://my-bucket-name.s3.amazonaws.com/my/item/key?uploadId=yT2W....iuiggs-&partNumber=0&AWSAccessKeyId=ASIAR...MY&Signature=i6duc...Mmpc%3D&x-amz-security-token=IQoJ...%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F...SWHC&Expires=1657135314

Yes, generating the presigned URL on a different machine than the user is pretty much the canonical use case for them. Can you give more details, including the code that uses the URL? — Anon Coward
Thank you for the use-case validation. Sample code has been added to the post. — Nevo

Nevo Nevo · Accepted Answer · 2022-07-07T13:18:33

There was a bug in my code somewhere. I ran the code under WSL as a test, and in the Linux environment got a more friendly error that helped me find and fix a minor bug, and now it's running as expected in the Windows environment. Whether that's because of the bugfix or some other environmental change I'll never know.

S3 presigned URL: Generate on server, use from client?

1 Answers