Download with AWS Presigned URL

[smchenry2014]

I'm trying to use the download tool to download data from AWS S3 using a presigned URL, but I keep getting the following error:  Error transferring data: Failure when receiving data from the peer.  What am I doing wrong?

[KGT]

There is not much info to go off here. First guess is encoding as you're trying to use a shortcut (pre-signed URL) in a programmatic process. 

• Is the URL encoded?
• Where did it come from?
• Are you able to hit that destination in a browser?
• Are you possibly re-encoding it? 

Break the problem into parts to track down the issue:

• Does the URL work?
• What is the download tool trying to connect to? This is not "What URL are you feeding it", but rather what is it trying to connect to. The encoding is a huge example of these 2 questions having different answers..
• Is there any other connection issue that could get in the road? (This sounds simple and not needed to evaluate, but sometimes the answer is that "The machine does not have permission to connect programmatically on a user permission level".)

[smchenry2014]

I will try to answer your questions to the best of my ability, with the caveat that I cannot give every bit of info due to company confidentiality.  But here goes:

• I do not know if the URL is already encoded, although it is generated using the boto3 python library.  There is probably documentation on that
• I am using this pre-signed URL to download a file from an encrypted S3 bucket.  Although there is a native S3 download tool, my use case requires the use of temporary credentials, which the native tool cannot accommodate
• It initially took a couple tries, but I am indeed capable of successfully hitting the endpoint in a web browser

I hope that this answers most of not all of your questions.  Let me know if you have more.

[apathetichell]

This is very not correct.

What you are saying is wrong.

Presigned URL  = credentials of the person who created the presigned url. the credentials are embedded in the link. there are no secondary credentials. they are included in the URL with keyfields like X-Amz-Signature=...

S3 download = temporary credentials. share what you are sending to your download tool.  If you have a file you need to download using a role/federated access -> you should authenticate outside of Alteryx via CLI and run AWS CLI commands via the run command tool -> or use boto3 with a config file which you use to setup your credentials.

you need to share some configuration of your download tool so we can troubleshoot this. Can you check in postman? Can you see the content-type? can you add this to the download tool.

Here's an example of the format of a presigned URL:

https://{{bucket}}/{{object}}?response-content-disposition=inline&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEBAaCXVzLWVhc3QtMiJHMEUCIEwpZuxkEL6Zjm4xvyp61GLGQDNxY9nA6XO%2FFkr%2FPo8mAiEA7NfAs%2FRGKmN5G3AyaV9RGrY3XcZFil3M9B3e6I2fVMIq1AMI%2Bf%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARABGgw2MTA0ODYzODE2OTIiDOMTQSV5rA4Ad%2FSNUiqoAwVBjye2iariFt2wtyK%2B83i7LLGfoPdHNhpiw1VVRwuVPPqov21gjUJHk5lrLuHhzoaplkpcWxUy3wc3WtOP4CX9G6GAdsRfSQUFYmZ4R737jCRrt0c9wLTFggY5g4IsK4cCP7ZliTBeIvYRVLaApl3WpAlshGes35fbPffKj3XnBUFnZtgTMLGiVYjIidVIL6Y%2F2HfXkL6GOObofY9f9CD9%2FmwImgqFWMYfQ1mXHlyooxyfOKr5kQWN%2F1bblGh3rEB6KVDbuaQYXTBoQygnQOuypoyX21epZ0szJw%2BMnTx3VANp8VwxE8vRxd9536e3TWw8qJjv49OHDVNKr3am%2BEOEr8SATxIEqFL9RQzx50dklLklD7dg9t3EUezTiboWBiLczEWL1mIwiBxSwcpH%2FDswKeBAr43RQPDsf%2FyR01pvmhxrsTsmkJ7UkOpIz1ZJY19sV9OtMn4nEJkPR%2BUrABZLnJarausBfOXLOuENoakqNSpnlHCZpyh0iV9U2noehuaL0ilurlT0LAvK%2BgZ4a9DMYDmYahns9toWJNRSBGZS0U67s2hSiBww0tmUvAY65AIzdokskPaEROENWuqly7i91RY33G1UHYAoU7oEeXuoJboIonp%2FJ9cqOYmt1fAqxAcaR6KHrFbXoGVGVNvj1oyOJcAGin6REpgxjpOfS2UEBLudSnqGLpoLHBD4cXL%2FBDeucq3k0ASf9Lx3ogi4kKb9M0HoXdnmMDxMNNtJORkDJJLSFh%2BJEtzEufgZTJK48ZkCV0qyY%2B%2FludhVXZmRb%2BK5oAxOwGPHE0dibbcog%2FSjCJJ0fs1f2lfkx%2BC642Fx1P7Y7Qk0q7ecRL2bxenvYmFpvMLVLc3FKQ2rn%2BhTQ9vO6Z35y9ecDLNXFCiUJV1nR2Buzt3XuQx6BY8KjLFvNaONGBNB%2BKNKtgVDzoO1KTuqEShKIA9th%2FZ%2F2fxzVXoFR%2Fz6CEGb%2B9O3Vn%2BXP0af8iywR1ZlvezsYd29gkjABjGAX%2FFEdnkKXKBUp3xYrUeWQdGVpKEV9ewaT75ZtcDbrH0NF6cCdw%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAY4I6S5R6HL52JJKI%2F20250114%2Fus-east-2%2Fs3%2Faws4_request&X-Amz-Date=20250114T001853Z&X-Amz-Expires=60&X-Amz-SignedHeaders=host&X-Amz-Signature=9dadaa2424abce8f8d21c1b0120d6d9ce4b68d06a27851ef65dbfccb71042433

[KGT]

To see if the simplest explanation works, double encoding... lets not skip past the easiest explanation and thing to investigate. Troubleshooting something like this will require learning some elementary conventions such as URL encoding and what the credentials/bucket/object etc are (@apathetichell has laid out most of this part, I advise getting comfortable with that terminology).

URL encoding changes the parts that won't work in a URL as they are either invalid or reserved. It is not some big algorithm that's hard to understand. e.g replace " " (space character) with "%20".

I assume the following:

• A pre-signed URL generated from boto3 would already be encoded.
• Any errors that you encountered hitting it in a browser were user error, as if they were issues that had to be resolved, then that's probably what to look at.
• The expiration on the URL is long enough.

Now, on the download tool, is the box "Encode URL Text" ticked? Does the result change with that toggled?

[smchenry2014]

Unfortunately, the error message is the same regardless of whether or not that box is ticked.

[apathetichell]

I am 90% sure this isn't an encoding situation. ->but yes - it should be encoded.

-> it's more likely:

1) -> your field is not large enough to accommodate the presigned URL field because--- who knows -it's your workflow. use a select tool-> see if it's truncated. This would usually occur if you had a specific field in a text input -> and then adjust it for full link

2) content-type. I don't see this as set in postman.

3) you are using the wrong http action. use GET.

4) you are not putting the entire presignedURL into the URL in the download tool. reminder ->>>>> this is the only thing you need. you do not authenticate a presigned link.

having said that -> without know what your error is -> all I can really do is shrug. Perhaps post your error? What response code are you getting?

[smchenry2014]

Okay, I've included as many screenshots as I can while omitting any URLs for security reasons.  I've used a select tool to make my Download URL a V_WString with a max size of 1,000,000 characters, just to be safe.  Still seeing errors however.

[apathetichell]

post your error? What response code are you getting? any of your earlier presigned urls should be timed out by now -> can you post them in the format I did above -> with the bucket/object keys omitted?

[smchenry2014]

Okay here's the error with the bucket and object omitted:

Error: Download (6): Error transferring data: Failure when receiving data from the peer for https://{{Bucket_Name}}.s3.amazonaws.com/{{Object_Name}}?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIASYNCP6NI6M2AIZR7%2F20250114%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20250114T140718Z&X-Amz-Expires=5000&X-Amz-SignedHeaders=host&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEB8aCXVzLWVhc3QtMSJHMEUCICMalkuupJE06XKLZLbE3m6qfk6v0DJZZyRNHMgUAdxbAiEAz0tyd86uKJ1RhgkHirOkHY7J0mHePtLpsz62rxqzOIUq1gIIFxADGgwxODk4NTYyMTU4ODkiDA50ceg%2FoOB%2FJTjDAyqzAj%2BWH7BZNfpuWxeb2VACrGJ1OebiW4omwI2gJs7SYA3sCnTqVvPOdmFxLFPTrFyPyx7PJXOAKp2QOUNT90MnJpgqxWzeo51n8O8PiesKnsh5BojylYJOnwKGwfc58h4b1nJX5lHgq9YSXBKY4KcklIdH12X468cEsRwXF9vF4Ys5mSeGI%2FxB4G61FM1eNKOcLGgHCgKBt3A6lch3e6X%2BOAHpUMMPd%2BNn8mjt11tCrOa9SNGPFQUDBIFbeffVgJcSQTTYVER%2BhPVCB8SRf1atu0%2FqY%2FslMgPGEHzQ1OgERRNhcLvLXVxuPBRl6uwdVGyiGmGjxWwx%2BY354%2BREkm%2Fg7OKuTfdzURoyE0B9O5A0hZH%2B9uwIKU4Q%2F7RVj141rS7u70uLKOxwPSrbW41RySLhRW9%2Ft0AwkN%2BZvAY6ogH2%2FJe3Jl9NY5qWCQjdttYErzSdSEzYkykjj%2FdgLhpfSxZnF97xla2eikUWG3ma%2FAI1UUKt5ntL9fU8U3lcVivgh%2Ft2W1pPNEQG82BHqJQTcYe0Wk7LkD%2Bt859xOL0GxiYEEKkqbvfCzRxkTzRsJ9vZhERPgk%2Ba7NlLVo%2FWbdF36OCYaba0YvZfJ%2FoQOQBrC2BeOY%2F4cANBvU0SPCSna9PLi4Y%3D&X-Amz-Signature=685b1192d3a5d497e077590f00030b3413c8c138322e8f15a0d7837f1d938361