Alteryx Designer

Definitive answers from Designer experts.

Tool Mastery | Amazon S3 Upload and Download

Alteryx
Alteryx
Created

This article is part of the Tool Mastery Series, a compilation of Knowledge Base contributions to introduce diverse working examples forAmazon S3 Upload.pngAmazon S3 Download.pngDesigner Tools. Here we’ll delve into uses of the Amazon S3 Tools on our way to mastering the Alteryx Designer:

 

 

The Amazon S3 Upload Tool and Amazon S3 Download Tool and are connectors that allow you to upload and download files to/from your Amazon S3 account with Alteryx Designer. You do not need to install an ODBC driver to use these tools, and they come with the default installation of Alteryx Designer or Server. You can find them conveniently placed under the Connectors tool category. This Tool Mastery Article is here to help you learn how to connect to your Amazon S3 account with Alteryx like a pro!

 

 

 

 

Capture.PNG

 

 

To upload data to Amazon S3 using Amazon S3 Upload Tool:

 

    1. First, you need to make sure you already have an AWS account and credentials.
    2. If you have step one covered, simply drag the Amazon S3 Upload Tool onto your designer canvas. The configuration window for the tool will load on the left-hand side of the canvas.
    3. To configure the tool, fill out the options in the configuration window as detailed below:

 


Capture3.PNG

 

 

 

1 -  an AWS Access Key is required to connect to your S3 account with these tools. You will need to reach out to your AWS admin if you don't know what your access key is.
2 -  an AWS Secret Key is also required to use this tool. Check with your AWS admin if you don't have the secret key.
3 - Expanding the drop down will give you 3 options for you to select from:

a. Hide (Default) - will hide the provided password using minimal encryption.
b. Encrypt for Machine - Allows any user on the computer to fully use the connection.
c. Encrypt for User - Allows the logged in user can use the connection on any computer.

4 - Select Default from this drop-down menu to allow Amazon to determine the endpoint automatically based on the bucket you select. To specify an endpoint for private S3 deployments, or if you know a specific bucket region, you can alternatively select an endpoint (S3 region), enter a custom endpoint, or select from one of ten previously-entered custom endpoints.
5 - You can select this option to use Signature Version 4 instead of the default Signature Version 2. Selecting Signature Version 4 will increase the connection's security but may cause the connection to be slower.
6 - Type a Bucket name to upload your data to a new bucket, or select one from the list of available Buckets.
7 - Specify the Object name (data file) to be stored in the above-specified Bucket.  You can either type an Object name or select one from the list.
8 - Server-Side Encryption is a drop-down option where you can select an encryption method for uploading to an encrypted Amazon S3 bucket
9 - The final configuration option allows to specify the File type you want to upload your data to S3 as.

 

4. Run the workflow. If the workflow runs successfully, you will see the message below in the results window.

 

 

 

Capture2.PNG

 

 

To download data from Amazon S3 using Amazon S3 Download Tool:

 

  1. Drag the Amazon S3 Download Tool to the designer canvas.
  2. You will notice the configuration is identical to the configuration of the Amazon S3 Upload Tool (detailed above). If you need help with the configuration, please refer to the screenshot above or the Amazon S3 Download Tool Help page.
  3. After you have configured the tool to access your intended file, run the workflow.
  4. When the workflow is finished running, you will notice data from the Object has been imported into Designer 

 

There is also a great video that provides an overview of this information available here.

 

Some of the common errors (or special request) you will run into with this tool:

 

Issue: S3 – Error from AWS: The HTTP version specified is not supported

Answer: Make sure any files you are downloading, or uploading do not contain whitespaces or colons.

 

Issue: Cannot read in more than one object at a time

Answer: The Community Knowledge Base Article Amazon S3 Download – Use Wildcards to Select a Single or Multiple Objects goes over how you can utilize a batch macro to read in the list of objects in a bucket and allow you to filter for the file(s) you want using wildcards.

 

Issue: To utilize Amazon S3 Tool to access a subfolder within a private External Bucket in S3.

Answer: Refer to this community thread.

 

If you'd like some additional tips and tricks on connecting to Amazon S3, please take a look at the following resources:

 

Download a Zipped Excel File from an Amazon S3 Bucket

Extending Integration with AWS

 

 

By now, you should have expert-level proficiency with the Amazon S3 Download/Upload tools! If you can think of a use case we left out, feel free to use the comments section below! Consider yourself a Tool Master already? Let us know at community@alteryx.com if you’d like your creative tool uses to be featured in the Tool Mastery Series.

  

Stay tuned with our latest posts every Tool Tuesday by following Alteryx on Twitter! If you want to master all the Designer tools, consider subscribing for email notifications.

Comments
8 - Asteroid
Hi Any plans to add Short term keys to the tool - https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_request.html , for security reasons long term keys are not allowed. Thanks
Alteryx
Alteryx

 Hi @sathiya,

 

Thank you for the reply! I am not aware if our development team has any plan to add short term keys to the tool. I would recommend you to add this feature request to our Idea page so our development team aware this is a feature our customer is looking for and potentially implement in future releases. 

 

Regards, 

7 - Meteor

Hi 

Can somebody help me upload one excel file with two sheets ( different schemas) to an S3 bucket using S3 upload?

 

 

5 - Atom

Hello, I can't see the Amazon S3 tools in my Connect tab.  The only option I have is ODBC loader.  Adding additional tools doesn't have any options for me to add to the Connect tab.  Has anyone had this issue?

 

clipboard_image_0.png

 

clipboard_image_1.png

6 - Meteoroid

Is there any way to use an S3 file as an external source for a Drop Down tool?

5 - Atom

The link to "AWS S3 access from EC2 Instance using IAM roles" goes to a "access denied" page - is the link broken? This is something I want our users to use over access keys, which is best practice when running within AWs.

Alteryx Community Team
Alteryx Community Team

@jmanduca that article has been archived, but here's what it said:

 

Using an IAM Role to Grant Permissions to Alteryx Designer or Server, Running on Amazon EC2 Instances:

  1. Create an IAM role in AWS with required access to AWS services
  2. Attach this role to the EC2 instance (Alteryx AMI) at launch time or while the instance is running
  3. Install the AWS CLI on the EC2 instance if necessary
  4. Open a command prompt
  5. Run the aws cli command to test access via the IAM role
  6. Open Alteryx Designer
  7. Go to Developer Tools
  8. Drag the tool Run Command to the canvas
  9. Configure the tool to run the aws cli command
  10.  Run the workflow
5 - Atom

in the upload to S3 bucket tool, what happens when the workflow is run a second time, does the object just get overwritten, or does S3 save it as a separate object even though the name hasn't changed. 

 

I don't see an option in the output tool configuration?

6 - Meteoroid

@dhofberg - Yes, it will overwrite the file/object name in S3 bucket when you run the workflow again.

8 - Asteroid

Is there anyway to get the last write date or updated date of the file using the S3 Download connector?  We were previously using files from a Shared Drive and getting that meta data was pretty straight forward. Not sure how to replicate with the S3 connector. 

5 - Atom

"Specify the Object name (data file) to be stored in the above-specified Bucket.  You can either type an Object name or select one from the list."

 

Unfortunately, if I try to select one from the list, Alteryx crashes... 

 

Everything else works perfectly fine. I can select a Bucket, so the Access Key and Secret Keys are set right....


Last week I was able to select a JSON file, but now I can't. Nothing seemed to be changed in the S3 Bucket. 
I did however reinstall Alteryx once... But that shouldn't cause this to happen right?