Parameterizing S3 Keys

Community,

Because Alteryx cannot operate off of AWS IAM roles, I have a use case where I must use a control parameter to update AWS Access/Secret Keys in the S3 Upload/Download tools as well as the Redshift Bulk Loader. However, when I attempted to replace the Secret Key values, I noticed that the real Secret Key value is obfuscated. This crashed the job upon run.

My question is: Is there any way to either A) Safely update the AWS Secret Key via control parameter or replicate the obfuscation that Alteryx performs on Secret Key?

Thanks for your help!

-Phil

Output

Amazon S3

Batch Macro

Accepted answers

phetzel1

Thanks Henriette,

I did attempt to use the method of downloading the access keys; however in order for those temporary keys to work, I found that AWS needs a "Session Token" defined. I did end up solving my problem for both S3 uploads and Redshift Bulk Loads.

In order to get away from using Access/Secret Keys entirely, I included a Run Command tool that used the AWS CLI `s3 sync` command to upload my files to S3. This will use the IAM role defined for the EC2 by default. As for the Redshift Bulk Load, I used the ODBC connection to load only the schemata of the appropriate tables. In the PostSQL of that output, I inputted a COPY command using the IAM credentials of the appropriate role. This effectively mimics the normal use of the Redshift Bulk Load.

All comments

hhaigh1

Hi Phil,

This just worked for me:

I set up my workflow the way I needed it with a "real" Access Key and Secret Key.

Then I changed both the Access Key and Secret Key to "ABCDEFG" and added two text boxes to configure the keys. For the Secret Key, Alteryx displays the value "4ADA3FA5CD22424E3223AAAC22FA3975C9B15" in the Action tool but when I run the app and enter my real keys, it works just fine. So it should work for you as long as you let Alteryx use the value it displays in the Action tool (even if that doesn't match what you are expecting to see).

Here is a screenshot of my app:

9-19-2017 11-32-38 AM.png

phetzel1

Thanks for your response, Henriette!

In thinking about my predicament again, I'm not sure if I'll be able to accomplish my goal solely using Alteryx. In order to circumvent the lack of IAM role integration I'm using temporary keys from AWS. Temporary access keys require a session token to be declared alongside the temporary Access and Secret Keys. Since there is not an option to declare session tokens in the S3 Download/Upload tool or Redshift Bulk Load tool, I believe I'll have to resort to using the AWS CLI.

Mahadeva

Hi @phetzel1 Glad to hear that you have cracked the Redshift bulk load without passing Access/Secret keys via Alteryx.

Even I'm having the same requirement of loading the data that we transformed in Alteryx into Redshift tables via bulk load. And since the Access/Secret keys get rotated every 60-90 days, I haven't found any solution to dynamically pass this in runtime.

Would be really helpful if you could provide some detailed approach on how you performed bulk load to redshift tables after transforming data in Alteryx.

Quick Links

This months top contributors

atcodedog05 19598

Qiu 15867

binu_acs 15708

MarqueeCrew 13708

apathetichell 13703