Amazon S3 Download Tool should support non-CSV File Formats

[aakarsh_kalra]

I recently cam across a limitation in Alteryx, where we can't download non-CSV files using Amazon S3 download tool. There is currently support only for CSV, and couple of other formats but we are using JSON files (.jl) extensively and not having the tool download the files into the workflow is disapointing as now i have to build a custom code outside Alteryx to do that before I can start my workflow.

Can this be please given prompt attention and prioritized accordingly.

[AdamR_AYX]

You should be able to use the csv mode to download your JSON files.  JSON files are just text files after all.

Just set your "Delimiters" to \0 (which means no delimiters) and un-check "First Row Contains Field Names" and you should be good.

Cheers
Adam

[aakarsh_kalra]

Thanks the workaround works but this should be an inherent feature in the product itself.

[PoornimaF]

Hi Aakarsh,

Thanks for submitting the request. We are currently investigating adding read and write capabilities for JSON within Alteryx due to high customer demand around this. Please reach out to me directly (pfarrar@alteryx.com) with your use case around JSON files, as it will help us ensure that your requirements are being met. 

Thanks,
Poornima

[Omid]

Hi,

Is it possible to add support for XLSX and SAS format files present in S3 when using S3 Download Tool? 

Also why is there a limitation on the type of files compared when we use S3 download tool compared to the big list available in input tool?

Regards,

Omid

[BenG]

As S3 use grows, we may be able to add new formats.  

The reason for the way we support files in S3 is that we believe streaming records is important when using large data sets.  We have to implement that specifically and it does not work with all file types.  This way, when we read a csv (or YXDB) file, we can start processing data as soon as we have a single record from the S3 data set.  With some other formats we might have to download a very large file before we can open and start processing the data.  

Thanks,
Ben

[Omid]

Thanks very much Ben for sharing the technical reasob for loading from S3. I totally understand the driver behind your deicision to make everything streaming in S3 so that there is no risk of degrated response from the interface.

However for smaller size files, the trade-off is not so evident.

IMHO ideally we can have a checkbox for streaming, checked by default and maybe in a specific advanced tab or simiar, so if we uncheck it we will have all the file types available as for normal input tool loaded in one go.

Kind regards,

Omid

[adrianloong]

Agreed. I think its an important limitation to overcome as we are storing all sorts of files in s3 due to price/performance and its great to be able to retrieve them

[dantony]

It will be super great if additional file formats such as Excel, compressed text files, etc. can be selected when using Amazon S3 Download connector tool.

[rdoptis]

Having all of the file formats available in the standard Input / Output tools would make the S3 connectors much more powerful and more usable for our organization.