Advent of Code is back! Unwrap daily challenges to sharpen your Alteryx skills and earn badges along the way! Learn more now.
Free Trial

Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.

[URGENT HELP] Working with large file (>1TB)

alteryx_user2022
5 - Atom

Hi,

 

Has anyone successfully worked with large file size > 1TB (.tgz format) on Alteryx Designer (or handle data size much larger than your current disk space)?

This is stored on a 2TB SSD and stored in one file (the decrypted file is a .txt).

 

My current hard-disk is 475GB (RAM of 32GB) - and looking at using the Input Data tool to process the .tgz file (my aim is to break the file into smaller pieces to process in Alteryx). Is it also technically possible to utilize the SSD to process the data, and/or send the data in batches from the SSD?

 

My last resort would be to use an Alteryx Server, but may take time to set up. 

 

Appreciate all the advice here. Thanks. 

 

 

 

 

1 REPLY 1
Prometheus
12 - Quasar

@alteryx_user2022 If you have a place to save your files, you can split them. If not, I would suggest using the Sample tool to only bring in a certain amount of records. I attached a workflow to show you how used the Tile tool, Formula tool, and Output Data tool to split files.

Labels
Top Solution Authors