Alteryx Designer Discussions

Find answers, ask questions, and share expertise about Alteryx Designer.
SOLVED

Insufficient space in CSV output

Hannia_Ruíz
7 - Meteor

Hi everyone!

I processed information in alteryx and now I need to export it in txt file, but I have more than 2,000,000 rows, for example CSV file only accept 1,048,576.

How can I solve this?

7 REPLIES 7
Luke_C
13 - Pulsar

Hi @Hannia_Ruíz 

 

1,048,576 is the Excel row limit. CSV format is not limited to that, are you getting an error or warning exporting the data from Alteryx? My guess is you're running into issues trying to review in excel.

T_Willins
13 - Pulsar
13 - Pulsar

Hi @Hannia_Ruíz,

 

The 1,048,576 row limit is an Excel limit, not a CSV limit.  You can create a CSV with over 2 million rows, it just cannot be opened intact in Excel.  You would still be able to open the file in a text editor such as Notepad.  The question becomes what needs to be done with the 2 million rows of data after processing it through Alteryx.  Is someone needing to review each line item?  If yes, you may need a database to store the data, but that would be a lot of data to look at.  If the data just needs to be browsed, then the text file could be brought into an Alteryx app with Interface tools attached to Filter Tool(s) to narrow the data to what the user needs to look at. 

Hannia_Ruíz
7 - Meteor

yes!, I can export it to DB, but sometimes I woutd like to divide the 2M into two sheets. 

Thank you for you answer. @T_Willins  and @Luke_C  

 

Regards!

T_Willins
13 - Pulsar
13 - Pulsar

Hi @Hannia_Ruíz,

 

CSV format does not allow for multiple sheets.  If you are looking to put all the data into Excel with breaks at each 1 million rows there are a few ways to do this, but you may run into issues writing and opening the Excel workbook.  Otherwise, you can break your data into multiple CSV files, each with 1 million rows (remainder in the last) by using RecordID and filter tools; or better an iterative macro if the row number will be unknown.  Let me know if you would like assistance with this as the solution will depend on if you are looking for Excel or CSV format.

Hannia_Ruíz
7 - Meteor

I think the macro is a great idea, but I'm just learning how to do it. 

I'd like to experience the Excel split, because with DB I haven't problem.

T_Willins
13 - Pulsar
13 - Pulsar

HI @Hannia_Ruíz,

 

I was thinking about this and there is a simpler "hack" that might work for your situation without a macro.  Look at the attached workflow.  It tags the data in groups of 1 million rows, then uses that tagging to create separate Excel sheets or CSV workbooks.  It will work for any amount of data lines.

 

CSV Output tool setupCSV Output tool setupExcel Output tool setupExcel Output tool setupWorkflowWorkflow

Hannia_Ruíz
7 - Meteor

Thank you so much @T_Willins !

Your example was very useful for me. 

 

Regards 😊

Labels