Really really wide file. no really

I have a file that is only about 60k records, but has over 92000 columns. has anyone ever tried to do anything in alteryx with a file that wide? it's a txt file with pipe delimiters. and even after managing to get one row and the header off the file it just hangs up the UI trying to attach anything to the input anchor. not sure i can attach due to data sensitivity, but just curious if there are any suggestions on things to try.

thanks!

Accepted answers

BrandonB

Workflow is attached. Make sure that when you bring in your text file you click the option for no delimiter and make sure that is says \0 for the delimiter in the Input Data tool. Also make sure that the "First Row Contains Field Names" box is unchecked.

dynamic text parsing.png

Dynamic Text File Parsing.yxmd

All comments

BrandonB

Alteryx likes data in vertical format rather than horizontal. Rather than parsing it all out to 92,000 columns, maybe it would be best to read in the text file with no delimiters, add a record ID, use a text to columns tool to split to rows. Then you can use a multi row formula to create a secondary "record ID" for each group of the original record ID values. Then you can use a filter on this secondary record ID which will correspond to which of the 92,000 columns that you have in your data. Finally, when you have narrowed down to only the columns that you care about after the filter, you can crosstab back where you group by the record ID, column headers will be the secondary record ID, and values will be the values in the data. This will result in a table that will have your 60k records, but only the columns that you actually care about.

nwhite

thanks for the help Brandon. Fortunately the file was deemed unusable, but i've since been given a smaller file - but almost 800 fields x about 31000 rows, that require audit statistics on. As i haven't used the multi-row formula at all or the transpose tool much at all, i'm hoping you can help. There are only about 50-75 fields that can be dropped, but the other fields require presence/absence. i've tried the data profiling tool also but the output is not what is needed. Thanks.

samp gf file.txt

Quick Links

This months top contributors

atcodedog05 19458

Qiu 15866

binu_acs 15708

MarqueeCrew 13708

apathetichell 13703