Import Errors SAP file

[jonghr]

I ran a FBL3N report from our SAP environment and try to read this into Alteryx.

The CSV file is read with a | delimiter.
Of the 600K+ line items, about 3K fall out in the initial Input step.

Error messages that I get are: 

1) Too many fields in record

2) Not enough fields in record

Error 1 comes from the fact in the text column there are "additional" | that are used to indicate new lines in the text.

Error 2 comes from the fact that some lines have special characters in the text column and creates 1 large field, rather than breaking out all delimiters after the special characters.

All lines for the errors are blanked out, so I can't do a error handling.
How can I make sure that I can import ALL lines and fix the errors on the way?

[danilang]

Hi @jonghr 

Since your "|" is used as both a field delimiter and a new line delimiter within a field, the best way to read the entire file is to import it as a csv with \n (new line) as the delimiter.  You may need to increase the record length from the default 256 as well.  This will read each line of data as a row with a single wide column.  From there you can start to parse out the various fields from the single large field.  

If you can provide a small sanitized sample, I'm sure the someone here will be able to help you with the parsing

Dan