How to restrict input files columns count

[Karthick461]

Hi,

 I am reading around 400 txt files from common share. Each txt files having 100000 records. Out of 400 , around 7 or 8 has additional columns with no header. its just # as additional column at only one record of row. so when i run the files, I am getting error as "Too many fields at 11848 line" like this.

when i manually open that particular txt files and clear # means, then its working. but its repeated process for me. 

Is there any way we can avoid this like take only 14 columns and ignore remaining.. like this  , do we have any option

[atcodedog05]

Hi @Karthick461 

Set Field length to a high number like below

Hope this helps : )

[Karthick461]

Hi @atcodedog05 

Still getting same error

[PhilipMannering]

@Karthick461 Change the delimiter to "\0"

[Karthick461]

Hi @PhilipMannering ,

 If i change the delimiter to \0 , then i am getting output as below . All 60 plus columns in single column

[atcodedog05]

Hi @Karthick461 

Can you provide a sample file

Happy to help 🙂

[atcodedog05]

Hi @Karthick461 

Here is how you can do it.

First read the file with below highlighted config 

Your data will be read like this where header is also in the data

Use text to column with delimiter as | (pipe) to split to columns like below. You can use a huge number and later remove blank columns using data cleanse tool option remove null columns

Use dynamic rename tool to take headers from 1st row and you will get output like below.

Hope this helps : )

[gabrielvilella]

Hi @Karthick461, your file is probably damaged. This error is probably related to some record with a missing delimiter. You can use the last @atcodedog05 suggestion to read the file and you will be able to identify which record is giving you trouble.

[Karthick461]

Here is the Sample file attached @atcodedog05 

[atcodedog05]

Hi @Karthick461 

Your output would be like this

Hope this helps : )