how to take input around 6 GB in a WF

Question

Team,

I have 100 plus (.txt) files placed in common share.. All files has same names except last 2 digits..

Is there a way we can import all in single input folder.. I tried with filename_*.txt , but after loading half a files, its shows error as too many fields..

Any way to fix this

afv2688 · Answer

Hello @Karthick461 ,

How did it fail? what was the error this time?

Regards

Karthick461 · Answer

My Actual Delimiter is '|' and i tried the same settings you gave with |

but still after 1 hr of load , its getting failed

afv2688 · Answer

Hello @Karthick461 ,

this error sometimes appears with some csv or txt files given a faulty field in a record.

Try chainging how you read the input file:

Change the delimiter to "\0", remove the first row contains field names or leave it (up to you) and increase the field lenght to accomodate all the data on the same column "5000" or more, depending on your needs.

Regards