Wildcard only read first file

[largychen]

Hi everyone,

I have 8 CSV files with same schema in the same folder. When I try to run wildcard on them, it only read one file and the warning goes "Record #2 not enough fields".

Then, I changed delimiters to "\n", but this time it still read one file but the warning is gone. I am confused. Thanks for help.

[largychen]

Hey Dan

Here is the file and screenshot. 

[danilang]

hi @largychen 

Neither of the files that you've attached is actually a csv file. These are text files with a .csv extension.   A well formatted csv file has exactly one line of field definitions followed by a series of data lines as in 

ID,Name,Salary

1,"Jane Doe",20000

2,"Matt Smith",10000   

Your files have a header in red followed by the actual data in green 

If you start your input on line 6, the files should be read without error.

Dan

[largychen]

Hey Dan, Thank you. I have actually two inputs one starting in line 1 and the other starting in line 6.

The reason why I still need line 1 is that I need that info in my output sheet. The only thing I need is Row 1&2 in below

[danilang]

hi @largychen 

In that case, set record limit = 2 in your second input tool to only read rows 1&2 and avoid the errors that caused by field differences in row 3.

Dan

[largychen]

Sorry Dan, record limit = 2 doesn't work I got the same messages. Only record limit =1 will work without warning and error, but the issue is record limit seems not working with wildcard. I want row 2 in all file stack together like this below.

Input Data (223) Record #2: Not enough fields in record.
Input Data (223) Error reading "C:\Users\XXX\Cambium22_MidCase_hourly_p1_2024.csv": Too many fields in record #3

Ideal output

[danilang]

hi @largychen 

This seems to work for the 2 files that you posted

Read all the lines with \n as the delimiter and 1st row as a data row.  Take the 1st two rows from each file.  Then take the 1st row and parse the header fields using text to columns.  On the bottom skip the first row from each file and parse the data fields.  Union and use a Dynamic Rename to moved the 1st row to the headers

Dan