CSV Output Skips Records

Question

I'm attempting to output my data into 3 pipe-delimited csv files, each with an equal number of records.

I've used the Tile tool to split my records, and I'm using the Tile_Num as a suffix in my output file names, all of which works great.

However, I'm finding that some, if not all, of my output files are missing the header row and the first 15 records.

Unfortunately, even this result is inconsistent.

For example:

- when I was using relative filepaths for my output, the error did not occur at all

- but when I chose one specific filepath, then 1 of my 3 files skipped records

- and when I chose a different filepath, then all of my files skipped records.

Also, sometimes the error occurs in output #1 in one run, but it occurs in output #3 in another run.

The inconsistency is puzzling.

This error occurs in both Designer and Server instances.

Does anyone know what might be happening? Or why? Or how I can avoid this?

I've attached a sample workflow and the resulting outputs. Output 1.csv is a good example of the error.

Tile to CSV Sample.yxzp

output 1.csv

output 2.csv

output 3.csv

luepked · Answer

@Luke_C  This is very strange. I am also able to see all of the data when I download Output 1 from this post.

However, when I open my previously saved Output 1 from my own directory, it starts with Record 16. I've attached a screenshot.

I wish I could consistently recreate the error so I could investigate further and find the underlying issue.

When I'm in Designer, it shows that all records are being output, but when I open the individual CSV files in Excel, I get different results.

My input is set to start at row 1 and my output is set to write all outputs.

I've never seen it create an output without headers unless "First Row Contains Field Names" is unchecked, but that's not the case either.

The only thing that seems to be consistent is that if any leading rows are skipped, it's always the header row and the first 15 records.

When viewed in Excel, Row 1 starts with Record 16

**UPDATE**

I'm able to see all of the records if I open up the CSV output in Notepad, so maybe it's just an issue with how my Excel is reading the data.

I've just never seen that happen before.

Luke_C · Answer

hi @luepked

I did the below steps to troubleshoot:

* Input the 3 csv files you attached - total record count 3294. Header rows were present
* Reviewed the workflow, record count going to output is 3294, does not seem to be data loss
* Checked output 1 specifically, saw similar results, no discrepancy in record count or headers

How are you verifying there is missing data, do you maybe have the input tool set to start the import at something other than 1?