Union: Verify that all rows are present

[tk327]

Hello everyone,

We started to use Alteryx last week and I'm trying to combine a very large dataset (100+ files, 100k-300k rows each file). I used Union to combine them and we would like to verify that all rows are present in the output file.

The most straightforward solution is to open individual input files and count the rows, add them together then compare with the number of rows in the output. However, I wonder if we have a better way to do this.

[Thableaus · Accepted solution]

Hi @tk327 

You can use the Output File Name as Field option to verify if all your files are there.

It creates a field that will give you the file name of your input data for each row of its data.

Cheers,

[tk327]

Hi Thableaus,

Do Alteryx have a way to do this to multiple inputs at once? We process 100+ files and the process can get very laborious.  

[MichalM · Accepted solution]

@tk327 

Here's a great post on how you can point Alteryx to a directory and bring in multiple files into one big unioned table using couple of simple steps (see method 2 and 3). In addition, you can also output the extra field with a reference to file the data comes from as suggested by @Thableaus .

[SubratDas5]

Please use the count records tool after the load and also use the name as filename option . This way you will be able to know how many records were loaded from each file.