Filtering and union count

[wonka1234]

Hi,

I am having a hard time seeing how things are filtered and unioned in my workflow.

How come when I filter the user id before the end of the workflow I get a count of 341, vs 340 when I filter my entire dataset at the very end?

What am i missing here?

[DavidP]

Hi @wonka1234 

Here's what I think is happening:

As far as I can tell, the only difference is the sequence of the Unique and Filter tool on [ACF2ID FINAL] coming out of the false output of the [Entitlement Name] filter.

Instance 1 does the unique test first. I'm guessing that [ACF2ID FINAL] is not part of the uniqueness criteria, so I think there are 2 records with matching uniqueness criterial but the 1st one that gets passed through is a record where [ACF2ID FINAL] != AGYEMB5. The record where [ACF2ID FINAL] = AGYEMB5 is discarded to the Duplicate output, but the other one is then filtered by the filter tool after the Union.

In Instance 2, the [ACF2ID FINAL] != AGYEMB5 is filtered first and therefore nothing is caught by the Unique tool, resulting in the extra record in the output.

[DawnDuong]

hi @wonka1234 

Judging from the picture, it seems like the issue is with how the Unique tool settings, like what @DavidP  pointed out.

You can check how many records there are at the #2 and #1 outputs to verify. I suspect that the #2 record counts are the same for both paths but the #1 record counts are different. If this is the case, go back to check the Unique tool settings to make sure that the combination of criteria is the desired outcome.

Usually, for spot check, I would use a smaller dataset (~20) with known attributes to test which path gives the desired outcome.

Dawn.