Help with duplicite information

Question

Hi all,

can you please help me with following,

I have a table , see below, that shows on numerous occasions essentially the same data in the row, only differences are in collumns FSLI 1 and FSLI2 as you can see. The issue here is that I cannot simply filter out the correct/incorrect entry as it depends on the number and the name of the FSLI can vary a lot, instead I want to "filter" duplicite values according to the LPYA#, Account, NET, and then use a filter on the sample.

Essentially, I am trying to get rows 2 and 3 separatelly from the entiry table, filter out the correct one of the 2, and rejoin it with the"correct" data.

DawnDuong · Answer

hi @jpolomsky

Some sample data for inputs and expected outputs would help.

Based on the general descriptions here, I would start with getting a Unique Tool to get all the possible unique combinations that are inside the data base. In this case, you can select the column for LPYA#, Account, NET then the output D contains all the "non-unique combinations".

With this "non-unique keys", you can use Join tool to the original data to filter out all the rows that have duplicate values to support further processing downstream to eliminate the error / pick the correct values based on the dequired rules.

Dawn.

Christina_H · Answer

Sample data would help, but I would use a summarise tool to find duplicate rows (group by the relevant fields and add a count), join that back to the original data then filter where count > 1.  This gets you your set of duplicate rows that you can work on to find the ones you want.