Duplicates

Hi,

I've uploaded 2 files in the tool. There are some duplicates between both files. I would like to delete those.

However, the duplicates that are included within 1 file I would like to keep them.

For exemple, here below I would like to keep line 1 2 and 4 and remove line 2.
FILE 1 - XXX
FILE 2 - XXX
FILE 2 - XXX
FILE 1 - YYY

Here below I would like to keep line 1 2 and 5 and remove line 3 and 4.

FILE 1 - XXX

FILE 1 - XXX
FILE 2 - XXX
FILE 2 - XXX
FILE 1 - YYY

Developer

Accepted answers

echuong1

I think I understand your requirements now.

I essentially found a count of each value per file. I then added a running total/count per value in the overall dataset. After adding the counts from File 1 to the overall dataset, I was able to use a filter to include any value where File = File 1 or where the running total was higher than that of the file 1 count.

New Workflow1.yxmd

All comments

phottovy

I'm not completely sure I understand the difference between the two scenarios but I attached a couple possible solutions.

In the first one, I assign a unique RecordID to all the rows in File 1 and then use the unique tool to keep all of File 1 but remove any duplicates from File 2.

In the second one, I use the Multi-Row tool to identify duplicates.

Hopefully one of these helps!

Duplicates.yxmd

GdeH

Hi, thank you for your reply.

Actually, I've extracted the 2 files from a system and when extracting those, I have an overlap.

Meaning that some lines that are included in file 1 are also included in file 2. I would like to remove the overlap items.

If I have 5 similar lines in file 1 and the exact same line appears 3 times in file 2, I would like to remove the 3 lines in file 2 and keep the 5 similar lines in file 1.

If I have 4 different lines in file 1 and those 4 different lines are also included in file 2, I would like to remove the 4 lines in file 2.

I hope this is more clear.

Quick Links

This months top contributors

AkimasaKajitani 389

mceleavey 388

mbarone 337

michael_weaver 335

Hollingsworth 335