community
cancel
Showing results for 
Search instead for 
Did you mean: 

community Lounge

Get to know your fellow analytics buffs and discuss a wide range of non-product related topics!
New Data Science Blog

Check out the latest post: All Models Are Wrong

READ MORE
SOLVED

Removing Duplicates

Highlighted
Meteor

Hi Team,

I have a training data set. A user can have a training status as "completed" and same training as "In progress". I would like to remove "training in progress" record, if that training has already been completed for same user.

I have 50 thousand record in real data set. Duplicates should be removed within each users data set, then look for same condition in another user's data. Please assist me here.

My sample Data set is attached.

 Thanks!

Alteryx
Alteryx

Hi @itahir04,

 

Thanks for your question! To achieve your desired result:

 

1. Sort data on "training status" such that training complete will come before training in progress (i.e. in descending order)

2. Use the 'Unique' tool to  de-dupe records. Use UserID and Training Title as the 'Unique' fields

 

See workflow below:

 

dedupep.png

 

Let me know if you have any additional questions. 

 

Thanks!

 

Amelia

Meteor

Thanks a lot @