topic replacing pandas drop duplicates functionality in Alteryx Designer Desktop Discussions

replacing pandas drop duplicates functionality

wonka1234 — Thu, 06 Jan 2022 19:21:28 GMT

Hi,

Is there a way to replicate pandas drop duplicates on all columns?

Is this the same as using Unique on all columns?

Thanks.

Re: replacing pandas drop duplicates functionality

SoccerTil — Thu, 06 Jan 2022 20:06:23 GMT

From what I can see about Pandas drop duplicates, Alteryx Unique will perform the same as the drop duplicates with the First option. The drop duplicates by default will be based on all columns. You can select them all or if you only require a subset of columns then select just those. To replicate the Last option you would need to number your rows and then sort them descending first. To replicate the False option, you will need to use additional data analytics. If this doesn't help, you can supply a test input and output data set with your drop duplicates statement.

Re: replacing pandas drop duplicates functionality

atcodedog05 — Fri, 07 Jan 2022 07:08:32 GMT

Hi @wonka1234

Pandas function

DataFrame.drop_duplicates(subset=None, keep='first', inplace=False, ignore_index=False)

Another approach is you can also use a sample tool to get the first 1 row for each group or the last 1 row for each group. This way you can keep 1st occurrence or last occurrence.

inplace=False is similar as taking data from the previous step (original copy) and after drop (new copy)

ignore_index=False and indexes are not used in Alteryx

Hope this helps : )

Re: replacing pandas drop duplicates functionality

mutama — Fri, 07 Jan 2022 07:29:11 GMT

Hi @wonka1234 , in addition to the suggestions provided by @atcodedog05 and @SoccerTil, you may also refer to the Alteryx Tool Mastery Guide here regarding the detailed, step-by-step explanation on how to use each tool (e.g. Unique, Sample etc): https://community.alteryx.com/t5/Alteryx-Designer-Knowledge-Base/Tool-Mastery-Index/ta-p/84593

Re: replacing pandas drop duplicates functionality

atcodedog05 — Fri, 07 Jan 2022 14:50:15 GMT

Happy to help : ) @wonka1234

Cheers and have a nice day!