Remove duplicates in a cell with newlines

Hello,

I used Concatenate in the Summarize tool to separate strings of (multiple words and characters) using newlines as the delimiter (\n).

For example, this is one row with multiple lines):

Flow Notification Processing / CHATRATH, DIVEN;#GO-AMO.06.02
Flow Notification Processing / CHATRATH, DIVEN;#GO-AMO.06.02
Hedged Share Class processing / CHATRATH, DIVEN;#GO-AMO.06.04
Hedged Share Class processing / CHATRATH, DIVEN;#GO-AMO.06.04

This is what I want as final:

Flow Notification Processing / CHATRATH, DIVEN;#GO-AMO.06.02
Hedged Share Class processing / CHATRATH, DIVEN;#GO-AMO.06.04

Any help with this?

Thank you in advance

Datasets

Transformation

Preparation

Accepted answers

flying008

Hi, @MilYanaa

Please see that below gif :

录制_2023_02_25_08_52_18_608.gif

All comments

binu_acs

@MilYanaa use summarize tool to remove duplicates

remove_dup_240223.yxmd

flying008

Hi, @MilYanaa

Please see that below gif :

录制_2023_02_25_08_52_18_608.gif

gaoa

Hi @MilYanaa

Have you tried the Unique tool? It's the first tool that came to my mind for removing duplicates. 😉

MilYanaa

Hello @gaoa, thanks for your input

I did try using the Unique tool, but this removes duplicates occurring in multiple rows and does not work within a single cell

MilYanaa

Hi @flying008, thank you for taking the time to make this Gif. However, my issue occurs within a single cell. For example, if your Row 3 said:

The output that I want is:

But this would all be in a single cell (same row and column) with newlines, as opposed to duplicates occurring across multiple rows.

binu_acs

@MilYanaa The summarise tool group-by option not working for you?

MilYanaa

Hello @binuacs, thanks for your quick response. However, my issue is that I have duplicates within a single cell, as opposed to duplicates being in multiple cells. So, the Group By tool did not work for this because I have > 1,200 lines and it was grouping without excluding duplicates.

Do you have an example where these rows occur in a single cell? In your workflow, these values are in 4 rows, as opposed to one row where these strings are separated by newlines. I've added brackets specifying that these are occurring in the same row

Flow Notification Processing / CHATRATH, DIVEN;#GO-AMO.06.02 [\n]
Flow Notification Processing / CHATRATH, DIVEN;#GO-AMO.06.02 [\n]
Hedged Share Class processing / CHATRATH, DIVEN;#GO-AMO.06.04 [\n]
Hedged Share Class processing / CHATRATH, DIVEN;#GO-AMO.06.04

flying008

Hi, @MilYanaa

However, Regardless of whether your text data is a single line or multiple lines in a cell, the idea of the processing plan is to first divide the text data into rows according to the specified line break, and then unique it .

Quick Links

This months top contributors

atcodedog05 19458

Qiu 15866

binu_acs 15708

MarqueeCrew 13708

apathetichell 13703