Duplicate Data Issue

Hello Team,

I've a two tables,

table 1 contains the cust information like cust_id and Cust_card Number and table 2 contains the cumulative information like total_cust etc. Both of them have only MTH column same. when I'm joining these 2 tables I'm receiving the duplicate values in the final table. Is there any way in alteryx to remove these duplicate cumulative values from the final table.?

Please find the table values attached

Thanks

IMG_4924.jpg

Datasets

Output

Data Investigation

Accepted answers

alexnajm

@swapsingh2712 our solutions provide the end result you are looking for, so please clarify why you need a different solution with examples and data, otherwise it’ll be hard to help further.

For example, the multi row option I mentioned earlier would work on a merged table, but without seeing these other tables you mentioned we are guessing on how it could work

Duplicate Data Issue.yxmd

All comments

alexnajm

What do you want the final product to look like?

swapsingh2712

The final output shouldn't have the duplicate values, I've attached how It should look.

IMG_4925.jpeg

alexnajm

You can then use the multi row formula tool to clear out the "duplicate" values for each column - you'll have to do it 4 times for each column. Or you can likely leverage this macro: Multi-Row -Field -Column Macro example - Alteryx Community

The other way would be to use a RecordID tool (grouped by Mth), filter the data to RecordID=1 only, do the Join on that record only, then Union that result with the other records. Let us know how it goes!

(If you want direct help, you'll have to provide the data in Excel)

swapsingh2712

actually I just provided the dummy data, my actual data set is pretty big (more then 1M rows) so I'm not sure how many times the values are being repeated , not sure If I can use the multi row formula. Regarding the other way to use the Record_ ID tool, Could you please provide an example on how to implement. I'll see If I can provide you the dummy data in excel from my end. Thank you so much for your response

alexnajm

You can use the multi row formula, that shouldn't be a concern. But I don't see data in Excel in this post - if you want direct help, please provide some sample data in Excel form. Doesn't have to be the full thing. Otherwise good luck!

swapsingh2712

Please find the excel attached. First tab shows the merged data and rest of the two tabs are the individual tables. If possible I want to fix the merged data as I cannot make changes to the individual table and the join conditions as it's running in the database. I need to just fix the final merged data. My actual data set have more then 20 rows and I'm not sure how many times the aggregated columns being repeated

Thanks

Swapnil

Book1.xlsx

alexnajm

Here you go - if you don't need it per month, remove the grouping from the RecordID tool!

Duplicate Data Issue.yxmd

binu_acs

@swapsingh2712 Another method using tile tool

Remove_dup_31102025.yxmd

swapsingh2712

Hey Binu,

I just provided the dummy data where I was just joining two tables, but I’ve multiple tables that I need to join, Is there any way to remove the duplicates from the merged data?

alexnajm

@swapsingh2712 our solutions provide the end result you are looking for, so please clarify why you need a different solution with examples and data, otherwise it’ll be hard to help further.

For example, the multi row option I mentioned earlier would work on a merged table, but without seeing these other tables you mentioned we are guessing on how it could work

Duplicate Data Issue.yxmd

swapsingh2712

Hello Alex,

Thank your providing the workflow. As I've mentioned earlier The dummy data I had provided only had two tables and by joining those I was producing the merged data. Here If you noticed that table 1 contains the data for each individual customer and table 2 contains the aggregated data for customers two, similar to table 2 I have around 6 more tables that contains in the aggregated data for customers and In total I have around 25-26 columns that are coming from these 6 tables as merged data. I see that you used the multi row tool and using the expression you're removing the duplicates from the "Total_tran_amount (In $)" and I believe in order remove the duplicates from the column "Total_cust_ID" and "Total_cards_owned_by_cust". I need to add these columns in the multi row expression formula, but similar to this I've 24 more aggregated columns where I need to remove the duplicates. So I'm looking for a dynamic way to remove the duplicates from these columns like you've removed for "Total_tran_amount (In $)" without adding the column names individually in the expression" IF IsEmpty([Row-1:Mth]) THEN [Total_tran_amount(In $)] ELSE Null() ENDIF", so that In future If I join any additional table and I receive additional columns, the workflow would work seamlessly

I'll try to attach the dummy dataset with 25 aggregated columns.

alexnajm

No problem. And yes we understand, but if we only have part of the picture we can only give part of a response. the multi row option works well if you have limited columns, but you'll need one tool for each column you'll want to affect. So that's why I posted about the multi row and column macro from before that could work well

please try that macro out, and feel free to post again here or to a new thread with any questions. good luck!

Quick Links

This months top contributors

atcodedog05 19458

Qiu 15866

binu_acs 15708

MarqueeCrew 13708

apathetichell 13703