Unique tool with Dynamic or Unknown Options

[vepp]

Hello All,

Does anyone has Macro or Idea to select "Dynamic or Unknown Fields" in Unique tool?

My required is to pass any  number  of fields and get unique records. Every time my input fields would changes

Please help.

Regards,

Vp

[DataDynamite]

I'm validating a large migration of data from SQL Server to SnowFlake and of course I want this validation to be as automated as possible. Most of what I wanted to achieve is done but I thought if could just do this one more thing I would be really satisfied with the process.

Within a batch macro I query a table from both databases, remove duplicate records and then count records.  I also union these two datasets together, remove duplicates and count records.  The idea being that I have an exact match if all three counts are the same.   I'm feeding this batch a large number of tables to check (all of them different).   None of usual tools for removing duplicates (Unique, Sum , Tile, multi-field formula) could handle the dynamic nature of what I needed to accomplish.

This post helped me achieve that. THANKS!!!  ~DD

[DataDynamite]

Hi Dan,

I ran into a situation when the concept wasn't giving me the results I expected.

I isolated a sample to test with, I would expect it to consider these records to all be unique but because there are (multiple?) duplicate name value pairs the workflow pattern determines a few of the records to be duplicates.

CLIENTID	VERSION	GL_ACCOUNT	BUSUNIT	VALUE	LASTUPDATED
PlanOpex475	Budget 2023	701	658	907.8068685	2/17/2023
PlanOpex475	Budget 2023	701	707	907.8068685	2/17/2023
PlanOpex475	Budget 2023	701	709	907.8068685	2/17/2023
PlanOpex475	Forecast 2023 0+12	701	658	907.8068685	2/17/2023
PlanOpex475	Forecast 2023 0+12	701	707	907.8068685	2/17/2023
PlanOpex475	Forecast 2023 0+12	701	709	907.8068685	2/17/2023

I think perhaps we need to add a summarize tool after the transpose to concatenate all of the values together for a given record id.  

What do you think?

~Data Dynamite