How to append without duplicating the lines

[WBCamargo]

Hi all,

I hope someone to help me.

I'm append my output in excel, but it is duplicating the lines. I would like to add just the new lines, and skip the previous lines. 

My example:

Date	SN	Time	Result	Status
10/jan/23	1A570	04:02:10	15,7	Approved
11/jan/23	20B251	05:25:15	20,7	Failed
15/jan/23	15C589	03:10:31	10,1	Approved

After appending:

Date	SN	Time	Result	Status
10/jan/23	1A570	04:02:10	15,7	Approved
11/jan/23	20B251	05:25:15	20,7	Failed
15/jan/23	15C589	03:10:31	10,1	Approved
10/jan/23	1A570	04:02:10	15,7	Approved
11/jan/23	20B251	05:25:15	20,7	Failed
15/jan/23	15C589	03:10:31	10,1	Approved
17/jan/23	40B870	07:10:59	30,5	Failed

It is duplicating the lines 10/jan/23, 11/jan/23 and 15/jan/23, which all columns are the same. It is duplicating the data.

My desired output would be this:

Date	SN	Time	Result	Status
10/jan/23	1A570	04:02:10	15,7	Approved
11/jan/23	20B251	05:25:15	20,7	Failed
15/jan/23	15C589	03:10:31	10,1	Approved
17/jan/23	40B870	07:10:59	30,5	Failed

I just would like to add the different lines (comparing all columns).

Does anyone knows how to do this?

Many thanks,

Wilson

[atcodedog05]

ignore my response

[WBCamargo]

Hi Congratulations @ atcodedog05 ! Thank you for replying.

[binuacs]

@WBCamargo Are you looking for the output like the one below?

[RobertOdera]

Hi, @WBCamargo 

Consider:

1. adding a Remove Duplicates tool after the Append in your workflow and configuring it to be unique for all columns (select all columns)

2. adding a Sample Tool after the Append in your workflow and configuring it to return the first, group by (select all columns)

Cheers!

[atcodedog05 · Accepted solution]

Hi @WBCamargo 

The only possible way I can see is

1. You read the data from the existing output file and append it with the new output rows (i,e performing append in the workflow instead of the output tool append option).

2. It is suggestible to use the record id tool before in case you want to maintain the order. For existing data start the record id from 1 and for new data start the record id 10000 or so a high number so that it always comes below existing data.

3. Remove duplicates using the unique tool or sample tool. 

4. Use the sort tool to reorder.

5. Configure the output data tool to overwrite the sheet.

There is no way we can achieve this without reading in the existing data to compare.

Hope this helps : )

[atcodedog05]

Happy to help : ) @WBCamargo 

Cheers and have a nice day!

[rbelski]

Your first option was a simple, straightforward solution that I'm sure I'll use often in the future!