lookup a master list by comparing other lists , then append as "existing" or "new"

[meghanavmr]

1) input a master list of items (this gets built as data collected)

2) get data from sources

3) combine data from sources then, lookup the master list, if the product is in master list append field "existing" else "new". The issue I have is that if there are products that get appended in master list are duplicated and "new" items are ignored. 

[binuacs]

@meghanavmr You can use a bath macro to combine these source files, I have a question, suppose your input file columns' names are different but are the fields will be in the correct order? the example below, if in that case you can union the fields based on the position. let me know

File1 ID, name1

File2 ID, name2

File3 ID, name3

[meghanavmr]

The batch macro will not work. As the file names, column names, and the position of columns will not be the same. 

for ex:

File1-name#-columnD

Fl2-name45-columnA

Filessup3-itemname-columnC

I am thinking that the files should be converted to a different format - like in json? explore the api option? Objective is get data that is in different formats , extract common values and maintain the list of common values to be compared with another set of incoming data that is not standard