Dataset Comparison

[maozhenn]

Hello,

We are migrating data to a new platform. The old database has 100+ tables and the new one has the same tables.  Is there a automation way to compare all records from the tables in the old database to the records in the new database tables? 

[Luke_C]

Hi @maozhenn 

What type of database is it? Are you looking to compare every field of every record or simply reconcile record counts? Depending on the database there may be a way to query the metadata of each table for record counts. I've also taken a more brute force approach and used a dynamic input to pass the table names in and return the count of each (i.e. SELECT COUNT(*) FROM TableA)

[maozhenn]

Thanks for your help Luck. What do you mean by database type? I need to make sure every record in the new database matches the the one in the old, not just records count. 

[Luke_C]

@maozhenn Is it a SQL database? Oracle? MongoDB? etc. could change the approach based on the answer, although I'm most familiar with SQL.

What kind of volume are we dealing with? The best way to compare field by field is to transpose the two data sets, join on the primary key and field name, and then filter after to see where the values don't match, as well as reviewing any drop outs.