Best Practice - Data Validation

[echenty]

Hello!

I was wondering if there are different techniques and best practices for data validation (as part of the data shaping process). One simple instance is for the workflow to error when the wrong number of rows populate after the join tool. Ideas/thoughts are welcome!

Emily

[jdunkerley79 · Accepted solution]

I am a big fan of the Test tool, especially since we got Browse Anywhere. I find it an essential tool for writing custom extensions - believe the in house devs use it for testing the core tools too.

I use it on joins where I expect complete coverage of one or more of the inputs and verify that Left or Right outputs contain no rows. You can attach filters as precursor steps to validate data rows as well. 

For column checking the Field Info tool is invaluable. Again often with the test tool.

Main downside of the Test tool is no pass through, can achieve some good verification checks with the Message tool as well.

[echenty]

Thanks James! Very useful feedback!