Flag Incorrect Data Types and Columns

I am looking to create a workflow that will validate output files from another workflow.

The workflow should be able to validate columns are in proper order and the datatypes are correct. I have an example schema file that I can put in as a key for the workflow.

Ideally the workflow will flag if a column or datatype is incorrect, for example flag "date" column if a country was in it.

I have never approached a task like this and any advice/ideas are appreciated.

Thank you!

Data Investigation

Accepted answers

JagdeeshN

@JCornn

If you are also looking to check for values too, a join to make that comparison will do the trick.

Best,

Jagdeesh

All comments

JagdeeshN

@JCornn

Can you please provide some samples.

JCornn

@JagdeeshN , Thank you for the quick help!

I don't really know what to provide for a sample. I do not have much materialized for this workflow that would provide any help, I haven't found a way that works.

I have a schema key that will be used, the workflow would simply need to flag if the file coming through had a different column order or data types then the schema key.

In the example below, the workflow would flag the column Date due to the incorrect data type.

Date	Country	Currency	Asset
12/12/2021	US	$	Stock
12/12/2021	US	$	Stock
$	US	$	Bond
12/12/2021	US	$	Bond

Quick Links

This months top contributors

atcodedog05 19458

Qiu 15866

binu_acs 15708

MarqueeCrew 13708

apathetichell 13703