Data Validation

Question

Is there any solution or tool with the help of which I can check if there is any data present or not.

If any data is there then it should pass through the workflow and if no data is there then it should block that data and stop the data to flow for any further validation.

leon2020 · Answer

What's the error?  I guess some columns are missing when the files are not loading...

It sounds like you are already doing this, but nonetheless a possible solution is to add a text input parallel to the file, and manually entering the column headings you expect, with null data.  Union this with the file input.  When you upload a file, there is no difference but if not file is uploaded, the data headings will exist in the workflow.

If the error occurs at the union, you can change the configuration to ignore "when fields differ"... might be worthwhile going through the configuration of the file inputs and other tools too to see if you can capture/handle the error.

vijaylnyadav · Answer

You are not getting my point.

Consider,

file a and file b

If i am uploading both files then there is no error.

if i am uploading only file a then i am getting error at false side,

and if i am uploading only file b then i am getting error at true side.

i am using union before filter so both the files have different column schema and after filter data is not passing but column name is passing with no data.

Please suggest me solution so that i can complete it.

Thanks

marcusblackhill · Answer

@vijaylnyadav

Well I'm not sure if I understood well but a way to solve this can be a combination of tools to be sure you don't will have any columns with no data. Look the example attached.

Hope that helps

example.yxmd