Data Cleansing

[johnrexrima_19]

Hello,

I have a screenshot below as an overview of my workflow. So there are potentially 2 to 3 reporting periods I need to report each month (Annual, Semi, and Quarterly). It varies so I have 3 Directory tools representing up to 3 source files. Basically, all available files will undergo a series of analysis and I'll be joining the result of the analysis into one file and output a new file.

So the problem I have is the use of the Data Cleanser tool prior to the analysis. I noticed that when the expected file isn't available, the field selections that will undergo data cleansing are deselected. So the next time I run the workflow and the file becomes available, the Data Cleanser no longer has field selections. I worry if the same thing is going to happen when the workflow is loaded in the server so I am looking for a possible solution to keep the field selections all the time.

Thank you.

[Assaf_m]

Have you considered using the select tool before the data cleansing tool ?

[Assaf_m · Accepted solution]

I have tried this but as you said the field gets deselected when the previous run does not have the file.

however, you can try by adding text input tool with a template that keeps all the fields required even if the original data is not available. this way the data cleansing tool will still see the fields.

let me know if this works for you.

[johnrexrima_19 · Accepted solution]

Hi @Assaf_m, Either works for me but this one is a safer approach. Thank you so much!