Dynamic Data Cleansing

[cstafford]

In most cases at the end of my workflows, I will use the data cleanse tool to cleanup the data set. However, in this particular case, the columns, which are dates, will change every day. So, as a new date appears, the data cleanse tool will not dynamically update and include the new date column in the process. In the join tools, there that *unknown, which will include it, is there anything like that for the data cleanse?

[kboeckholt · Accepted solution]

Hi @cstafford 

I tend to use the Multi-Field tool for data cleansing in cases like this as it allows for unknown fields

[cstafford]

I am trying the MF formula tool but it is asking for an expression. Is this the right tool? If so, how did you work around this? 

[martinding]

Hi @cstafford,

Would you be able to provide some sample data and which cleansing options are you using?

[kboeckholt · Accepted solution]

@cstafford 

The expression to use will depend on what type of cleaning you want to do. If you can share your data cleansing configurations I could provide more specific examples.

I tend to use it for replacing null fields with blank by configuring like this:

[mwong]

This worked for me!  Thanks for the guidance

[cstafford]

This worked, I created an expression to look at the current field and made sure the unknown box was checked so it sees all new data fields.