Detour when no records are present

[iCFO]

Once my system converts generic data to my format, it logs the converted / altered files in my format and then compares it with subsequent generic imports so that only new entries are run through the conversion process (which is extensive). The basic structure works great for when new data is present, but once all of the generic data has been converted a user has to remember to turn off containers to avoid field errors from zero records going through the format conversion process.

Does anyone have an example of a workflow where a detour is being triggered by a count of zero records? It is my first encounter with the detour tool, and thus far I am failing miserably!

[cgoodman3]

The detour tool is normally controlled by an interface tool to specify if you want it to detour to the right.

An option would be to add a Count Records tool and append this onto your data, after this have a filter where the condition is set to count records > 0. When that condition is true you can process your records downstream otherwise do something else when the condition is false (i.e. send a error message).

[AngelosPachis]

Hi @iCFO,

Not sure if the detour tool will give you what you are looking for, but an alternative would be to use a count records tool to count the number of records, then append that number to your main data stream and use a filter tool to check if the records count is greater than zero. If yes, pass the data through the format conversion part.

I think this will answer your question, although I would like to see what other users think. Hope that helps.

EDIT : I was slow, essentially try what @cgoodman3 suggests above

Regards,

Angelos

[iCFO]

Thanks for the quick feedback! This is pretty much the approach that I have been taking up until now. The problem is that when there are no new records present, the filter approach still passes 0 records / 0 bytes downstream through my conversion modules instead of bypassing them altogether. This causes a run delay and sets off a string of missing field errors in the transpose / crosstab sections. What I really want to do is to either completely bypass or disable the entire container so that the workflow does not process that section at all...

[iCFO]

It appears that dynamically disabling filters is only possible within a macro or app at this point. I decided to to join and union in text files that maintain the missing field names when they disappear because of the transpose / crosstab process. Not as clean as an automated bypass / container disable, but it keeps the errors and warnings in check and has a minimal impact on runtime.

[ripcityscorp94]

I agree with the author, extremely annoying that there isn't a clear solution to stop the portion of the workflow if there are no records

[ripcityscorp94]

This doesn't answer the question. The author, and now me, would like a solution that stops the half of the workflow from running if there are no transactions. Otherwise any filter tool, etc. errors out expecting a certain field to appear that no longer does because there are no records. 

If the Filter tool had an option to function like the Detour tool, or the Detour tool was able to have a condition built within it, that would be best solution

[Hsandness]

I've also used the text input tool to maintain field values so that the workflow doesn't error downstream but like iCFO and ripcityscorp94 I'd like to see a cleaner solution.

Especially since this doesn't work when you're creating directories for output files to be saved to. In the past I've created a separate mkdir command to create the directory with the suffix "_NoRecords" which was fine for that workflow but will not work for this one due to the potentially large number of directories this could create.

[tmonroe]

I am working on same type of problem.  I expected the filter to not allow any records through and if so I had created some other steps to be processed. I thought it was working until I intentionally created an error and the zero records kept processing.  Following, hoping someone has a solution.

[ed_hayter]

I have a flow attached that activates the new control containers depending on the number of records - if more than 1 then the flow flows through one control container - with no records then it can flow down another container and simply stop.

When you run the flow with more than 1 record note only the correct control container highlights in a green outline indicating it is running. Whereas the other container does not run. Inverse for the other input.