Iterative macro troubleshooting

[FilipR]

When I run an iterative macro from inside of it, the results I will see will be from the first iteration only (even if normally the macro would take much more iterations to finish). Is it possible to set the number of iterations for the macro to run before it gives me the results, and in an easy an convenient way? And I mean: within the macro itself, not when it's used in another workflow (then I'd use the Maximum Number of Iterations option).

Example: Let's say I have an iterative macro that runs 100 times. For the first 15 iterations it gives me exactly what I need, but after that it starts to fail. I would like to see the input it's getting in iteration #16 to troubleshoot.

[gawa · Accepted solution]

hi@FilipR 

For debugging of Iterative Macro, this is one of my best practice:

1) Put Output tool at the same position to Iteration Output, and configure output YXDB file.(call this output "debug.yxdb" herein after)

2) In Iteration Input, specify File Input(not Text Input) and select "debug.yxdb" as output of step 1.

3) Run this yxmc workflow until wrong/unwanted output comes out from Iteration Output (in your case, run 16 times), and debugging.

4) After debugging is completed, check "Disable Tool" in Output tool so that it does not create the debug.yxdb anymore.

 

Please be careful, if you use the variable [Engine.IterationNumber] in the workflow, that variable is always "0", that may result in wrong output in each trial iteration as stated above.

=================================================

For just information as per my experience...if your output fails suddenly after some iterations, the cause is typically that field type is not configured correctly. 

For example, you put initial input data in Macro Input tool, and you assume that filed type is INT64. But, if all of the value of that filed in Macro Input tool are between 0 and 255, it automatically detects "Field type is BYTE", while you assume it should be INT64. That difference of field type will cause the issue after a particular iteration passes.  

So I strongly recommend to put Select tool right after Macro Input tool, and change filed type to what you actually need.

[danilang · Accepted solution]

Hi @FilipR 

If you're looking to debug issues with the macro input on various iterations, use a Message tool immediately after the Macro Input.  Set the Message Type to Messages.  In the Message Expression, build up a string that represents the values of your inputs.  The message tool will push this string to the Results window.  In your main workflow Runtime config check "Show all Macro Messages"

Dan

[FilipR]

@gawa @danilang , good ideas, guys. However, both are not exactly "easy and convenient". I hoped there's a hidden option somewhere that would let me change the default number of iterations the macro will go through when I click Run before showing me the results. 

Also, by "failure", I didn't mean the macro would necessarily end with an error. The failure can be just some logic flaw in the transformation, which would make the data I receive from iteration #15 incorrect.

[gawa]

@FilipR

Hmm,,, I'm still not clear about "hidden option" you mentioned. You can specify "Maximum Number of Iterations" in Interface Designer. Beside, if you specify "Output left over records", you will get the result of last iteration at output of that macro even if iteration does not end within the specified Maximum Number of Iterations, I suppose you already have tried these things though.

If you want to see the output(=input to next iteration) for each loop, you just have to put another Macro Output tool, and, prior to that additional Macro Output tool, put Formula tool to append the column "Iteration" to indicate the iteration number. By doing so, you can see all of records after WF runs and distinguish them from which iteration the record comes.

[FilipR]

@gawa, as I mentioned in the initial post, I was looking for an option to show me the results when I click Run inside the macro, not when I use it in a workflow.

I even mentioned I'd use the Maximum Number of Iterations if I wanted to see results in a workflow the macro is used in, but that's not what I was asking for..

[gawa]

@FilipR OK, so proposed method in my first reply(using debug.yxdb) is the easiest way I can propose, although it needs running yxmc WF 16times(only for debugging purpose, I would have patience😅).

If somebody else could come up with easier/more convenient idea, I want to learn it.

[danilang]

Hi @FilipR 

There is no option to run a macro more that one iteration in designer.   When you're in a specific macro file, the engine treats it like any other workflow and runs once from start to finish.  To find out what's happening on iteration X you'll need to embed the macro in a driver workflow and use a combination of the techniques that @gawa and I have provided. 

Dan     

[FilipR]

@danilang @gawa, it's a pity Alteryx Designer doesn't have the option to do that. Thanks for your suggestions, though.