Batch Macro : Same behaviour for any use case

[saubert]

Today, the behaviour of batch macro can be strange.

If I refer to https://community.alteryx.com/t5/Alteryx-Designer-Discussions/Batch-Macro-not-looping-after-running-...

we can have big behaviour differences between :

-wf and app

-designer and scheduler

Example here with a batch macro running for all lines in designer and only for line in scheduler

Designer works fineScheduler only runs the first line!!

I know the turnaroud (just use a message box) but it's not natural and I think

-at least the same behaviour is needed in any use case
-if you want to do some optimization, ok, but make it an option!!

[fharper]

The explanation provided in the Link you referenced is good but left out some nuance. 

1. TRUE - When a flow is made into an App it turns off browse tools, makes it more efficient and there is no expectation for manual observation and thus no purpose in a browse.  
   1. However...when using "play" instead of the "wand" to test an app browses will work.
2. If the only output tool/method inside the macro is to a browse and/or to a Macro Output tool that is NOT connected to something that leads to a tool that writes, (output tool, API output, etc.),  then it will only run one iteration due to the aforementioned optimization Alteryx employs.
   1. Why do otherwise?  if running as an App there is no way to see the Browse info and if there is no output then the data stream goes nowhere in the end and for no reason...
      1. I do not see a reason to want this feature you are asking for...as described at least.
   2. Remember...For testing the App you can connect a Browse to the macro output and use the "Play" button to run it and it will run all iterations...browse is enabled when using "play".  But if you use the "wand" then it will do as before and run 1 iteration and shut down.  Again this makes perfect sense to me.
3. HOWEVER...if you add an output tool inside the macro, to any input stream, be it a text input tool with one row of junk data or the actual data stream of interest then when running the App with the "wand" it will run all iterations.  BUT if you add a MACRO Output Tool that goes nowhere then it will revert to 1 iteration.
   1. The idea here is you are not finished because you left a data stream going to nowhere and this must be a work in progress so don't waste time processing multiple iterations...if you wanted to see what is in the data stream you would capture it in an output in the macro or output/browse in the calling App.  

So given this additional nuance maybe you could explain more fully the value of what you are asking for?  Why would it be valuable to a developer or an end user running the app to have a job run and leave nothing to look at? 

When run as an App the only evidence it even ran the iterations is if you can access the output log which end users would likely not be able to do.

[saubert]

Hello @fharper .

Here my batch macro is used to drop a table through SQL code.

What is after a drop? Well, nothing. Or sometimes some logging (and that's why there is an exit) but sometimes I use it, sometimes not, depends of the workflow.

But the main reason I want this functionnality, it's that it's pretty dangerous, I mean, really dangerous and counterintuitive to have a different behaviour between running in desktop and scheduler.

[fharper]

I disagree on your characterization of the current behavior as very dangerous in this case.  But that is just my opinion.  

That said you may have an argument on modifying the optimization based on executing a tool that uses sql. 

I would like to see the actual macro to see what u r are doing.  What tool are you using to execute sql to perform a drop table?

Everything I suggest will sound like a work around rather than a solution but you can

1. Make your sql dynamic and put the tool executing the sql in the main flow vs macro.  

2. add an output tool or pass out a record confirming the tables dropped which is more natural and good practice than using the message box as you mentioned.

3. Another option is to write a dynamic script to drop the tables in the macro and execute the batch from a run tool. 

I suspect the optimization logic is complex and far reaching on how Alteryx runs and to tweak it is likely a more costly and complex problem than one thinks.  So finding the best alternative for your need is probably the best you can do for now.  But I’m not Alteryx so this is again just opinion.

[troyfurnace]

I ran into this with Python and Command cases, where the meat of what's occurring happens outside of Alteryx's jurisdiction, and it thinks it's done before it really is.

Adding a browse as some threads suggested only outsmarts the Designer, but when in Gallery (which implicitly disables browses), once again it would end the iterations early.  It really, really needs the output stream to be used.

Based on what I could find, the options were:

a) Generate dummy output to satisfy it (invites trampling or race condition)

b) Modify all contributing macros (several in overall process) to not have an output stream (rather not make two versions of each macro)

...neither desirable.

But I found a way to trick it.  I created a simple standard macro with an input stream (template = single column "Any Field"), a count records, and browse.

Placed on the output streams where I didn't want to do anything with the output, this was enough to trick it into still going.

Guts of the macro:

Guts of macro

Calling the macro:

Attaching to output stream I don't care about

Someone more versed than me can tell me a more stripped down version of what I did that would still work (I ran out of fuse a long time ago and stopped at "working").

[msmith6986]

This is maddening that a batch macro would run differently via the Scheduler and when run manually. 

I have a file output tool inside my batch macro.  When run manually, it runs as expected (a file is output for each date in my batch).

However, when run via the Scheduler, it only runs once. 

I had to add ANOTHER (garbage) file output tool AFTER the macro to get the macro to run more than one iteration via the Scheduler.  And I only knew to do that after significant time searching the community and finding this thread.

I agree that this is VERY DANGEROUS (and just plain awful "functionality").