AMP Engine & Block Until Done Tool Behavior

[MRAgamir]

This is more of a note, but I believe that when running workflows in the designer mode with the AMP engine enabled, the traditional behavior of multiple Block Until Done tools is not handled 'as expected'.

I have a workflow that is outputting different data streams to three separate tabs within an Excel workbook. As always, I was using multiple Block Until Done tools to prevent write collisions to the same file, but I kept receiving the error associated with that. I noticed that the Block Until Done tools were functioning individually, but not as they normally do in a 'group' (i.e. higher tool # waits until the previous is complete). When I disabled the AMP Engine for the workflow, the execution happened as normal and desired -- so I am very confident that the AMP Engine was the driver of the issue that I was seeing.

I am not sure if this falls into the bug or feature category of the AMP Engine, but I didn't see documentation on it yet, so I figured I would make a post just in case someone is running into the issue as well.

Respectfully,

Mike

[TonyaS]

Hi @MRAgamir 

Thank you for trying it out and for providing feedback on the AMP Engine. 

I encourage you to continue to report any use case issues that you find with running workflows with AMP Engine enabled. We worked hard to identify differences from the original Engine as well as provide guidance on how to better optimize workflows to run with AMP. 

If you haven't already, please review our articles related to using the new AMP Engine:

https://help.alteryx.com/current/designer/alteryx-amp-engine

https://help.alteryx.com/current/designer/Alteryx-Engine-and-AMP-Main-Differences

https://help.alteryx.com/current/designer/AMP-Memory-Use

https://help.alteryx.com/current/designer/tool-use-amp

Especially the https://help.alteryx.com/current/designer/Alteryx-Engine-and-AMP-Main-Differences mentions Block Until Done with AMP. By chance does your workflow contain multiple parallel branches? AMP will run those simultaneously since it is multi-threaded. The specific details about BUD with AMP that we have documented are: 

Parallel Branch Execution and the Tool Run Order

Some workflows read from a file and then write back to it. This requires sequencing to ensure that the read is complete before the write can start. Similarly, a workflow that wants to write several sheets in one .xlsx file needs to write the sheets one at a time. Alteryx Designer provides a Block Until Done (BUD) tool to help partition the work into phases that won’t get in each other’s way.

When working on a workflow with multiple branches (largely separate streams from inputs to outputs), place the BUD tool in the workflow branch with the lowest numbered Input tool ID. This ensures every subsequent branch waits to run until the previous branch is done and the tool works as expected.

There are additional articles and documentation available about AMP:

https://community.alteryx.com/t5/Analytics/Accelerate-Your-Analytic-Processes-with-the-New-AMP-Engin...

https://community.alteryx.com/t5/Engine-Works/AMPlify-your-Workflows/ba-p/617590

https://community.alteryx.com/t5/Alteryx-Designer-Discussions/Best-practice-with-the-new-Multi-threa...

AlterEverything Podcast: https://community.alteryx.com/t5/Alter-Everything-Podcast/66-The-Alteryx-AMP-Engine-Explained/ba-p/5...

https://community.alteryx.com/t5/Engine-Works/AMP-Engine-Technical-Deep-Dive-Part-1-Why-AMP/ba-p/570...  

https://community.alteryx.com/t5/Engine-Works/AMP-Engine-Technical-Deep-Dive-Part-2-Key-concepts-of-... 

Please let me know if you still have questions and we can look into your issue further. It would be helpful to have an example workflow to illustrate the specific behavior you are seeing. 

[MRAgamir]

Hi @TonyaS,

Interesting -- in that case it appears to be an issue with something else in my workflow that is causing the issue, however the issue only happens when I turn on the AMP engine. For example, this basic test workflow DOES work with AMP enabled:

...but my workflow with a lot more preceding logic receives an error when running with AMP:

The error is the pretty basic: "Unable to open file for write: ... " -- there is no error if the workflow is ran without AMP.

Respectfully,

Mike

[TonyaS]

@MRAgamir 

Thank you for the additional detail. In order to look into this more, we would need a workflow that illustrates the issue. 
Would it be possible for you to provide a workflow with a sub-set of data that still results in this behavior? 

I can provide a secure way for you to supply that workflow if needed. 

[TonyaS]

Actually Mike, I have an update for you.

We have a future feature on our roadmap to handle Branch Ordering to allow users to define the order of execution for multiple branches. 

There is no timeline for that, but that is the cause of that particular error that you are seeing. 

[LeeST]

I have the same issue with a longstanding workflow, switch on AMP and I get the workflow "ignoring" block until done. switch off and it works as normal, if I can get a working example I can send I'll drop it on here.

Just to know you're not alone.... 

[JeromeZ]

Same problem. 

If you cach the flow befor the block until done it works but if you rune the flow without caching it dosen't work.

[DaveWeir]

Hi Tonya

Any update on the branch ordering point? (i see you wrote this post a while ago).

Seems i'm still having the same issue as most here in this post (the new AMP engine also seems to handle blank fields differently and had to reconfigure a lot of select tools)

Mike - on the block until done issue, i had the exact some road block and eventually fixed it by combining the CReW Parallel Block Until done tool (which alone with the new AMP engine - also doesn't work) and insert normal BUD tools in between (see attached).

Would be awesome if this could be handled more simply as often people want to write back data from multiple sources into in excel file across multiple tabs.

[BGirkins]

Same problem. Tons of issues with this tool after switching to AMP. I'm seeing behavior where no data continues past a block until done tool. It just stops, no data makes it past. Which is of course breaks configurations downstream as it "forgets" the fields it used to remember.

[TonyaS]

The data not passing the Block Until Done was fixed for 22.1 - are you on an older version?  There was a big push to address almost all known AMP defects before the 22.1 Release that went out on May 4th. With that release AMP is the default for new workflows that users build in Designer.