AMP Engine & Block Until Done Tool Behavior

This is more of a note, but I believe that when running workflows in the designer mode with the AMP engine enabled, the traditional behavior of multiple Block Until Done tools is not handled 'as expected'.

I have a workflow that is outputting different data streams to three separate tabs within an Excel workbook. As always, I was using multiple Block Until Done tools to prevent write collisions to the same file, but I kept receiving the error associated with that. I noticed that the Block Until Done tools were functioning individually, but not as they normally do in a 'group' (i.e. higher tool # waits until the previous is complete). When I disabled the AMP Engine for the workflow, the execution happened as normal and desired -- so I am very confident that the AMP Engine was the driver of the issue that I was seeing.

I am not sure if this falls into the bug or feature category of the AMP Engine, but I didn't see documentation on it yet, so I figured I would make a post just in case someone is running into the issue as well.

Respectfully,

Mike

Workflow

Developer Tools

AMP Engine

AMP-Engine

Accepted answers

All comments

Tonya_Smith

Hi @MRAgamir

Thank you for trying it out and for providing feedback on the AMP Engine.

I encourage you to continue to report any use case issues that you find with running workflows with AMP Engine enabled. We worked hard to identify differences from the original Engine as well as provide guidance on how to better optimize workflows to run with AMP.

If you haven't already, please review our articles related to using the new AMP Engine:

https://help.alteryx.com/current/designer/alteryx-amp-engine

https://help.alteryx.com/current/designer/Alteryx-Engine-and-AMP-Main-Differences

https://help.alteryx.com/current/designer/AMP-Memory-Use

https://help.alteryx.com/current/designer/tool-use-amp

Especially the https://help.alteryx.com/current/designer/Alteryx-Engine-and-AMP-Main-Differences mentions Block Until Done with AMP. By chance does your workflow contain multiple parallel branches? AMP will run those simultaneously since it is multi-threaded. The specific details about BUD with AMP that we have documented are:

Parallel Branch Execution and the Tool Run Order

Some workflows read from a file and then write back to it. This requires sequencing to ensure that the read is complete before the write can start. Similarly, a workflow that wants to write several sheets in one .xlsx file needs to write the sheets one at a time. Alteryx Designer provides a Block Until Done (BUD) tool to help partition the work into phases that won’t get in each other’s way.

When working on a workflow with multiple branches (largely separate streams from inputs to outputs), place the BUD tool in the workflow branch with the lowest numbered Input tool ID. This ensures every subsequent branch waits to run until the previous branch is done and the tool works as expected.

There are additional articles and documentation available about AMP:

https://community.alteryx.com/t5/Analytics/Accelerate-Your-Analytic-Processes-with-the-New-AMP-Engine/ba-p/574809

https://community.alteryx.com/t5/Engine-Works/AMPlify-your-Workflows/ba-p/617590

https://community.alteryx.com/t5/Alteryx-Designer-Discussions/Best-practice-with-the-new-Multi-threaded-processing-AMP/m-p/603329#M134951

AlterEverything Podcast: https://community.alteryx.com/t5/Alter-Everything-Podcast/66-The-Alteryx-AMP-Engine-Explained/ba-p/599058

https://community.alteryx.com/t5/Engine-Works/AMP-Engine-Technical-Deep-Dive-Part-1-Why-AMP/ba-p/570964

https://community.alteryx.com/t5/Engine-Works/AMP-Engine-Technical-Deep-Dive-Part-2-Key-concepts-of-the-AMP/ba-p/611500

Please let me know if you still have questions and we can look into your issue further. It would be helpful to have an example workflow to illustrate the specific behavior you are seeing.

MRAgamir

Hi @TonyaS,

Interesting -- in that case it appears to be an issue with something else in my workflow that is causing the issue, however the issue only happens when I turn on the AMP engine. For example, this basic test workflow DOES work with AMP enabled:

...but my workflow with a lot more preceding logic receives an error when running with AMP:

The error is the pretty basic: "Unable to open file for write: ... " -- there is no error if the workflow is ran without AMP.

Respectfully,

Mike

DaveWeir

Hi Tonya

Any update on the branch ordering point? (i see you wrote this post a while ago).

Seems i'm still having the same issue as most here in this post (the new AMP engine also seems to handle blank fields differently and had to reconfigure a lot of select tools)

Mike - on the block until done issue, i had the exact some road block and eventually fixed it by combining the CReW Parallel Block Until done tool (which alone with the new AMP engine - also doesn't work) and insert normal BUD tools in between (see attached).

Paralell BUD with normal BUD (AMP engine workaround).png

Would be awesome if this could be handled more simply as often people want to write back data from multiple sources into in excel file across multiple tabs.

BGirkins

Same problem. Tons of issues with this tool after switching to AMP. I'm seeing behavior where no data continues past a block until done tool. It just stops, no data makes it past. Which is of course breaks configurations downstream as it "forgets" the fields it used to remember.

Tonya_Smith

The data not passing the Block Until Done was fixed for 22.1 - are you on an older version? There was a big push to address almost all known AMP defects before the 22.1 Release that went out on May 4th. With that release AMP is the default for new workflows that users build in Designer.

BGirkins

Oh nice! I was not aware and am on 2021.4 (updating this week).

BGirkins

Unfortunately the problem persists even after the upgrade. The module just stops running at that tool.

Tonya_Smith

Would you mind please going through Support so that they can submit a ticket for us to look at fixing the specific issue? If you can provide them with a small workflow to reproduce the issue that would be very helpful.

BGirkins

Yeah I'll do that next. I think it is because this particular module has the Block Until Done tool before a batch macro. I have other modules that are not having issues with the BUD tools.

hartlogan

Hi Tonya,

Do you have any update on the Branch Ordering piece? With 22.1, our users are now being prompted to turn on the AMP engine for older workflows and we are seeing this error frequently. Additionally, do you have an example of what the below suggestion looks like in practice? We've tried implementing this a few different ways, but haven't gotten it to work and wonder if we're interpreting it wrong. In the particular workflow we are testing, we have 2 input tools that branch into 8 different branches (Input 1 is used in all branches and Input 2 is used in only two branches).

"When working on a workflow with multiple branches (largely separate streams from inputs to outputs), place the BUD tool in the workflow branch with the lowest numbered Input tool ID. This ensures every subsequent branch waits to run until the previous branch is done and the tool works as expected."

Logan

Tonya_Smith

We still have not received a Support issue on this.

@BGirkins or anyone else who can provide a workflow and scenario could you reach out to Support to get this logged with them and it will be escalated to my team.

BGirkins

Thanks for the follow up! I have a broader case opened on a number of issues related to AMP. But right now everything is stable (turned off AMP on any modules with macros and BUD tools) so until we can upgrade to 2022.1, the case is on hold.

SailAway

Has this issue been resolved? I'm using Designer 2022.1.1.40869 and was recently prompted to use AMP which caused my 6 parallel excel tab outputs using BUD to get the same error as above.

RyanWade

Now in March 2024, I am seeing the opposite effects take place in my workflow. With AMP turned on, Block Until Done is starting at the 1st anchor and everything is working in the order I expect. When I turn off the AMP engine, anchor 3 takes place before any other steps. I am not sure if this error was addressed in the past or previous update, but am posting to see if anyone after 2023 is seeing different results than these older posts.

Rob48

I am also having BUD problems when running with the AMP engine. I use Version: 2022.3.1.508 Patch: 4 and with the AMP switched on in the configuration box no results come through after my BUD tool. I've "solved" the problem by switching off AMP but what gives? if I leave AMP off am I really losing any functionality?

RyanWade

Following up with the BLOCK UNTIL DONE tool

Where I work and use AMP, that solution of turning AMP off was the way it was resolved for my colleagues with a 2022 version (not sure exact)

However, I currently use Alteryx Designer 2023.2.1.51 - Patch: 1 - and I have been able to solve all of my Block Until Done issues by turning on both of the Configuration - Runtime options:

1. Use AMP Engine

2. Engine compatibility mode

amp on.PNG

When turned on, the multi-threaded process does speed up my workflows in a notable way. With this off, I would expect it to process slower but not detrimental unless you have a large amount of data.

Quick Links

This months top contributors

AkimasaKajitani 389

mceleavey 388

mbarone 337

Hollingsworth 335

LanisC 335