Block until done

How is this possible? Shouldn't the second macro be BLOCKed (0%) UNTIL the first macro (79%) is DONE? How can the second macro be 28% finished while the first is still running?

I've validated my data and the flow did run in sequence but I'm curious about what displays on my screen when the job runs.

Workflow

Macros

Accepted answers

GarthM

i must've misspoke but I should be forgiven 'cuz, at this point, my knowledge -more truthfully, my guesses- are riding the edge of a lightning bolt with how i'm describing things.

anyway...

it's my understanding that upstream tools, and by extension their output anchors, care A LOT about downstream processes of the tool(s) they're passing data to. naively speaking, in order for Alteryx to finalize an "output connection process" on a given data stream, downstream tools must communicate to their upstream tool counterparts something along the lines of: "everything is ok. feel free to close out and head home." given that the 'Wait A Minute' macro must exit itself before output #2 "opens the gate" has me believe that a string of cascading dependencies exists amongst the downstream sequence of tools connected to the BUD output #1. another one of my shallow understandings is that the Output tool requires a complete profile of the data flowing into it before the engine will initialize the incoming connection to it. if what i know is accurate, and my suspicions about the dependency cascade is also true, then the BUD can't ask the engine to initialize the Output tool that writes to out3.csv until output #2 is fully aware of the size and structure of the data it will be passing down to it.

i hope this makes sense.

All comments

MarqueeCrew

@roadhouse,

I look at the block until done as a queue of row data. Once all of the records are read from the up-stream data, then the first gate is opened. Once all of the data has passed into the first gate, the data is passed to the 2nd gate.

If you want to see all of the data go through macro 1 and then go to macro 2. I would do the following:

After macro 1, get a count of all of the output rows. You won't actually use the data, it just is a process that requires macro 1 to complete.
Put the count of all rows (a single record) into an APPEND FIELDS tool (source or bottom). You will put the input data (same data that went into macro 1) into the append fields. Now configure the APPEND FIELDS tool to un-check the count field from the source.

What will happen now is that macro 2 can't start until macro 1 is done. Take the output from the append fields tool into macro 2.

Cheers,

Mark

roadhouse

My question is why do I see a completion percentage on gate 2 while gate 1 is still running? The workflow pictured is simply my hacky way to manage flow control (the Achilles Heel of Alteryx IMO). The input is just one dummy row and the first macro makes an insert into my db while the second uses the newly inserted data.

Like I mentioned in my first post, the job is running perfectly in sequence, I am more curious about the percentages being displayed. It's not how I would expect Block Until Done to work and I had to do a bunch of validation to make sure things happened in sequence.

jasonvassallo

Wow, thank you so much Mark! While this may not have directly answered roadhouse's question, you just helped me solve a puzzle that was mind-boggling! This is exactly what I needed to do to get a daily workflow to make a "yesterday's" copy of a yxdb before it writes over it with the new data for today.

Jason

mcarrico

@MarqueeCrew

I am trying to troubleshoot some Block Until Done issues that I have run into similar to what @roadhouse was seeing in his original post. I have heard the BUD functionality explained similar to your gate description, but have found certain instances where it appears this is not the case (see attached video).

The gate explanation does not seem to hold here given that out3.csv is not written as soon as the the data clears Gate 1 and hits the Select tool. It actually does what I would expect the BUD tool to do, which is wait until the workstream outputting out2.csv completes before kicking off Gate 2 and outputting out3.csv. That being said, I have seen the exact scenario occur from the original question in this thread.

In my case, I have macros that conclude with a Run Command, but no Output Data tool. My next testing scenario is to modify some of these macros to have dummy output to test the theory that all outputs on a branch have to complete, but I wanted to see if there was an explanation for the behavior shown in this video in light of my current understanding of BUD behavior. I have also used the append fields tactic in the past, but this can get a bit busy when trying to control order of executions in a complex workflow.

Thanks,

Mike

BUD_Gates.7z

Quick Links

This months top contributors

atcodedog05 19458

Qiu 15866

binu_acs 15708

MarqueeCrew 13708

apathetichell 13703