Output to file - and use file as input later in flow. Beeing able to cache data?

Hi

I have tried several different methods, without achieving what I want.

Basicly I have a group of supplier id's basic on the selected customer:

I want to generate this list of ids (SubCarrierNumberUB):

And use as input to a macro later in my flow so I only read data related to these subcarriers:

The issue is, my flow starts reading the Subcarrier data, before the flow has written a new file.

How to fix?

Macros

Common Use Cases

Accepted answers

All comments

ChrisTX

Try using a Block Until Done tool. Or from the CReW macros, use Parallel Block Until Done.

Chris

DanielMS

Hi @Hamder83,

You can achieve this buy using the Block Until Done tool.

This tool will block a workflow progressing until it has completed all of the steps prior to the tool.

https://community.alteryx.com/t5/Alteryx-Designer-Knowledge-Base/Tool-Mastery-Block-Until-Done/ta-p/33417

Kind regards,

Dan

Please like and mark this as a solution if this has solved your problem

Hamder83

Hi @ChrisTX I have a block until done in the flow.

I have to load some data, and process them before I can write the file.

And already before the data is processed it starts reading the 2. part?

ChrisTX

In your screenshot, you're reading in a YXDB file. Is that the data source that's being read, before another stream, earlier in the workflow, has finished writing to the same YXDB?

If yes, I've successfully used a Dynamic Input tool with a Parallel Block Until done to prevent reading a YXDB file that I'm writing to earlier in the workflow.

Chris

Hamder83

hi @DanielMS & @ChrisTX

Thanks for your replies.

My issue is, that I want to be able to cache the data, which I can't this way. (since its many gb its loading).
And I don't know before the initial data is loaded which id's to fetch and you for the lookup in the 2. part.

And I can't seem to find a workaround?

ChrisTX

I don't think I can help because I don't understand the issue, I don't understand exactly what you're trying to do. In the workflow screenshot you posted with the red boxes, we can't see exactly what you're trying to point out.

I'm guessing the workflow image shows that you're writing out to a YXDB file, then reading in that file in the same workflow, but you don't want to do it this way because the data stream is very large?

I'm making assumptions about the actual problem. Can you specify exactly what you're trying to do, step-by-step?

If you are trying to cache a large data stream, what testing have you done with the Block until Done tool? Is the tool not working right?

Maybe the problem is you're using the CReW macro for "CReW_LabelledBlockUntilDone.yxmc"? That macro seems to send all output from the output anchor 1, so I'm not sure how that one works.

Have you tried to use the native Alteryx macro for Block Until Done?

Chris

Hamder83

Hi @ChrisTX

I have attached a small workflow that simulates what im trying to do. But the 2. macro in that flow, im not able to cache, and I assume its because of the "dynamic" input.

Thats why I tried to either write to a file, and use the data in the file to load data into the 2. macro. I tried taking the data directly, but that didn't work either.

My main purpose was to be able to cashe the data from macro 2, and using input from the flow to load it.

Does it make sense?