topic Re: Alteryx Union With Dynamically Switchable Control Containers in Alteryx Designer Desktop Discussions

Alteryx Union With Dynamically Switchable Control Containers

bmillar777 — Wed, 05 Mar 2025 11:32:49 GMT

Hi everybody,

I've got a question that's been bothering me for a while and I haven't been able to find a solution to. I've included a screenshot with a simplified example of the issue.

It relates to a scenario where I have a set of PDF reads to do and have to dynamically come up with a solution to determine which ones. This works well using the Filter tool to turn containers on and off and if all processing is done inside each control container, all is fine.

However, if I need to combine the results outside the control containers, the Union tool fails with the "No incoming metadata from one or more upstream tools". I've tried various solutions to resolve this issue with no luck. Limiting the number of PDF reads is critical for my use case as one of my routines is already taking over 2 minutes to complete, unacceptably long.

Does anybody have any ideas on how I can resolve this?

Thanks :)

Re: Alteryx Union With Dynamically Switchable Control Containers

Warcry — Thu, 06 Mar 2025 03:38:55 GMT

Change the configuration to Ignore - Continue Processing Records

Re: Alteryx Union With Dynamically Switchable Control Containers

bmillar777 — Thu, 06 Mar 2025 09:11:24 GMT

Unfortunately, that didn't resolve the problem.

I've also tried to set "Auto Config by Position" and even to "Manually Configure Fields" and enabled the non-blocking option, but nothing from this angle seems to resolve it.

Re: Alteryx Union With Dynamically Switchable Control Containers

adighe — Mon, 24 Mar 2025 17:59:47 GMT

I have the exact same issue. Have tried all the options for the union tool. Amp on and off did not make a difference. Any clue as to what different approach to take in such situations?

Not all streams will have data. There are only two fields coming into the union tool with same names and datatype and size. I have set it to "Auto Config by name"

Error: Union (458): No incoming metadata from one or more upstream tools. Check the input data and run the workflow again

Re: Alteryx Union With Dynamically Switchable Control Containers

bmillar777 — Tue, 25 Mar 2025 10:38:41 GMT

I never found a solution in the end and maybe there isn't one. The closest I got was adding the Record ID tool to the Union inputs which removed the errors but didn't output any data :) It might be possible to use the Python tool to make it work, although I haven't had a proper go at that to see.

Re: Alteryx Union With Dynamically Switchable Control Containers

GMG0241 — Tue, 25 Mar 2025 11:09:06 GMT

Hi, I think the issue COULD be that it's due to having a split stream. Please see the attached screenshot/example which I believe should solve your problem by creating a single stream, although I'm not 100% sure as I don't have access to the Computer Vision tool set

Re: Alteryx Union With Dynamically Switchable Control Containers

flying008 — Wed, 26 Mar 2025 06:18:12 GMT

Hi, @bmillar777 @adighe

There are some issues that need to be confirmed first:
1- Do all your test*.pdf files exist?
2- Do all your PDF file paths have access permissions and are they set correctly?
3- Do all your PDF to Text tools have normal output? (The output here means at least an output header, or a standard header and data record, not nothing.)

If the answers to the above three questions are all yes, then under normal circumstances, the error in your screenshot will not appear. At the same time, the most likely problem is problem 3. For some reason, the PDF to Text tool will not output anything, so the Union tool will report an error due to the lack of metadata for one or more incoming connections.

**************
Solution:
1- You need to modify the existing workflow, add an Output tool after each of the three PDF to Text tools, and output each data to the file format you specify. Remember that the three output files should be placed in the same folder.
2- If the data structure of the three files is exactly the same, you can use the wildcard option of the Input tool or the Dynamic Input tool to Union all the data, so that the above error will not appear and affect your work.
3- If the data structures of the three files are not exactly the same, you may need to use a batch macro to complete the above Union operation. There are too many such macros in the Gallery, you can download them directly and use them.
4- You can also use 1 Control Contain to control the order of the above two steps relative to the three PDF to Text tools, which should be a very easy thing for you.

That's all, I wish you good luck!

Re: Alteryx Union With Dynamically Switchable Control Containers

adighe — Wed, 26 Mar 2025 14:08:29 GMT

@flying008,

In my case it's not files but the download tool with various DB or API calls that might not be executed. That causes errors before the flow reaches the join. The errors were hidden in control containers that collapse. But this causes the metadata to not reach the union. I have to figure out how to pass metadata or dummy data through every path that reaches the join and then remove it.

Re: Alteryx Union With Dynamically Switchable Control Containers

apathetichell — Wed, 26 Mar 2025 14:24:13 GMT

can you share a screen shot of the config of your union tool? union should not require a connection is initialized - but I've seen this be sticky. I think some things to try are:

1) delete the union - re add.

2) in the containers create a text input tool with a dummy column name and no data. union this in your container. run.

the above is in standard Alteryx pre-control containers. It's possible that the control containers deploy is a bit 'special' -> if that's the case maybe throw things in a macros instead of control containers?

basically - what you are trying to do is resync the workflow cache.

Re: Alteryx Union With Dynamically Switchable Control Containers

adighe — Wed, 26 Mar 2025 16:32:52 GMT

I have used all configs for the union tool with no luck, and current config is auto config by name. Adding the dummy data is what will work. The downloads in red may or may not execute and that results in the formula/filter downstream to error due to missing variables. I added the missing dummy variables with a formula marked in green. That took care of error in that stream. Any errors upstream with or without containers will prevent metadata to flow downstream and union does not like that.

@apathetichell adding text input is a better idea as its very obvious what the purpose is.

Re: Alteryx Union With Dynamically Switchable Control Containers

apathetichell — Wed, 26 Mar 2025 17:49:00 GMT

@adighe- I use text input - and I'd actually use a field name which must be in the union tool (ie something from your parsed download tool data). It does not have to have rows -> it just needs a field name and it will populate into the union tool.

Reiterating - old school this worked fine -> control containers may be 'unique'

Re: Alteryx Union With Dynamically Switchable Control Containers

bmillar777 — Wed, 26 Mar 2025 18:19:34 GMT

Unfortunately, I don't think any of these solutions will resolve the original problem, I think I've tried them all already. Let me define the issue with bullet-points.

* The original purpose was to read as few PDF's as possible. If the problem could be solved by reading PDF 1, then that's good. If not, then we need to read PDF 2, and then PDF 3 and so on. Containers are necessary for this problem as the PDF tools expect an input if they are not inside a disabled container.

* The screenshot I provided was a simple example I constructed to demonstrate the problem. The following is how things work depending on the situation:-

1) The three Filter tools turn the containers on and off automatically depending on the data in the Text Input tool. However, if we ignore those tools, the same problem still applies. The real issue is with the containers and the Union tool.

2) If we leave all the containers switched on, the workflow runs perfectly. However, if we disable even one, the workflow fails reporting that no "incoming metadata could be found from one of the upstream tools"

2.1) It seems that having no input at all from a disabled container is a problem for the Union tool which generates the error.

2.2) No setting changes in the Union tool change the issue, not even the non-blocking one.

3) The problem doesn't exist if all processing is done inside an individual container. For example, having three Union tools, one in each container doesn't generate any error. But then you can't rejoin the rest of the workflow to do further processing. In my use case, that's a problem as after reading one PDF, I need to exit into the workflow to determine the next PDF container to switch on before doing further processing.

4) Using any kind of macro isn't a solution here for the same reason as the Union tool, you need to leave the container to exit the macro and that can't be done without error.

I think a possible solution is to use nested workflows. Then you could remain in the container, write an output and then read it in the next workflow etc. But it's a very complicated solution which I didn't try. Another possibility is to use nested control containers, but if you have ten PDF's to read, that's going to be very ugly :)

Re: Alteryx Union With Dynamically Switchable Control Containers

apathetichell — Wed, 26 Mar 2025 23:33:51 GMT

Hey @bmillar777 - sorry was confusing the two tool sets - I think part of your problem is that the PDF tool you are using uses the Python tool. The Python tool wants to run - > even with no data.My hunch would be:

1) disable all containers.

2) create a text input with whatever downstream schema you need.

3) run -> confirm that it does not error.

4) set up your containers to optionally RUN - not optionally NOT RUN. Re test.

If that does not work -> I'd try 1) putting each container in a separate macro.

2) going old school and using disabled standard containers vs control containers with them set to run with data (and their standard setting being disabled) - again in a (Batch) macro -> albeit you could use one.

Re: Alteryx Union With Dynamically Switchable Control Containers

flying008 — Thu, 27 Mar 2025 03:34:19 GMT

Hi, @bmillar777 @adighe

1- No matter you use PDF to Text tool or Download tool, they will cause no output for some reason. So the problem is not the control container, but the fact that as long as one of the incoming connections of the Union tool is an empty connection (no metadata is passed in by the upstream tool), an error message "incoming metadata could be found from one of the upstream tools" will be reported.

2- So you need to analyze which step before the Union tool may have no metadata output, either create a text input with whatever downstream schema you need as @apathetichell said, or use dynamic input to process the previous data later as I said above.

3- In fact, there is another solution here, which is to use the Branch plug-in similar to Until ...to or If...then logic to implement the branch process. It does not require the use of the Control Contain tool, does not rely on the AMP mode, and does not require the use of multiple nested macros. But since your company may restrict the use of such 3rd external tools, we will not discuss it in detail.

Re: Alteryx Union With Dynamically Switchable Control Containers

bmillar777 — Thu, 27 Mar 2025 09:43:10 GMT

Hi all, thanks for your replies.

@apathetichell, I'm not sure what you mean by optionally RUN, I assume you mean disable the control containers and then enable them using the Filter tool. If that's what you mean, it doesn't unfortunately work as the Filter tool won't enable a disabled container.

@flying008, a control container is necessary for this procedure as the PDF tool needs an input if it's not disabled. So when it's disabled, it will always lead to an empty connection in the Union tool. Adding a Text Input tool to add input doesn't work as any joins or unions will simply receive no input from the disabled PDF tool. It's a circular problem.

A dynamic input solution could work, it's not something I've tried to date. I don't think we can use third party tools, but this is something I've been wondering about lately and might ask the management.

I'm actually no longer working on this problem so I don't need an answer to this question any more, I found the time delay in reading all PDF's for my implementation was not too prohibitive so I went with that. But it's an interesting case to face in Alteryx development.

Re: Alteryx Union With Dynamically Switchable Control Containers

apathetichell — Thu, 27 Mar 2025 16:03:14 GMT

@bmillar777- I just assumed I was a curmudgeon for hating on control containers - looks like the product has issues. So back to old school containers.

1) you'll need a batch macro.

2) you'll need an old school container in the macro (set it as disabled).

3) you'll turn your filter tool into a boolean.

4) you'll add a control parameter

5) you'll hook up an action tool -> it will feed from the control parameter to your container

6) - I prefer update value with formula here not enable/disable container.

7) you'll update the value which says disabed = true (or enabled=false or whatever it says) using a formula like ![#1] or [#1] whichever you are using based upon your boolean logic.

Test this with one. If it works -> replicate to all of them (or build out the dynamic logic so one macro can process them all. That's your call. I am high 90s% sure that you can do this -> because I have done this. Albeit on like 2021.3 so who knows.

I'd show you a sample but 1) I don't use that PDF tool 2) that is moot because I currently do not have an Alteryx license and do not use Alteryx at my 9-5.

Re: Alteryx Union With Dynamically Switchable Control Containers

bmillar777 — Wed, 02 Apr 2025 16:16:28 GMT

If I understand your meaning correctly, this is the kind of setup you mean?

If so, this does not work because if the container is turned off by the Boolean filter in the workflow, the workflow fails with "The output of the macro is not valid". If the container is turned on, it works fine.

Re: Alteryx Union With Dynamically Switchable Control Containers

apathetichell — Wed, 02 Apr 2025 16:25:38 GMT

disable the container. The container in the macro should start as disabled.

the action tool should enable the container when you want it to run (and should have required input fields.).

Re: Alteryx Union With Dynamically Switchable Control Containers

NeoInfiniTech — Wed, 02 Apr 2025 19:40:21 GMT

An alternative solution to this that doesn't require the use of any macros could be turning this into an analytic app, where the full path would have to be provided through a Text Box or File Browse tool from the Interface category, then an Action tool originating from the Interface tool of choice would connect to the Text Input tool to modify the full path provided, while another Action tool that would also originate from the Interface tool of choice would connect to the Union tool to modify the raw XML of the Union tool. If the file names are standard and RegExable in terms of dynamically determining which container(s) would be disabled or enabled, then this information could be used to dynamically delete the incoming metadata line (#1, #2 or #3) and/or any other configuration lines from the Union tool's raw XML.

If you are not familiar with the XML structure of the workflow, you can open the workflow file with Notepad or any other text editing program and/or enable displaying the XML of individual tools from User Settings in Alteryx Designer.

I could prepare a sample workflow but I'm away from my computer right now.

The only caveat with using this method would be to use the Debug Workflow option provided in the Interface Designer each time if you wish to see the progress directly for performance profile purposes.

Re: Alteryx Union With Dynamically Switchable Control Containers

apathetichell — Wed, 02 Apr 2025 20:03:15 GMT

@NeoInfiniTech- I'm fairly sure (high 90's confidence) that this won't work for python tools.