Hi everyone, We're encountering an issue during the implementation of a workflow and would appreciate your insights. Context: We're pulling data from two tables (MARA and MAKT). Occasionally, one of the two input of the union may be empty. When this happens, the Union tool's output is also empty, and the workflow fails to populate the expected data downstream. We've tried a few workarounds: * Using the CacheDataset Macro to materialize the data in Alteryx's cache. However, this requires running the workflow twice, which is not ideal given its complexity and runtime. * Disabling AMP engine or using an Action tool to cache the dataset in a temporary folder. Unfortunately, these approaches still result in an empty output unless both input streams to the Union tool are populated. We're looking for a more efficient solution that allows the workflow to handle cases where one of the input tables is empty, without compromising the output or requiring multiple runs. I've attached a screenshot to help illustrate the issue. There are also some try and workaround we tried! Thanks in advance for any suggestions or ideas!

Data issue union with an empty input

Hi everyone,

We're encountering an issue during the implementation of a workflow and would appreciate your insights.

Context:
We're pulling data from two tables (MARA and MAKT). Occasionally, one of the two input of the union may be empty. When this happens, the Union tool's output is also empty, and the workflow fails to populate the expected data downstream.

We've tried a few workarounds:

Using the CacheDataset Macro to materialize the data in Alteryx's cache. However, this requires running the workflow twice, which is not ideal given its complexity and runtime.
Disabling AMP engine or using an Action tool to cache the dataset in a temporary folder. Unfortunately, these approaches still result in an empty output unless both input streams to the Union tool are populated.

We're looking for a more efficient solution that allows the workflow to handle cases where one of the input tables is empty, without compromising the output or requiring multiple runs.

I've attached a screenshot to help illustrate the issue. There are also some try and workaround we tried!

Thanks in advance for any suggestions or ideas!

Datasets

Bug

Accepted answers

All comments

jrlindem

Could you consider using a Control Container to introduce conditional logic. If the stream is empty go one direction, if not go the other? Meaning, if no data, then don't union. If there is data then union?

Cutugnos

Unfortunately I can't use it, we're in a Macro envirnoment

jrlindem

Interesting. I'm not sure why being in a "macro environment" is prohibiting you. Are you saying that where this is occurring is within a macro you don't have access to edit?

For what it's worth, I'm not convinced you're not having another problem that's not associated with the union. The union tool should be outputting the values from either stream inputting it regardless of whether one or the other contains no values. I've tested this, best I can, on my end and my union tool is working fine when the two streams are populated, and/or when only one is...

I still think there's a path to leverage conditional logic to insert a dummy row of data when the input stream is found to be empty and then simply filter that dummy row out downstream, simply to ensure your union is allowing data through. But again, I'm not sure why this is necessary... I would recommend copying the workflow and removing out other parts to isolate down to the union tool in question and verify, for sure, that it's really that tool that's causing the problem.

Sorry i can't offer more specific solutions, but hope that helps. -Jay

dreldrel

Maybe use the block until done tools if you cannot use the control container?

Mathias_Nielsen

I agree with @jrlindem, we have solved similar issues with dummy records and then filtered them out after. We did not use conditional logic for this though, just did it for all inputs that might be empty

Gaurav_Dhama_

When you say that the data flowing into union is empty from one stream, is it because the container that passes the data is disabled or data is just filtered out due to filter or some join?

MinhLO

@Cutugnos This is a bug I raised to Alteryx a few months ago. Unsure if they are planning to fix it.

For example if a container is disabled before an union, then you will get an error message like 'No incoming metadata from one or more upstream tools'. Can you confirm this is what you see?

To solve this issue, we took two actions:

- on workflows where it was possible, we removed AMP.

- on workflows where it was not possible (for example we were using a custom tool with latest SDK, hence AMP needed), we segregated the workflow in two different workflows. One workflow would conserve AMP and create the dataset, the other one without AMP would consume this dataset.
However this is highly linked to our uses cases, hence it might not be reproducible for you.

In a nutshell:
- If you can, remove AMP

Cutugnos

@MinhLO

Hi MinhLO, we've already contacted them, I followed the steps you recommended but unfortunately it still doesn't work. We removed the AMP as well, the situation is the following, really simple, then one of the two Summarize is empty (and we don't know when, it is aleatory) the union doesn't reproduce an output? Do you have any ideas except:

use a control container, it is not feasible for this workflow split in two

use a container to disable the empty summarize, it still doesn't work and we don't know when the summarize is empty

use a cache, we should run twice every workflow

or we don't know...we're still looking for a workaround

thanks to everybody for the suggestions

Gaurav_Dhama_

@Cutugnos
Since i cannot see whats before the summarise tool, see if the below helps.

If you have a container that gets disabled based on some scenario, then any tool outside this container and before union will fail your union tool.

To handle this scenario, put your summarise inside the container which gets disabled. This should fix your issue.

MinhLO

@Cutugnos , based on your latest screenshot, it seems the issue is not with the union but with the tool right after.

what is the error message?

From what I guess - but lacking elements here - you are missing some columns from one of the two summarize, and randomly as you do not know which part of your workflow will have some.

Then what I would suggest is that you 'Ensure Fields' .

For that, create an input text with no rows, and columns names = all your column names from summarize one and two. (do not forget a select after to cast the correct data types)

Union this to your data. This will Ensure all your Fields are present, even if there is no data coming from one summarize.

Please try this - otherwise I need more details (error message, full picture of the workflow) to better understand your problem

anabel1224

Hola me pasaba el mismo problema , me funcionaba al correrlo localmente pero al cargarlo al servidor me salía ese error . Lo que hice fue desactivar en Runtime el "Use AMP Engine" y funcionó

jrlindem

@anabel1224's message:

"Hello, I was having the same problem, it worked when running it locally but when I uploaded it to the server I got that error. What I did was disable the 'Use AMP Engine' in Runtime and it worked."

Quick Links

This months top contributors

atcodedog05 19458

Qiu 15866

binu_acs 15708

MarqueeCrew 13708

apathetichell 13703