Union sporadically breaks when an incoming stream doesn't have data??

[Alexandra_Mannerings]

I have a bunch of analytic apps that rely on unions to manage variable numbers of incoming data streams. This has always worked in the past for me. Suddenly and seemingly randomly, it has stopped working for some of these workflows (but not all?). 

The problem looks like this:

The data just stop and don't flow through a union when stream #2 has no data . But if input #1 doesn't have data but #2 does, the union works fine and passes data through. So I feel like it can't be the union configuration? But I don't know what else is wrong. A week ago this worked fine. I was on 2023.2, and hadn't updated or changed versions, or even added a patch when it stopped working. Since then, I've tried upgrading to 2024 but that didn't fix it. If I copy the section of workflow that isn't working and paste it into a new workbook, it still doesn't work. If I replace the incoming data streams with yxdb copies of the data at the point they go into the union, the union works fine regardless of which streams are 'active' with incoming data and which streams don't have data. So it's not the contents of the data that make it not work. 

I tried the configuration settings recommended in this post (Union when no data is available - Alteryx Community) but that also did not help.

I'm totally stumped. I've opened a support ticket a week ago, but haven't gotten any answers yet. 

[danilang]

Hi @Alexandra_Mannerings 

If a workflow has worked consistently and suddenly stops, the reason is probably a change to the input data.  I found a new issue just last week where a Dynamic Rename set the Take Field Names from 1st row gives a Out of Memory warning if there are no rows in the input...but only in AMP mode.  The workflow doesn't error.  It just doesn't output any data.  In non-AMP, the Dynamic Rename just passes through the original field names 

Are you getting any errors upstream of the union?

Dan

[Alexandra_Mannerings]

Nope, no errors upstream. The reason an incoming data stream might not have data would be because that stream was deactivated through a closed container from an app selection. 

[danilang]

Hi @Alexandra_Mannerings 

In your debug workflow, look at each of the inputs to the union in turn and look for one where the results say "Memory limit reached".  If there is one, trace upstream and see where the error may be produced.  If there isn't one, make sure that at least one of the streams has records in it.

Try running the workflow in non-AMP mode and see if the error persists.

Dan

[Alexandra_Mannerings]

I never use AMP, so that wasn't the issue. I've confirmed that when I run it, there are no points at which I get "memory limit reached" in a Browse anywhere. And there is definitely data in one of the two streams:

[caltang]

Why not use AMP? Have you tried with AMP - any different results?

[danilang]

hi @Alexandra_Mannerings 

The select tool that is just outside of the bottom container could be causing issues since when the container is disabled, it has no metadata.  Try moving it inside.  The general idea is that all tools up to but not including the union should be disabled.   

@caltang: As to why not use AMP?  Converting workflows that were developed with e1 to use AMP is a long and time consuming process.  There are also changes in functionality caused by AMP that are difficult to work around.  I found one of these just last week.  If you have a Dynamic Rename set to take headers from first row of data and there are no rows, AMP will silently give a Memory Limit Error and pass through no metadata causing errors further in the workflow.  In e1, if there are no rows, the Dynamic Rename, does nothing and passes the metadata with the original column headers.  In general it's easier to just continue using e1 for existing workflows and possibly use AMP for new ones.   

Dan

[apathetichell]

Can you post a screenshot of your config of your union tool? Curious if it's something to do with auto config by position or manually configure fields...

[caltang]

Thanks @danilang ! I didn’t know the issue extends that deep. I understand better now.

[Alexandra_Mannerings · Accepted solution]

Yeah, to build on @danilang, I've just found that AMP breaks random things and I get tired of figuring out what it did wrong or worse yet NOT realizing it's messed something up and then moving a flawed workflow into production. I've had past versions of AMP screw up the outputs from Summarize tools for example - the workflow runs without errors, but the actual values out of the Summarize tool were wrong! I think they've since fixed that, but it's hard to trust AMP. 

Regarding the issue with the union, I've finally solved it. It happens if there is any tool between the deactivated container and the union. Unions need to at least get told "there's no data here" or have EVERYTHING along the stream without data deactivated. I had a select tool outside the container and before the union, and that was what was breaking it. So you can have a stream with no data going into a union, but it has to SHOW 'no data' (like, say 0kb on the data stream) so the union isn't expecting anything. If you have a deactivated container but tools outside it after that, the tool doesn't have any metadata to share, and the union just waits for that.