Engine Works

Under the hood of Alteryx: tips, tricks and how-tos.
RithiS
Alteryx
Alteryx

 

If you’re an active member of our Alteryx Community, then you may already know that the top Alteryx Designer idea is Control Containers. Posted in 2015, the idea has gathered 270+ likes, 60+ comments and has caused many hours of discussion. It’s been a long time coming, and we are ecstatic to introduce your new favorite tool… Control Containers!

 

Using Tool Containers as a foundation was clear from the beginning, as evidenced by the Community idea. Over 70% of Designer users ran a workflow with a Tool Container in January 2023. There is high familiarity with Tool Containers as they are easy to use for workflow documentation and organization. It was quite a challenge to dive into the container and tool architecture, combine them, and a data processing container tool, but we are very happy with the results!

 

Two important notes: (1) Control Containers work on Server, and (2) they only work with AMP engine.

 

Enterprise Readiness & Benefits

 

New Alteryx users may be asking ‘why Control Containers?’ Simply, users want to be able to control the order of a workflow’s execution. Remember that working in Designer means working with a visual programming language. In programming, it’s vital that your logic executes in a specific order or that only specific conditions trigger downstream logic. Control Containers give you, the visual programmer, new abilities to solve even more complex data and business problems.

 

Enterprise organizations will receive significant value with the new tool. You’ll be able to orchestrate sections of the workflow, integrate with other processes where the order of events is critical, improve testing and auditing processes, and productionize sequenced logic in simpler workflows without having to learn concepts like batch macros and chained apps.

 

Let’s get into the basics and review simple examples now.

 

Control Container Basics

 

Like Tool Containers, you can place tools and sections of the workflow inside to improve organization, but Control Containers come with one input anchor and output anchor each. The anchors are called Control and Log, respectively.

 

image-20230224-203657 (1).png

 

Control (Input) Anchor

 

The Control anchor is how the Alteryx Engine knows when to trigger a Control Container execution. Connecting data to the Control Container will trigger it to execute once all data has been received from upstream (and only if at least one record has been received). This is called activating the Container. Once all the data has been received by the Control anchor, the tools within the Control Container will run. After those tools are completed, data is sent through the Log anchor.

 

This figure shows a Text Input connected to the Control anchor. The container will only run when it receives at least one data record.

 

image-20230223-181439 (1).png

 

 

 

The Control anchor is optional. If there is no incoming connection, then the Control Container is treated like a Tool Container and runs in the order normally determined by the Alteryx Engine. However, it will still output data through the Log anchor. If there is an incoming connection, but no data is received, then the container will not execute. I will show an example at the end of this post.

 

Log (Output) Anchor

 

The Log anchor sends out the engine messages of the contained tools as actual data. This allows messages to be used in downstream tools for activities like auditing and testing.

 

This figure shows an Input Data tool loading a dataset that is passed to a Browse tool. The Log anchor’s data output contains the messages generated by both tools. The data object includes ToolID, MessageType, and Message. ToolID maps to a tool’s ID (surprise!), MessageType maps to the tool’s engine name (which can be viewed in the workflow XML), and Message gives you information from the tool itself (the same messages that normally appear in the Results grid).

 

image-20230313-233449.png

 

Data from the Log anchor can also be used to trigger a second Control Container. This figure shows a Text Input triggering the first container, which then triggers the second container.

 

image-20230224-184005.png

 

For full technical details, check out the Alteryx Help documentation.

 

Use Cases & Examples

 

Now that you have a basic sense of how Control Containers work let’s explore some simple use cases. These examples are also attached so you can download and play with them. They should run successfully with no configuration required.

 

Use Case #1: Writing multiple outputs in a preferred sequence

 

This first use case is like the previous example, but here we are writing out three yxdb files based on our Filters. As you can see in the Results messages, the containers run in order. This is very helpful when writing separate sheets in a single Excel file or following sequenced database operations.

 

image-20230224-191930.png

 

Use Case #2: Writing a new file and using it within the same workflow

 

This example shows that you can easily write a file (existing or new) and then bring that file back into the same workflow and use it downstream. This is primarily helpful if you need specific orders for database operations and the same tables.

 

image-20230224-193139.png

 

Use Case #3: Conditional branching

 

This example checks a dataset for null values. If there is a null value, then the bottom container runs. If there is a connection to the Control anchor, but no data is sent, then the container does not run. This can be used to send alerts or optimize workflow run time. The first figure shows that the bottom container does not execute.

 

image-20230224-202716.png

 

This second figure shows a null in the dataset, so the bottom container executes. Also, notice that each container activates whenever it gets a data record and does not care about the order since they branch from the same tool.

 

image-20230224-202807.png

 

That’s it for three simple use cases! Imagine the possibilities you and your colleagues can do. Be sure to download Alteryx Designer 2023.1 today and let us know how you are using Control Containers!

 

Rithi Son
Product Manager

Rithi started at Alteryx in March 2016 as a product engineer before becoming a product manager in 2019. He has worked as a business and data analyst in ecommerce and health care business intelligence utilizing Excel and SQL. Rithi lives in Denver enjoying life in the Colorado front range.

Rithi started at Alteryx in March 2016 as a product engineer before becoming a product manager in 2019. He has worked as a business and data analyst in ecommerce and health care business intelligence utilizing Excel and SQL. Rithi lives in Denver enjoying life in the Colorado front range.

Comments
maria_ho
Alteryx
Alteryx

Control Containers!! 

SPetrie
12 - Quasar

This is awesome!

 

fmvizcaino
17 - Castor
17 - Castor

YESSSS!!!!!

 

fmvizcaino_0-1684352390298.gif

 

AkimasaKajitani
17 - Castor
17 - Castor

This is exactly what I want!!

kkjbk
5 - Atom

This will be fantastic, but will this also work on server? We have a lot of apps deployed to our server which currently use workarounds to achieve this sort of functionality and it would be great if we could simplify them to leverage this development!

akadwa
7 - Meteor

Best news of the year 😁

RithiS
Alteryx
Alteryx

@kkjbk, Control Containers will also work on Server like most other tools. If there's any issues with that, please let me know and we can help resolve it.

DataNath
17 - Castor

This is huge. Great addition team!

Charity_K_Wilson
10 - Fireball

This is really great.  My teammates and I have struggled with how to sequence things.  I have teammates who use Alteryx for various things related to our database.  We've gone thru batch macros, events upon sucess, block until done, and parallel block until done.  I didn't realize there was a whole thread on Community or I would have been there adding my voice.  We have cobbled together solutions.  Solutions that had limitations, that broke when we tried to max them out.

jrobiso2
8 - Asteroid

When is this new, incredible version being released as GA?

jrobiso2
8 - Asteroid

Hmmm . . . will Control Containers be available for use inside Macros?

NicoleJ
Alteryx
Alteryx

@jrobiso2  - It's already released! Check out the (newly revamped!) downloads portal for the 23.1 GA release that includes this great feature (and so many more!!)... and yes, it can absolutely be used inside a macro! Use it just about anywhere you'd want to use a regular Tool Container, and about a million other places & use cases too 😁 (Just keep in mind - this is an AMP-only tool, so you'll need to ensure you have that setting enabled.)

 

Enjoy!

Eli_B
10 - Fireball

I was getting so excited about this, then just read it requires using AMP...which I still don't trust due to bugs/inaccuracies when comparing results to normal engine. This is crushing news!

At least there is a dark mode!

EmilyVA
Alteryx
Alteryx

So cool!  I can think of so many times when this would have made my life better!

Malsadeh
6 - Meteoroid

I love this!

KimLamNg
8 - Asteroid

This is amazing!

jrobiso2
8 - Asteroid

@NicoleJ "AMP-only tool"?

 

Not sure if you are aware, but AMP breaks a lot of API uses (usually around issues between AMP and Proxy settings).

 

Essentially, I can't do AMP with API calls if I am inside a corporate firewall. Only if I am outside.

 

Is there no way to remove the AMP requirement?

cplewis90
13 - Pulsar
13 - Pulsar

This is game-changing! I can't wait to get to this version to test it out.

RithiS
Alteryx
Alteryx

@jrobiso2, there are currently no plans to remove the AMP requirement for Control Containers. We will track your feedback, but please submit a Community Idea for additional user support if this is a larger blocker than realized.

Eli_B
10 - Fireball

@jrobiso2, I'm with you, AMP breaks more than just API, I would upvote a version that doesn't require AMP usage.

mkosmicki1
8 - Asteroid

I'm having issues manually running WFs on server (via gallery) that have control containers. I'm not entirely sure why. I'd like to test the WFs before the Schedule runs them. 

I did test locally in designer and everything ran. I'm testing the containers with Runner tools to only let one runner execute at a time, to see if writing out my hyper files to Tableau Server will work if the runners are containerized.

NicoleJ
Alteryx
Alteryx

@mkosmicki1 If I'm not mistaken, Runner tool contains warnings about using in Server/Gallery, particularly with AMP, so I wonder if the issue might be with Runner tool + AMP rather than Control Containers? https://help.alteryx.com/20231/designer/runner-tool 

mkosmicki1
8 - Asteroid

@NicoleJ You are not mistaken. However, it is the only way we can run WFs in a specific sequential order. 

The WFs do run without the control containers...but the Tableau Output tool is not writing out to Tableau Server. I was hoping to be able to use the control container to keep the runner from initializing before the previous runner was fully completed. It works beautifully on my local computer though!

TrangPham1209
5 - Atom

Hello,

Thank you for this exciting new feature.

However, at my first attempt to use the control containers, it doesn't work with a regular container inside and would show error "The connection destination (name=input) is not valid"

Have anyone know about this error? And is there a way we can fix this?

 

TrangPham1209_2-1688024521337.png

 

TrangPham1209_1-1688024511857.png

 

NicoleJ
Alteryx
Alteryx

That’s a known issue we are working to fix! In release notes you’ll find several known issues listed at the bottom of the page - we will update in patches and/or future releases when fixed. 

Keep in mind that Control Containers can also be used as regular containers if you don’t connect them to other tools, so you can temporary resolve by nesting a Control Container inside rather than a Tool Container. 

 

if you find new defects, you can also always submit them through Support to help bring to our attention! Thanks!

caltang
17 - Castor
17 - Castor

I received some warnings about using Interface Tools and control containers... I was really hoping to stop the Chain Apps and use control containers fully... 

 

But so far, exporting sequentially is a great benefit. Happy about that

marcus_augusto
5 - Atom

i will try this !! 

amtraker1777
7 - Meteor

@TrangPham1209 Why would you want to place a regular container within a control container? Why not just rename the control container? I'm just curious what your use case is.

hroderick-thr
11 - Bolide

@jrobiso2, Let us know when you submit a community idea to remove AMP requirement so we can vote up

jrobiso2
8 - Asteroid

@hroderick-thr No need any more. 

 

The issue with AMP and proxies has been fixed in more recent versions of Alteryx. Unfortunately for me, I'm stuck on 2021.4, so I'm still waiting for my company to upgrade the servers and then the Desktop package for us. 

trant114
5 - Atom

This is awesome, make running workflow/ writing outputs in sequence so much easier. Thank you.

Eli_B
10 - Fireball

THE issue with AMP is kind of funny, because it's not ONE issue with AMP, there are TONS of issues with AMP. My recommendation to everyone at my company is the first thing you do is go into user settings and turn the use AMP as default off. Not exaggerating, 80%+ of the issue resolutions I provide to my company are as easy as telling the user to turn off AMP & try again.

harsh_alang
Alteryx
Alteryx

Hello @Eli_B,

 

We have over the last year made lot of improvements with AMP. We would love to know more about your pain points with AMP and address issues that other users in similar situation may be experiencing. Please let us know and we would be happy to look into it. 

 

 

Thanks,

Harsh

Eli_B
10 - Fireball

Hi Harsh!

 

Basically, it doesn't seem like Alteryx has perfected bringing the separate threads back together for analysis, like sorting/joining, etc. Also, I do a lot of spatial analysis, & having AMP on has messed with those results as well, where if you compare the area of a spatial process done on normal engine with the AMP results, there will be differences. I can't offer up too many examples as I lost trust in it & have turned it off in my settings.

 

Have a great day!

harsh_alang
Alteryx
Alteryx

Hello Eli_B,

 

AMP has evolved into a very performant engine. There were some known differences that were expected in the Output between AMP and e1, to mitigate that we had introduced Engine "compatibility mode" setting when using AMP. 

Further we have released an utility that will provide valuable insights on a run for a workflow/Workflows using both Engines and provide users with analysis. You can find more info using the link.

Unleash AMP Prowess with Engines Parity Utility - Alteryx Community

Rahulkumarrk
6 - Meteoroid

This make running workflow/ writing outputs in sequence so much easier. Thank you

CSmith16
8 - Asteroid

Would be nice to have a solution for companies who still use older versions of Alteryx. We are on 2020.4 with no immediate plans in sight to upgrade. Desperately need a solution like this but for the more seasoned versions.

akadwa
7 - Meteor

@CSmith16 I'm curious as to how they could implement a new feature solution without requiring the application to be upgraded?.

 

CSmith16
8 - Asteroid

@akadwa I'm not talking about a retroactive fix, I am looking for a solution to the same problem, but for an older version. Would be nice to have one is all I am saying. A solution doesn't have to be official, it can be a workaround that a community member may have thoughts on.

hroderick-thr
11 - Bolide

@CSmith16  we use crew runner macros in old versions. These have their own reasons that Alteryx has never recommended them but they work in older versions and allow us to control the flow similar to the new containers. They continue to serve us well on 2023.1 but we are working towards changing them out for control containers. One of my stretch goals this year is a process to convert sub-workflows managed run by crew runner macros into sub-macros run by control containers.

CSmith16
8 - Asteroid

@hroderick-thr Hey thanks for the suggestion! Will have to look into that, I have heard of the crew macros before, and actually a few of our workflows use some of their tools but haven't ever come across that particular one. I will have to give this a look and see if it's something we can use. Love how there's always some way to get something done on the community, official or not.

jrobiso2
8 - Asteroid

Is there any way to export out the Control Container Tool so we could import it as a macro or such into 2022.x versions?

akadwa
7 - Meteor

@CSmith16 Aah I get you. I would recommend the "Parallel Stop until done" macro by Crew, as some others have suggested. We used it successfully (mostly) in a couple of our workflows where we needed to sequence certain flows.

akadwa
7 - Meteor

@jrobiso2 Control containers are quite embedded into the fabric of the Alteryx application so I am almost certain it's not possible to just export the tool - however lest see what others have to say.

RithiS
Alteryx
Alteryx

@jrobiso2 - akadwa is correct. You cannot export Control Containers as a macro and use it in older versions. The workflows (included macros) are XML structures that would not be recognized. Older versions would also be missing the engine code needed to process it.

kpere007
5 - Atom

Thank you for sharing!