community
cancel
Showing results for 
Search instead for 
Did you mean: 

Alteryx Designer Ideas

Share your Designer product ideas - we're listening!

Ability to execute tools in parallel within the same workflow

Tools within a workflow needs to be able to run in parallel whereever applicable.

 

For example: Extracting 10 million rows from one source, 12 million rows from a different source to perform blending.

currently the order of execution is the order in which tools are dragged into the canvas. Hence Source1 first, Source2 second and then the JOIN.

 

Here Source1 & Source2 are completely independent, hence can be run in parallel. Thus saving the workflow execution time.

 

Execution time is quite crucial when you have tight data loading window.

 

Hopefully alteryx considers this in the next release!

56 Comments
Meteoroid

I'm surprised that this functionality, requested by numerous users have not been addressed since 2015. I spent 120+ hours to create a macro to download data from API and Ingest to the database. I wanted to run that parallel for multiple accounts, tried different tricks but it won't run things in parallel. A process that is supposed to not take more than an hour is taking 8 hours now. Any update on this functionality? The only way out I have is to split accounts into workflows, so I would have 50 workflows. If I need to replicate this for all of my clients' that would be 500+ workflows for this specific process. We also have other processes running in the gallery. Additionally, the absence of centralized scheduling (all gallery uploads go into personal space unless specifically shared, team members would see a scheduled job but won't see the output messages) would create a maintenance & monitoring nightmare if we have 500 workflows running in the gallery which could easily be just 10 workflows with parallel processing! I'm surprised why can't this be solved in the product functionality? 

Bolide

I have along with the rest championed this cause but there are workarounds potentially for you @shouvikdas.

  • Make your flow dynamic so it can get the account and any other run specific info from a database. 
  • You will need another mechanism to populate the database with the info to control the run, could be a flow or a form or whatever you choose.
  • Have another flow fire every 10 minutes or so that reads the database to see if any records are not processed and if there are then fires your flow else not.
  • Your flow will be modified to read the database for the run control info like which account to run and will update the database records being processed so the next iteration will not reprocess it.

takes a little effort but can be a reusable technique applied to other flows for most shops.

 

Also there are a variety of variations depending on how slick or basic you want to be...based on time to build and or skill...like using flat files to feed iterations instead of a DB.

 

agreed on the scheduler limitations and many other issues with Server...i wrote my own scheduler so we could have a much richer set of schedule criteria with a range of business rules we can apply.

Meteoroid

@AlexP - Any update on the time-frame for the implementation of this feature. It has been under review since 12-31-2015!

Alteryx Partner

looong time no updates on Devops... @AshleyK 

Meteoroid

@AlexP and @AshleyK - hoping to get an update on this. Thanks.

Meteor

I feel like this functionality should have been made available a long time ago.

 

UPDATE PLEASE!

<H1>
<b>
PLEASE!
</b>
</H1>

 

Any update on this? This feature would be very helpful for my organization as well. Thanks

Meteoroid

@LibbyD  - Requesting your support in getting an update on this feature request as our requests for updates on this feature from the product management team have not been successful.

 

It has been 2 months since my initial request for an update and more than 3.5 years since this feature has been under review.

Chief Customer Officer
Chief Customer Officer

Thanks @Abhas_Chandra  for the inquiry.  Let me ask @DanM to loop you into our process with our Product Management team.

 

Best,

Libby

Community Operations Manager
Community Operations Manager

@Abhas_Chandra Thanks for checking in on this idea. We are working on a new process with our PM org to better communicate with the Community on the status's of their ideas. We hope to have this update implemented soon.

 

I have reached out to our PM org to see if they have a status update on your idea and will post once we do. Please understand that some ideas can take a long time for us to figure out how and when it makes sense for us to implement the idea and that can take years. There are many factors we must consider before moving forward with an idea and in this particular case this is a request to make a change to the Alteryx engine. Which may seem like an easy change, but it's a vital part of the software, so we must make sure this idea along with the many others as well as the roadmap are all in alignment with our strategy as a company before ideas like this can be implemented.

 

DanM

Labels