Alteryx automation Desktop (Scheduling)

[AT_AYX]

Hey everyone,

I keep having the same error while scheduling a workflow (Remote Storage Error.)

This error only pops up when the workflow involves DBs (writing and/or reading) even though I configured the scheduler as this article showed https://knowledge.alteryx.com/index/s/article/Configure-Desktop-with-Scheduler-1583460917472 and added the run as a user.

The error exists whether I choose to run from disk or in Scheduler DB.

Could you please help me resolve this?

Thanks in advance

[EdP]

Please see the following KB for this error:

https://knowledge.alteryx.com/index/s/article/Remote-storage-error

[MartyScherr]

So this article did not resolve my Alteryx Schedules throwing the Remote Storage Error

1. I am NOT using DCM.  I'm using DSN connection strings.  I'm also not using Python or other strange setup as suggested by some of the Knowledge base articles.  

NOTE:  This Schedule worked fine a few days ago so it can not be the workflow design that is the issue.

2. I'm using the Store on Disk Option in the schedule as I thought maybe it was an issue trying to store a large amount of data but that doesn't seem to be the case.

3.  See attached screenshot for the the Output and Error Message I received.  Please don't point me to an article.  I'm looking for a practical fix to this issue.

[apathetichell]

This is timing out at 15 minutes. This makes me think some authorization/setting expires at 15 minutes. I don't know what's in your workflow - and I don't know any system settings you have  - but it looks like something somewhere is set to timeout after 15 minutes. Perhaps this changed on your system a few days ago and something which used to be longer lived (like an authorization token) now expires in 15 minutes?

[MartyScherr]

So what is interesting is that the Workflow Schedule runs fine when I run it manually in Designer.  It's only when I run it with WorkFlow Scheduler that it appears to timeout in 15 minutes.  Still looking for a resolution on this.  For Clarity:  I'm using the Input Data and Output Data tools in this workflow.  The Input Data is using a DSN connection string pointed at NetSuite Oracle ODBC.  The Output Data tool is pointed at Azure SQL.

[apathetichell]

I'd ask the admins for those platforms what the timeout is for however you authenticate. If you previously could run this workflow on scheduler -> but now can't -you should ask if anything changed recently in terms of timeouts.

timeout may be totally different when you are running in desktop --> 1) are you monitoring the execution time, and is it under 15 minutes 2) are you (inadvertently) doing something which allows for a refresh. You're not using DCM - correct?

[MartyScherr]

I am not using DCM

I am not using the Amp Engine  - which had apparently caused issues for some users running schedulers.

I don't think it's a Netsuite timeout issue because why would a manual run of the same workflow execute successfully using the same workflow?

I was able to get the workflow to execute successfully by using a Windows Scheduled Task using the Alteryx Command Line Utility.  That's the option I am going to use for now.

Anyone else would may have experienced a similar experience with Scheduler timing out I would be interested in hearing about how you resolved it.  Thank you.

[drewwoodall]

I'm running into this issue as well. The connection is a DCM S3 connection using IAM access keys. 15 minutes then it cuts out, the alternative would be to hard code the access credentials into each S3 upload and/or download tools.  

[MartyScherr]

I'm not sure how hardcoding the credentials would make any difference.   I suspect it has something to do with the internal Alteryx engine settings that are enforcing the timeout.  But since I haven't been able to confirm that it's only a theory.  Seems like the command line utility doesn't have the same timeout restrictions for whatever reason.

[apathetichell]

@MartyScherr- @drewwoodall is correct that hardcoding credentials could effect the timeout. @drewwoodall -> ask what the timeout setting is on the AWS STS session.

Hypothesis:

IAM role is native to the system.

IAM role uses AWS STS to get short term credentials to access S3.

Credentials expire mid operation (say 15 minutes in)

Fixes:

1) extend STS credentials lifespan for this role.

2) redesign workflow to split data into 15 minute lots.

15 minutes of S3 upload seems like a decent chunk -> perhaps you can turn this into multiple async flows?

Note - I would use AWS CLI vs the Alteryx S3 tool/DCM because I know what it's doing and how to control it and I don't have to worry about code bloat.

[drewwoodall]

Hi thanks for the quick response, but these aren't temporary session based keys. These are the IAM Access Key and Secret key that have to be manually cycled by our admins to expire. So timeout really isn't a problem,

I've run download workflows taken 25+ hours without hitting a timeout. They only seem to work when we run workflows manually from designer to upload and download from our S3 buckets. Once we get scheduler involved we get hit with that remote storage error.