Event running AzCopy.exe occasionally throws an error message when run via the scheduler

[Leif]

I have a workflow with an event (before run) which runs Microsoft's AzCopy.exe to copy some files into Azure Blob Storage:

For some reason, when this workflow is run through the scheduler, it occasionally throws this error message:

"Error running Event #1: The external program "..\..\dependencies\Azure\AzCopy\AzCopy.exe" returned an error code: 3: The system could not find the environment option that was entered."

Sometimes, however, the event succeeds without any error messages (throughout there are some data conversion warnings). As evident by the results tab in the scheduler:

 

Note that, even when the error message is thrown, the workflow apparently succeeds, as evident by the fact that the files are indeed updated in Azure Blob Storage.

This error does not occur when the workflow is run manually with Ctrl + Enter via the designer.

Although the workflow executes properly, this issue is annoying, because it interferes with our operational notification scheme (emails are sent out if workflows and steps therein fail).

I can't figure out why it only throws this error occasionally. Does anyone have any ideas?

[ToPoissonOrNot]

I have the same problem. Would appreciate some advice on how to circumvent this.

[Leif · Accepted solution]

My coworker discovered the root cause of this issue.

Long story, short: It was because the journal path for AzCopy was defaulting to the user directory.

That's why it was working when the workflow was run manually via Alteryx Designer, but not via the scheduler.

The scheduler did not have access to the necessary user directory (environment variable) which is the default path used by AzCopy for its journals.

The fix is to provide a custom journal path by passing a parameter to the -Z flag of AzCopy.

More detailed info and examples can be found here:

https://docs.microsoft.com/en-us/azure/storage/common/storage-use-azcopy#journal-file-folder