Run Command Tool: Problem with Redirecting Output? (use case: Amazon S3 CLI)

[Scott_Snowman]

Hi Alteryx Designer folks - I am trying to tighten up a process that uses the Run Command tool and keep running into roadblocks. I'm hoping someone might have some insight!

I am trying to execute the following process: compose an S3 CLI copy command based on data in my workflow, then execute the command and log the results using a redirect statement.

Since there may be 0-N dynamically written commands per run of the workflow, I designed a batch macro to inject the command into a Run Command tool and loop through all the commands. Below is a masked version of the command I've written, very straightforward. (I'm using a dummy input with no meaningful data on the input connection.)

When running the workflow, I get this, almost as though something is confusing the role of the ">" and redirect path strings.

So it almost looks like the Run Command can't properly parse redirect commands.

I know that I can implement this net effect by concatenating the commands and using the Run Command tool to write into a .bat and execute once (i.e. not in a batch macro). But I'd prefer to avoid that implementation if I can. Any ideas?

[BrandonB]

I think that you might benefit from this article by The Information Lab that walks you through configuring this: https://www.theinformationlab.co.uk/2020/04/08/working-with-the-aws-cli-in-alteryx/ 

[Scott_Snowman]

Thanks @BrandonB , this is a great resource (and one I wish I had when I had started the endeavor)!

But it still relies on writing to a .bat file to actually capture the commands. I'm looking to instead pass them directly to the Run Command tool -- the reason being that if there are 0 commands generated by the workflow, the Run Command tool generates an error when it attempts to read the (0B) .bat file created from 0 commands.

We're trying to avoid having the workflow throw an error when it has 0 commands to run. Do you have any insights on how I can achieve this without writing to a .bat first?

[BrandonB]

Maybe you could still take the bat approach but add on a line that says EXIT at the end? That way you would always have a script that at least has the word exit to finish and close. 

[Scott_Snowman · Accepted solution]

@BrandonB -- ah that's an elegant solution and should indeed prevent errors when there are no transactions to run.

I was also able to avoid the .bat process entirely by adjusting the command within the batch macro.

Rather than call the aws executable, it calls cmd.exe with the /c parameter. This lets me pass the entire string in with the redirect parameter included and have the output redirected as desired with individual calls to Run Command rather than writing externally to a .bat file.

So the macro now looks like this, with the input parameter string of 

/c aws s3 --profile [NAME] cp s3://bucket_name/file_with_parameters.csv C:\local\parameter\file.csv > output_log_1.csv

I'm not sure what's being handled differently with the two processes -- i.e. if the Run Command tool is parsing the Command Arguments differently in the two cases, or more likely the aws executable is mishandling the redirect string -- but it looks like there's a few workarounds depending on how one wants to implement!