Automated Workflow/App Deployment to Server

[chrisha]

Hi everyone,

we are currently migrating our workflows from our network fileserver to Git repositories for better version control and collaboration on different workflows.

Many of our workflows get uploaded into our company Alteryx Gallery. As there is an API for the gallery, I was wondering if anyone already has implemented some kind of automated build-and-deployment-pipeline. Something similar to yarn or gulp. My goal would be to automatically get the latest build from our Git repository and upload workflows to the server.

Anyone working on or with something like this?

Best

Christopher

[tlarsen7572]

Hey @chrisha, this is a neat idea!  I am working on implementing a CI/CD process with Git/BitBucket/Bamboo with our workflows and would like it to eventually extend to the Gallery.

I took a look at the API docs.  Right now, I do not think this is possible.  All but 1 of the API endpoints is a GET (read) endpoint.  The only POST (write) endpoint relates to scheduling jobs.  The API in its current version does not seem to support uploading workflows.  Unless Alteryx expands their API a bit, I don't think this is something we can easily do.

[MichalM]

Hi @tlarsen7572 

Have you seen the below?

https://community.alteryx.com/t5/Alteryx-Server-Knowledge-Base/Migrating-Workflows/ta-p/335774

[tlarsen7572]

@MichalM, I did not - sounds perfect!  I will play around with that API...

Btw, I was working off of this reference...is this not the current version of the API docs?

https://gallery.alteryx.com/api-docs/#admin

Tom

[BlytheE]

That is the latest version of the docs. The migration endpoints will be added to the interactive docs at some point in the future, but for now you can find information on them in the help docs for Server or the Community post @MichalM mentioned.

[tlarsen7572]

Thanks for the link @BlytheE, that has been a huge help.

I almost have a custom python tool completed to upload workflows to a gallery.  My thinking here was to use Alteryx to perform the automated deployment.  Unfortunately, for the last few hours I have been getting a 500 response from the server with an 'UnknownException' message, despite my best debugging efforts.  If anyone can (or wants) to take a look at my code and tell me where I am going wrong, it would be greatly appreciated.  It took me a day just to get past the authentication/signature process (oauth signing is evil), so I'm feeling a bit fatigued here.  I will keep working on this as time permits, but any help or direction would be awesome.

[tlarsen7572]

I did some sniffing around the Gallery logs and found this right after my POST request:

2019-05-17 14:00:33.684786,ERROR,197,ErrorHandler,HandleError,,,,,,,,,,Exception caught by ErrorHandler and marshalled to client,"Ionic.Zip.BadReadException:   ZipEntry::ReadHeader(): Bad signature (0x6D783F3C) at position  0x00000000->   at Ionic.Zip.ZipEntry.ReadHeader(ZipEntry ze, Encoding defaultEncoding)->   at Ionic.Zip.ZipEntry.ReadEntry(ZipContainer zc, Boolean first)->   at Ionic.Zip.ZipFile.ReadIntoInstance_Orig(ZipFile zf)->   at Ionic.Zip.ZipFile.ReadIntoInstance(ZipFile zf)->   at Ionic.Zip.ZipFile.Read(String fileName, TextWriter statusMessageWriter, Encoding encoding, EventHandler`1 readProgress)->   at DotNetZipCompression.CompressionEngine.GetArchiveComment(String archiveFile)->   at Alteryx.Cloud.Engine.AppEngine.GetWorkflowsFromPackage(String packageFile)->   at Alteryx.Cloud.Models.Presenters.AdminApiPresenter.PublishApp(AppFileInfo fileInfo, ExecutionModeType runMode, Int32 validationPriority, String requestId)->   at Alteryx.Cloud.Models.Presenters.AdminApiPresenter.<PublishAppAsync>d__24.MoveNext()->--- End of stack trace from previous location where exception was thrown ---->   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()->   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)->   at Alteryx.Cloud.Server.Services.Api.AdminApiServiceV1.<PublishYxzp>d__10.MoveNext()->--- End of stack trace from previous location where exception was thrown ---->   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()->   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)->   at System.ServiceModel.Dispatcher.TaskMethodInvoker.<InvokeAsync>d__16.MoveNext()->--- End of stack trace from previous location where exception was thrown ---->   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()->   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)->   at System.ServiceModel.Dispatcher.TaskMethodInvoker.InvokeEnd(Object instance, Object[]& outputs, IAsyncResult result)->   at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeEnd(MessageRpc& rpc)->   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage7(MessageRpc& rpc)->   at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)"

Looks like some sort of issue with the header?  Perhaps my request is malformed?  Not really sure where to go from here...also, should I maybe post this in a separate thread?

[chrisha]

Hi @MichalM,

that looks promising. I will have a closer look - we are currently running an older version of the server, so I cannot test the new environments feature. As soon as we are able to upgrade (our Python SDK tools are not working properly on the latest versions), I will have a closer look.

@tlarsen7572: Seems like you are already working on something very similar. Would be keen to have a closer look on your work later! :)

Thank you & best regards

Christopher

[tlarsen7572]

@chrisha, I would be happy to share when I fix the error!  I have opened a separate thread for it and will post back here when I have a working process in place...

[SeanN]

@tlarsen7572 , if interested, we also made it so the OAuth info can be passed in an authorization header rather than the URL.

EX:

Authorization: OAuth oauth_consumer_key="8D61F0AC3DFB939fa1e0d2c0b058cf14c5e5f08a85ffd7f8e2fc49f", oauth_nonce="fsZDMPnJGLsFIKBD0hzBAFSgeQkXzc8K", oauth_signature="%2Fxtf5FYcwHXZbgz624tzhzjZ2iE%3D", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1542143917", oauth_version="1.0"

Side Note: the reason this isn't available in the swagger docs is because the tool that generates those docs can't handle multipart/form-data posts at this time.