Posting Workflows to the Gallery API - sourceId Parameter Ignored

[ice_cow]

Hello, I am posting this to see if anyone else has had this problem and found a way to solve it. I am posting workflows from a Test Alteryx environment to our Prod environment using the Gallery endpoint /gallery/api/admin/v1/workflows/ (using a POST call). 

I am attempting to create an automated promotion process, and the workflows I am promoting will be overriding the previous versions. However, when I use the sourceId parameter in the API call, it never overrides the specified workflow. It will give a new App ID, and the posted workflow works as expected, but it is a duplicate and will not override the ID given. Strangely, if I repeat the call it will give me the same ID back (meaning it created a new workflow on the first call, and overwrote that ID on the second call, even though I didn't pass that ID). If I change my sourceId parameter to the ID given back to me, it will again generate a new workflow.

My example parameters are below. Please let me know if you have encountered this error.

[KevinP]

@ice_cow This API was intended to migrate/promote workflow from an existing environment to a new one. It was not intend to publish/update content in a single environment. The POST to this endpoint will only create a new workflow in the target environment or replace an existing workflow that environment. It will not revision or modify an existing workflow.

In order to replace an existing workflow it must have been uploaded via this API with a sourceid provided, and the second API request must match the originally specified sourceid exactly. The id value returned by this API is the id in the target environment of the workflow that was created or replaced by the API call. The sourceid provided to the API isn't the id of the workflow in the target environment, but rather is intended to be the id of the workflow from the source environment. 

For example lets consider that workflow A is stored in environment A with an app id of ‘1234’. To move this to environment B you use the API to retrieve workflow A from environment A as a yxzp. You then use the POST API to create workflow A in environment B and provide the sourceid of ‘1234’. If workflow A is later updated in environment A and you want to move that update to environment B you would follow the same process above again providing sourceid ‘1234’. This will replace workflow A in environment B with the updated workflow. If the sourceid isn’t provided at either step or doesn’t match exactly the API will just create a new workflow.

The sourceid can be any string value and doesn't necessarily have to be the id from the source environment. The id is stored as part of the workflow record in Mongo under the AlteryxGallery database in the AppInfos collection in a field called 'SourceAppId'. For more information on this I would suggest reviewing the help documentation for the Migration API and for the MongoDB Schema documentation. 

[mercra]

@KevinP 

Do you know if the sourceId is exposed via the Get Workflow API?

Thanks,

Craig

[mercra]

@KevinP 

If someone creates a flow manually in an environment the SourceAppId is null.   Without setting the SourceAppId to something unique we can't update using the API.  To that end, is there an API to update the SourceAppId?  is it safe for us to update directly in Mongo?

Thanks,

Craig

[KevinP]

@mercra sourceId isn't currently exposed via GET /workflows or GET /workflows/all in the currently available API's. The sourceId can be updated via Mongo directly. However, please keep in mind that manual updates to the database aren't recommended or support. Doing so may cause issues if done incorrectly or during database migrations/updates if a values falls outside of what we would expect. As such I would urge extreme caution before making any manual DB changes, as well as ensuring you make a back of the DB before any changes.

A few additional notes. We have updated the POST workflows functionality since I originally responded to this so that if a matching sourceId is found it will revision the workflow instead of just replacing it. Also we are currently working on some significant API improvements. These changes will include improvements to publishing and versioning workflows via the API, and API workflow management in general.

[mercra]

@KevinP thanks for the quick reply and feedback on the mongo updates.    Also great news on upcoming API improvements.   Do you know what version of the software adds revision history via the post?  We're currently on 2020.1