topic Re: Alteryx Server Storage of Inputs/Outputs in Alteryx Server Discussions

Alteryx Server Storage of Inputs/Outputs

caltang — Wed, 06 Sep 2023 04:03:39 GMT

Hi all,

I am trying to understand and configure the Alteryx Server better - right now making changes to UAT side server.

For context:

My company approved the Alteryx Server on the basis that it does not store end user data such as inputs/outputs.
This means that if we use any workflow or analytic apps on the Server, it must not be reading from Input Tool or Output Tool because by default, the MongoDB does store that information.
I am not sure if we can purge the data instantly in MongoDB after each run on the Server (scheduled and manual runs).
Right now, we've identified the SharePoint Input and Output tool as a mechanism for ingress and egress to and fro our Server, but the configuration of this SharePoint is made complex with the Client ID, Secret, and Tenant ID configuration we have. Not to mention, this expires after x number of days.
The issue is, if we were to get end users to upload to the Server riding on their Client ID, Secret, and Tenant ID, it will allow other end users who have access to those collections to see whatever the end user can see on their SharePoint. Not to mention if the end user resigns or moves up/down/out of the org, we have to change the workflow/analytic app again and restart the process.

My questions are:

Is there a way for instant purge of the data so no data is stored at all?
Are there alternative ways in which I can use the Server in accordance with policies?

Kinda stuck at this point. I'm discussing with my Alteryx official team as well, but I would appreciate input from other users or Alteryx staff abroad as well.

Thanks all!

Re: Alteryx Server Storage of Inputs/Outputs

caltang — Wed, 06 Sep 2023 13:03:34 GMT

Also the limitation with the SharePoint Input tool is that it calls one file at a time. For more complex builds that call in many files with * or even with Batch Macros/Dynamic Inputs, the whole concept of SharePoint being a conduit just falls apart.

Re: Alteryx Server Storage of Inputs/Outputs

fmvizcaino — Wed, 06 Sep 2023 14:08:44 GMT

Hey @caltang ,

AFAIK, Inputs/Outputs are stored on your MongoDB only if you send the assets with your workflow. By unchecking the box, the files are read/written directly from/to your folders. By unchecking the output box, there is no output download, now your users will have to access a shared folder to access them.

Best,

Fernando Vizcaino

Re: Alteryx Server Storage of Inputs/Outputs

caltang — Wed, 06 Sep 2023 14:15:56 GMT

Hi @fmvizcaino

Sorry, maybe I was confusing - what I meant is, I know that the assets can be unchecked when we upload it to Server - that part is fine.

However, end users still interfact with a file browse interface tool on the Server no? Because of that interaction, their uploads via file browse or downloads - these are also stored on MongoDB based on our discussions with Alteryx.

So, if we set the purge data to 1 day storage, the problem is that it is still stored for a day.

Re: Alteryx Server Storage of Inputs/Outputs

fmvizcaino — Wed, 06 Sep 2023 14:53:31 GMT

Got it. Sorry for the misunderstanding.

You can access the AlteryxService' AS_QueueInputs and AS QueueInputs.Files periodically and delete the content of both collections.

This is just an example of a file being loaded to the AS_Queue collection and the workflow erroring after deleting the document from the DB.

Best,

Fernando Vizcaino

Re: Alteryx Server Storage of Inputs/Outputs

caltang — Thu, 07 Sep 2023 00:13:45 GMT

Nice! Is there any way to write a script that does this automatically the moment a workflow finishes running?

Reason being is we can’t store the Inputs or Outputs even for a moment.

Re: Alteryx Server Storage of Inputs/Outputs

fmvizcaino — Thu, 07 Sep 2023 14:26:30 GMT

I have heard about MongoDB triggers, but I don`t believe the embedded version has that. Even though the data is being stored there, important to mention that it is all encrypted and not possible to access or extract the content by yourself.

I can think of 2 options here.

Create a macro that will be included in all analytical apps to delete all rows from that collection automatically. Easy and simple to do with the MongoDB Output tool
Use an RPA software to monitor the size of the file and trigger a workflow to erase the content of the collection on %ProgramData%\Alteryx\Service\Persistence\MongoDB
1. If you access your MongoDB folder, you will see your files have a name similar to this: collection-10--8610824439640616305.wt
2. To find out the correct collection, you need to access MongoDB from the command line and run this command: db.getCollection('AS_QueueInputs').stats()

Best,

Fernando Vizcaino

Re: Alteryx Server Storage of Inputs/Outputs

caltang — Fri, 08 Sep 2023 12:15:10 GMT

Thank you my friend @fmvizcaino Fernando!

I'll review and implement. But I think it's much better to review processes and go through the discussions again to enable some sort of storage. Otherwise I cannot use the Input and Output tools with File Browse - defeats the whole purpose of Server.

But your comments are very helpful! I will implement them regardless. Thank you!