Community Spring Cleaning week is here! Join your fellow Maveryx in digging through your old posts and marking comments on them as solved. Learn more here!

Alteryx Server Discussions

Find answers, ask questions, and share expertise about Alteryx Server.

Allow download of output files from App run on Gallery without saving the files in MongoDB

rfoster7
9 - Comet

We have a size issue with our MongoDB. It's climbed stratospherically in the last few weeks in 500GB+ and consumed all the available space on the server. 

 

We know the reason is because of files embedded into the Mongo DB by run workflows on the gallery. A number of our apps are set up to generate outputs as pcxmls or csvs, but instead of outputting them to file shares, they output so as to be downloadable at completion of run by the app runner. 

 

This apparently means that the outputs are embedded in AlteryxService AS_Results table in a BLOB file. So far so good. 

 

We have now lowered our persistance settings to hopefully clear these out, but what we really want is a solution where we can still give our users the same general experience (I run an app, at the end of which, I am presented with files I can choose to download)  but without clogging up the MongoDB with these enormous files. 

 

If we change the files to output to file shares, then the option to download the files at the end of running the workflow disappears. Even if I check the file outputs in Interface Designer, because we've provided a file location, we don't get the option to download. 

 

So I wonder if anyone has run into something similar and has some ideas? 

 

Ideally, we could save the files to a network share but still have a download option.

Otherwise, it would be fine to have a way to tell Gallery to NOT store the files in the Mongo DB. 

Otherwise, if there was some way to pop a hyperlink at the end of running with links to the file shared files that would work as well. But I am not having any luck with that so far. 

Using a run command at the end of the workflow is a no starter because it launches the file on the gallery rather than on the user's desktop. 

 

Any ideas or thoughts would be appreciated. 

7 REPLIES 7
mbarone
16 - Nebula
16 - Nebula

I don't think there's a way to suppress it all together.  The only option that I know of is to set your persistence "number of days" to something very small.  What did you set that to?  Have you tried just 1 day for the setting?  

rfoster7
9 - Comet

Yes. We had it at 30 days because we like the reporting aspects. When this problem popped up, we set it to 3 days. Unfortunately, that doesn't shrink the MongoDB size by itself, until you do a DB rebuild/restore. 

 

So right now we have it set to 3 days, however, we would LIKE to have the persistence longer for reporting uses. We are really hoping there is a solution that lets us keep the persistance longer for most workflows, while being able to manage these few huge database hogs. 

mbarone
16 - Nebula
16 - Nebula

How much data is being downloaded?  We have about 100 apps, and is set to 30 days.  We have several apps returns hundreds-of-thousands of records at a time.  Ours never seems to go over 100 GB.  Wonder if there's something going on that's blowing it up.

rfoster7
9 - Comet

Its a good question. He have a handful of apps that are causing the problem. We've pulled the as_results reporting on them and they are sometimes creating blobs in the mongoDB of near 100GB in size. So I'll go back and talk to the users and see if they know why they are trying to download such huge datasets. The workflows in question generate both a csv resultset and use Render to make what should be a small pcxml report. Unfortunately, I can't figure out how to tell which of the two embedded files is taking up the space, all I can figure out how to get out of Mongo is the total file size. 

 

 

mbarone
16 - Nebula
16 - Nebula

Yep, we do that a lot as well - give them a CSV to download, but present them via pcxml an on-screen summary of the data.  Those are some huge blobs for sure!!

rfoster7
9 - Comet

Update: Turns out it wasn't the results files at all. the Size was being misread as GB instead of MB, so the results files werent' excessively large after all. There was a problem with how some workflows were using dynamic inputs. Apparently, if you run a massive number of dynamic inputs by feeding a list of ids into it and telling it to run once for each id, when the gallery runs out of active memory, it starts adding chunks of space into the MongoDB to store the results and doesn't deallocate it until you backup and restore the MongoDB. Seems like a pretty severe memory leak issue and I hope Alteryx resolves it sometime in the future. We corrected the workflows by fixing how they misconfigured the dynamic inputs and seem to have resolved the issue for now. 

 

 

mbarone
16 - Nebula
16 - Nebula

Thanks for the update; that certainly explains it.  You'd think it would be deallocated shortly after the app is complete!!