Handing large data (>80gb) yxdb files while publishing to Alteryx Server

[nushh]

Hi folks,

I am trying to save a workflow on my company's gallery which has approximately 80gb historic data in .yxdb format. I am unable to publish it on Server. Are there any possible workarounds?

Thanks,

Anushka 

[mbarone]

A little more info needed . . . what error are you getting that tells you you're unable to save the workflow to the Gallery?  Are you trying to send the big YXDB to the Gallery along with the workflow as an asset?  Unless there's restrictions behind the scenes on your server that have been set by your IT department, that should not be an issue.

This brings another question - why are you trying to send a big YXDB to the Gallery as an asset?  Would it make more sense to leave the big YXDB where it lives (on a shared network drive or even the server itself, presumably) and have the workflow read/write to it through a network connection?

[nushh]

hey,

I tried the second approach i.e, saving the large yxdb file in a shared folder and accessed it using UNC path.

While publishing I unticked validation and deselected the output files that were being written in the shared folder. 

I got the following error while publishing..

Please let me know do ALL the input files also need to be in the shared folder itself or can I just keep the big file? And, are there any other changes that I need to do on Designer/Server like changing the path to ALL UNC in workflow dependencies, etc.

Any help would be appreciated.

Thanks

[mbarone]

I always keep validation checked.  Try checking it and see what error it gives you.

[nushh]

we're taking data from google sheets, and it hasn't been configured on our server. So unchecking validation helps after successfully running the workflow.

Any suggestions as to what the error could be?

[mbarone]

Not without an error message (you're probably not getting any because you unchecked validation).  Can you run the workflow in Designer?  If so, then maybe the server doesn't have access to the network UNC folder.  Without those error messages it'll be very difficult to pinpoint it. 

[mbarone]

And if the workflow is taking data from Google sheets, but the server doesn't have access to Google sheets, then that could be a problem too.

[nushh]

I tried running it on Designer and it ran without any errors. And as a trial, I tried reading and writing to the shared drive and it caused no issues.

Data input from Google Sheets publishing to server without validation is working fine for other workflows. Just for this particular workflow with the huge file, I used the shared drive and seem to be getting some error.

Could you please let me know where to check

[mbarone]

If it runs in Designer fine, but not in Gallery, then the most likely culprit is that the server does not have access to the location.  Can you check with your IT group to ensure the server has access (not you when logged into the server, but the server itself).