community
cancel
Showing results for 
Search instead for 
Did you mean: 

Alteryx Designer Ideas

Share your Designer product ideas - we're listening!

XML Parse Tool: Do not pass parsed field

Hello

 

My problem: I've used the download-tool to download a 40mb XML. Parsing the DonwloadData-field containing this XML results in about 6600 records. The XML-Parse Tool passes the orginal DownloadData field to each record, resulting in quite a bit of memory usage:

 

 

XML-Parse Problem.PNG

 

Suggestion: An option in the XML-parse Tool to not pass the parsed field in its output. 

 

Marco

9 Comments
Magnetar

I'm guessing there's a way to trim that down: we should be able to tweak the XML parse tool to just return the data we're interested in rather than generating as many copies of the entire XML as there are records of interest. If your workflow can be shared, feel free to attach it and someone can help spot the correct adjustments.

Alteryx
Alteryx

I, too, would like to see this feature to prevent sending the original XML, in each record, out of the XML Parse Tool.  I am parsing a 19 MB XML file whose resulting output should be ~3835 records.  19 MB * 3835 records is ~73 GB. Way more than my 16 GB of RAM. In fact, I may not even have 72 GB of disk space remaining. Which would explain why Alteryx locks up on this tool.

Bolide
Bolide

This post needs to gain some traction. We broke a PETABYTE last night.

PEd2.jpgA PETABYTE...

@Treyson I'm shocked they had a "PB" tag! 

Bolide
Bolide

@patrick_digan I'm shocked my desktop didn't melt all over my desk

Quasar
Quasar
Nice work hitting 1PB in a flow- i'm sure Ned would be proud!
Magnetar

We really need "like" buttons for comments here.  That PB picture would be piling 'em up.  That's awesome!

What is even more impressive is that this was accomplished on a standard desktop computer with only a 500GB hard drive and 16 GB of ram, so I'm curious about how it is even possible to store or process 1.2PB of data on this desktop machine in this workflow.

 

I suspect the file size is overstated...

Alteryx
Alteryx
Status changed to: Under Review

We will be making some improvements to the XML Parse tool and will consider this among those improvements.