Trouble loading large .rds matrix into Alteryx

Question

I have a large data matrix that I created in R that is 5.6GB. I want to load it into Alteryx for further transformation and analysis. I am having trouble loading and writing it to Alteryx. Here is the script I have tried using in the R tool, and the outcome:

----------------------

memory.limit()

memory.limit(60000)

dat <- as.data.frame(readRDS("~file.rds"))

write.Alteryx(dat, 1)

-------------------------

After 3.5 minutes, I get this error message: "The R.exe exit code (4294967295) indicated an error."

If I run this code to test if it can read the matrix:

----------------------

memory.limit()

memory.limit(60000)

readRDS("~file.rds")

-----------------------------------

I see it read the matrix and outputs some of the data. It just seems to be a problem when I try and assign the data somewhere.

creily · Answer

I was using the most recent Alteryx, and what ended up fixing the problem was uninstalling and reinstalling the previous version.

MichaelF · Answer

Hi @creily

This could be related to the "fake" R error listed here -

https://community.alteryx.com/t5/Alteryx-Designer-Discussions/R-tool-Fake-Errors/td-p/25163

Another possibility, however, is that the connection between R and Alteryx is timing out. When writing out to Alteryx, are you able to write a portion of the data? (you may try something small like 1 record, and if it works, increase the size). If this is the case, there is a defect in place for this type of behavior. (you may still want to follow the above thread to get rid of the error if this is occurring)

As a workaround, I would suggest trying to write it out to Alteryx in batches to multiple outputs (or multiple R Tools if need be)

Cheers,

Mike