Memory Error & Data Type Inference: read_csv Python tool for 900MB file throwing

[ashirsat9]

Hi,

I was having issues with a script in my Python tool since Input Data Tool does not infer type for numeric columns after reading csv.

I tried using a workaround using another python tool to read_csv and output it into the workflow. I faced a performacne related MemoryError in my Python tool notebook for a 900MB file with the messsage. It is potentially the memory limit per anchor setting since it fails on the Alteryx.write("#1"). 

How do you load large csvs with the proper data type inference similar to read_csv in Python?  ( I cannot set it using Select tool since the csv will be new every time)

What do you set the max field length size to? to avoid truncation. 

How to fix this error related to memory limit while ouputting from a Pythn tool

[OTrieger]

@ashirsat9 

I think that it will be easier to use Input Date tool and then figure out how to assign the correct data type, as there are multiple ways how to do it in Alteryx.