Adjusting String Column Length when uploading using HDFS

Question

I am having issues loading data to Cloudera via ODBC connection. I found this forum post with what I believe to be the solution but as I do not understand why it is causing the slow write times I am having a hard time convincing our IT guys to change the settings to try the solution given. What is the reason that lowering the string column length from 32727 to 255 improves performance?

It is my understanding that this number is the max character length rather than just the character length.

Thanks for your help

Jim7 · Accepted Answer

The longer your maximum string length, the more memory the system has to allocate to process the data stream.  For that reason, it's always a good idea to use the minimum string length necessary to capture your longest string. That's a good rule of thumb when defining all of your fields in Alteryx as well :)