Encoding categorical variables without having biased data

Roche

Hi everyone, 

I am currently busy with a Classification Machine learning workflow and some of the string variables needs to be dealt with in a different manner since the number of categories contained in them exceed 54.  I have found the attached flow from a conversation a few years ago which shows me how I can do the encoding.  I work with sales data and the type of variables that I need to encode (or use an alternative method) are End user ID, sales office, reseller, reseller parent etc. 

I would like to find out if this method of encoding will yield biased data, i.e. as the numbers increase etc.?  I can understand that this type of encoding will not work for low, medium, high, but will it work for my use case?  

If anyone have some expert advice on encoding variables, I would appreciate it!

Thank you for helping

ChangingStrings_UniqueCategorical.yxmd

gabrielvilella

Hi @Roche, the attached workflow only converts the a certain string value to a corresponding number. On your model, if you use a field set as string, the different values within it will not lead to a biased model. In other words, if you have on your field 1 and 2 and this is a string, the 2 will not mean that it is higher value than 1. This would only apply if you have numeric fields. Therefore, this 1 and 2 would be no different as having letters on it.