Text Pre-Processing : minimum word length

[simonaubert_bd]

Hello all,

Here is a very, very simple idea :

just having the right to put a minimum length to Text Pre-processing
Why ? A lot of words (especially in French) with 1 or 2 characters does not change the meaning, the sentiment: la, le, à, un, une, etc... In English : in, the, etc..

Best regards,

Simon

[CristonS]

hi @simonaubert_bd you bring up a good point, thanks for posting!

[veeliang]

Hi @simonaubert_bd, thank you for the feedback. Can you elaborate? I would love to better understand your use case. You mention sentiment here – would you be able to provide additional details? Is this a reference to the Sentiment Analysis tool? As of right now, the Sentiment Analysis tool is only supported for the English language. We also recommend not using the Text Pre-processing tool with the Sentiment Analysis tool. The Text Pre-processing tool can remove features that the Sentiment Analysis tool relies on to determine sentiment. 

[simonaubert_bd]

Hello @veeliang 

I planned to use the Text processing just before two different tools : the sentiment analysis (which I know does not work in other languages than English) and the word cloud. In French, far more than in English), we have a lot a short prepositions, pronouns and determiners, sometimes with even a single or two letters : à, y, t',n',s',m',c',l',d',,ça,ce, ma,ta,ce,te,le,la,etc... That would pollute the analysis and for now, I have to remove it manually. Of course, it would be even better if you can provide a tool that remove determiners, articles, preposition, postposition, pronouns by language (with a box for each ) but it seems difficult to implement.

Best regards,

Simon