Text Pre-Processing : minimum word length

[simonaubert_bd]

Hello all,

Here is a very, very simple idea :

just having the right to put a minimum length to Text Pre-processing
Why ? A lot of words (especially in French) with 1 or 2 characters does not change the meaning, the sentiment: la, le, à, un, une, etc... In English : in, the, etc..

Best regards,

Simon

[CristonS]

hi @simonaubert_bd you bring up a good point, thanks for posting!

[veeliang]

Hi @simonaubert_bd, thank you for the feedback. Can you elaborate? I would love to better understand your use case. You mention sentiment here – would you be able to provide additional details? Is this a reference to the Sentiment Analysis tool? As of right now, the Sentiment Analysis tool is only supported for the English language. We also recommend not using the Text Pre-processing tool with the Sentiment Analysis tool. The Text Pre-processing tool can remove features that the Sentiment Analysis tool relies on to determine sentiment. 

[simonaubert_bd]

Hello @veeliang 

I planned to use the Text processing just before two different tools : the sentiment analysis (which I know does not work in other languages than English) and the word cloud. In French, far more than in English), we have a lot a short prepositions, pronouns and determiners, sometimes with even a single or two letters : à, y, t',n',s',m',c',l',d',,ça,ce, ma,ta,ce,te,le,la,etc... That would pollute the analysis and for now, I have to remove it manually. Of course, it would be even better if you can provide a tool that remove determiners, articles, preposition, postposition, pronouns by language (with a box for each ) but it seems difficult to implement.

Best regards,

Simon

[simonaubert_bd]

@AlteryxCommunityTeam @JessL 

Hello, it's in "coming soon" for almost 10 months now. Will it be packed with 2022.3?

Best regards,

Simon

[veeliang]

Hi @simonaubert_bd ,

We released a brand new tool called the Part-of-Speech Tagger as part of Intelligence Suite version 22.1 earlier this year. This enables users to identify a given word's part of speech (i.e. whether it is a verb, noun, adjective, etc.) to enhance the data for downstream applications. You could use it to filter out certain parts of speech in your analyses.

[simonaubert_bd]

Hello @veeliang 

Thanks. I will test it once I get my IS partner license back (because as a partner, we don't have an IS license for every Designer License )

Best regards,

Simon