Is there any way to read traditional Chinese Characters?

[caltang]

Context:

https://en.m.wikipedia.org/wiki/Debate_on_traditional_and_simplified_Chinese_characters

Related posts: 

• https://community.alteryx.com/t5/Alteryx-Designer-Desktop-Discussions/Traditional-Chinese-Chactersti...
• https://community.alteryx.com/t5/Alteryx-Designer-Desktop-Discussions/Character-encoding-text-alignm...
  
  

Long story short, I have a few use cases that references some Traditional Chinese characters. But Alteryx is not able to read them somehow… it’s coming in as gibberish. I don’t know if the CodePage can be expanded or not but I’m stumped. 

I raised an idea here:

https://community.alteryx.com/t5/Alteryx-Designer-Desktop-Ideas/Alteryx-Designer-Add-Language-Tradit...

Has anyone been through this? Or is the solution just to use Python and read the data as is?

[NeoInfiniTech · Accepted solution]

Hello @caltang,

If you have a case where you simply need to read in a CSV with an extended code page option list (which is a scenario that allows the code page to be selected), you can refer to this solution I previously posted: Input tool has limited varieties of code pages.

Looking at the encoding list, I would assume that 3 other options apart from the UTF-8/UTF-16 would have the capability to read Traditional Chinese characters properly:

ANSI/OEM - Traditional Chinese Big5

ISO-2022 Traditional Chinese

MAC - Traditional Chinese Big5

I am also attaching the sample workflow here. For such cases, the only way to extend the code page options in Alteryx Designer is to use an analytic app or macro to modify the encoding parameter accordingly.

For other cases where a tool used to read in the data do not have an option to modify the code page, I unfortunately do not have an answer.

[caltang]

This is pretty neat! It solves one portion with CSV. But a majority of the files I’m dealing with end users are using actuarial related files, most of them delimited in nature. If I put them into CSV, it messes up some parts too because my computer’s Excel is reading it weirdly as well. 

It will be nice if the code pages can be expanded, but I learned a neat trick from you today! Thanks @NeoInfiniTech !

[NeoInfiniTech]

I would also like to add to this post where I got the code page information from, as my original post that was linked in the previous reply contains a link that no longer redirects you to the extensive list but to the standard list.

https://help.alteryx.com/archive/20214/designer/CodePages.htm

This list, unlike the current code page list contains all the code pages that can be used with ConvertFromCodepage and ConvertToCodepage, but I also discovered that they could be used with CSV Input code page (at least most if not all of them). It helped me to properly output a file containing characters from a non-English alphabet for use with CMD by selecting an OEM encoding, which happens to be the code page type that properly works when you want to use CMD.