Non English characters messed up when imported via Directory tool

[snigdha25]

I have 7 csv files that I am importing in Alteryx using Directory tool , some files have CJK  (Chinese , Japanese , Korean) data within them. These characters when exported to an excel file as Output are messed up and are replaced by question marks like - "???". How can we avoid messing these characters up ? 

[Kakuffo]

@snigdha25 

Have you tried selecting the unicode uption for the document coding option 11.

[snigdha25]

Hi @Kakuffo - I am using Directory tool so this option is not available as in Input data tool.

[Kakuffo]

Ok can you provide a screenshot? Or kindly explain exactly where the characters are being converted

In your explanation you mention the characters being converted to "???" on export to excel.

[snigdha25]

Not sure where exactly is it getting converted because it is a directory tool so I can only see file names and not the data within them while processing. As per example a file within folder imported contains -

安妥（北京）旅游文化发展有限公司（Antuo and Culture ）

青岛信舆数识科技有限公司 (QinginYu)

and gets converted to this-

?????????????????Antuo and Culture ?

???????????? (QinginYu)

[Kakuffo]

Thank you for providing an example, strange as i have tried to replicate this on my machine and the directory tool is able to feed through the chinese characters just fine.

Which version of Designer are you using at this time?

perhaps take a look at your user settings localization tab, here you can adjust the coding also

[snigdha25]

@Kakuffo -All the setting are similar to what you posted in screenshot. Can you try importing the csv files attached using directory tool and output them in excel to see if it works at your end. The file 2 has Chinese data

[Kakuffo]

What method are you using to import the files after the directory tool?  from what i can see both files have differeing schema so a basic wildcard or dynamic input would not work in this instance. Are you using a bacth macro to import thr files in one go?

I have used a more manual and visual process here to try and match the field and combine the data using union tool, my outputs work fine in Excel.

Perhaps i have missed something in your process that is causing the error.

I have attached my output and workflow to the message

[snigdha25]

@Kakuffo - Yes, I have used batch macro as the schemas are different not sure if that is causing an issue.
Your workflow works perfectly in case of INPUT TOOL but as I mentioned I want to achieve it using DIRECTORY TOOL. I have 200 such kind of files and doing this using INPUT TOOL is not feasible.

[Kakuffo]

Are you able to send the macro across? so i can troubleshoot that