topic Re: How to convert broken letters? in Alteryx Designer Desktop Discussions

How to convert broken letters?

knozawa — Thu, 24 May 2018 04:03:54 GMT

Hello,

I am using Grid database and I found multiple broken letters.

The data field included " $◻ {\displaystyle \sqâ" that supposed to be "-", therefore, I used a formula to replace it. Although the letter "â" was replaced to "-", the two squares were not removed in the data. It seems like the actual data does not include two squares though...$

$Does anyone know how to handle this issue?$

$Sincerely,$

$Kazumi$

Re: How to convert broken letters?

Aguisande — Mon, 07 Aug 2017 19:31:02 GMT

Hi @knozawa

One thing that works for me in this cases is to handle all String fields as WString (Use a Select Tool to change them).

Can you try this and let us know?

Thanks

Re: How to convert broken letters?

knozawa — Mon, 07 Aug 2017 20:01:54 GMT

@Aguisande,

Thank you for your suggestion. I converted the field from V_string to WString, but the result was the same. The two squares were still there.

I also tried to use the ConvertFromCodepage() and DecomposeUnicodeForMatch(), but both didn't work well. I believe the code is ISO639, but there was no choice for the ConvertFromCodePage() formula. After using the DecomposeUnicodeForMatch(), some of the letter was converted to "?" mark.

Sincerely,

Kazumi

Re: How to convert broken letters?

Aguisande — Mon, 07 Aug 2017 20:04:42 GMT

Can you share a sample of the data you're using, so I can see the exact case?

Re: How to convert broken letters?

knozawa — Mon, 07 Aug 2017 20:34:55 GMT

@Aguisande,

I attached the sample .yxdb data.

Thank you for your help!

Sincerely,

Kazumi

Re: How to convert broken letters?

knozawa — Mon, 07 Aug 2017 20:40:39 GMT

@Aguisande,

grid.413735.7 (name: Harvardâ€“MIT Division of Health Sciences and Technology)

grid.10211.33 (label: LÃ¼neburg‚)

Above are some examples that I would like to fix.

Sincerely,

Kazumi

Re: How to convert broken letters?

Aguisande — Mon, 07 Aug 2017 20:57:15 GMT

Hi,

Reviewing the data you sent, I got this results:

The coding of the data seems to be UTF-8

In which format (.csv, Excel) is the original data ?

Re: How to convert broken letters?

knozawa — Mon, 07 Aug 2017 21:17:37 GMT

@Aguisande,

The original data is csv (type: Microsoft Office Excel Comma Separated Values File). I downloaded data from Grid database and combined all 11 data together. Attached the workflow.

Thank you very much for your help.

Sincerely,

Kazumi

Re: How to convert broken letters?

Aguisande — Mon, 07 Aug 2017 21:25:18 GMT

Using your workflow, I changed the Encoding of the label.csv file to UTF-8. All accents, foreign language characters and symbols are there (See below)... I think if you this to all your inputs, all characters should be right...

Re: How to convert broken letters?

knozawa — Mon, 07 Aug 2017 21:38:11 GMT

@Aguisande,

Thank you very much!

Sincerely,

Kazumi

Re: How to convert broken letters?

seven — Sun, 19 Jan 2020 23:36:27 GMT

@knozawa

You may not always have control over how the code page when inputting data. In any case, you can always use a function to change the code page. Use the MultiField Formula tool with this formula

ConvertFromCodePage([_CurrentField_], 65001)

Re: How to convert broken letters?

maleryx — Wed, 05 Feb 2020 10:00:09 GMT

Can this be done from a data input non csv? like SQL DB? I tried the method TS used but failed as well

Re: How to convert broken letters?

warrencowan — Fri, 10 Jul 2020 09:18:16 GMT

Great shout @Aguisande , saved me alot of time stumbling on this.

I had been finding and replacing these from my own custom list as I discovered them, and tbh was beginning to lose the battle and my will.

Alteryx is truly an iceberg and evey day's a school day.

Many thanks for being such a sterling contributor.

Re: How to convert broken letters?

Aguisande — Fri, 10 Jul 2020 15:09:40 GMT

Hi @warrencowan

Thanks for your words.

I'm so glad to be of help!

Keep unveiling the iceberg, you'll find awesome things beneath...