topic Re: Elegantly remove all ASCII characters outside the range 32 -126 in Alteryx Designer Desktop Discussions

Elegantly remove all ASCII characters outside the range 32 -126

jhollingsworth — Thu, 24 May 2018 01:25:14 GMT

Hello Artisans,

I'm working on a module that includes a geocoder tool. I'm running into an issue in which the data the users are submitting to the module are copied from a web page and sometimes include weird characters (e.g. ASCII 160). This causes the geocoder problems. And by problems, I mean that the geocoder can only find the zip code when without the odd characters, it can find the address.

What I'm looking for is the most elegant way to remove any characters from a text value that fall outside of the ASCII range of 32 -126. Any characters inside that range *shouldn't* cause the geocoder problems.

Extra cool points to anyone from Alteryx that will fix incorporate this fix into the Address tool. :-)

Cheers,

John Hollingsworth

Re: Elegantly remove all ASCII characters outside the range 32 -126

ToddW — Wed, 23 Sep 2015 15:51:49 GMT

I am definately in need of some extra cool points, so here is my most elegant way to remove ASCII characters outside of the range of 32-126. I was able to remove the undesired characters using RegEx (you can use the RegEx tool or the Formula tool for this) using the following expression:

[^x20-x7e]+

Everything within the brackets represents a specific set of characters to match. The x switch matches an ASCII charater using hexidecimal representation. x20-x7e translates to ASCII characters 32-126. The ^ preceeding the range indicates that we want to match anything NOT in the specified range. The + outside the brackets indicates that we want to search for this pattern in our string 1 or more times.

Using the RegEx tool:

Using the Formula tool:

Let me know if you have any questions.

Todd Williamson
Client Services Rep
Cool Points: 7

Re: Elegantly remove all ASCII characters outside the range 32 -126

jhollingsworth — Fri, 20 Sep 2013 04:01:58 GMT

Outstanding! That was exactly what I needed.

Whenever someone uses RegEx I think of this.

Thanks Todd!

Re: Elegantly remove all ASCII characters outside the range 32 -126

ToddW — Fri, 20 Sep 2013 04:42:13 GMT

Glad it worked out! Since posting my last response, a couple of my colleagues shared a simpler expression that accomplishes the same task:

[^ -~]

Apparently there is no need to use hexidecimal values for the ASCII charaters - you can literally type them in (please note there is a space between the carat^ and the dash-.

Todd Williamson
Client Services Rep
"Everybody stand back! I know RegEx!"

Re: Elegantly remove all ASCII characters outside the range 32 -126

sd — Fri, 08 Jan 2016 04:43:05 GMT

Todd,

I'm a little late with this...

I've been searching for an answer and your solution worked great! Thank you for the detailed post and for following up with a more elegant solution to your initial response.

Re: Elegantly remove all ASCII characters outside the range 32 -126

Inactive User — Fri, 21 Jul 2017 12:27:52 GMT

Don't forget the backslashes in the expression, which are shown in the graphic of the RegEx box but not in the accompanying text. [^\x20-\x7e]+

Re: Elegantly remove all ASCII characters outside the range 32 -126

NotQuiteClueless — Thu, 01 Nov 2018 15:38:30 GMT

You Sir, are a star.

Re: Elegantly remove all ASCII characters outside the range 32 -126

MarqueeCrew — Thu, 01 Nov 2018 16:06:58 GMT

@ToddW,

I like your expression (having used it myself), but wanted to describe the request in English and to offer @jhollingsworth an alternative solution.

Remove any characters from the incoming field that are outside of the range of ascii values from "space" to "~".

This expression can be written in a form that more closely matches these requirements as:

REGEX_Replace([Field1], "[^ -~]", '$1')

[...] is the set of values

[^..] is then, the values not in the set

[^ -~] uses (space) followed by a dash followed by a tilde is then, the values not in the range of (space) to ~ (tilde)

Cheers,

Mark

Re: Elegantly remove all ASCII characters outside the range 32 -126

MarqueeCrew — Thu, 01 Nov 2018 16:26:14 GMT

@ToddW,

.... just read a latter post showing the same.

Re: Elegantly remove all ASCII characters outside the range 32 -126

Jordken — Mon, 30 Sep 2019 18:23:17 GMT

This worked great for the most part: [^ -~]

However, I think I want to keep the carriage returns (13) and new line (10) characters as well. I'm still trying to figure this regex out, and I'm not getting it. Any help?

Re: Elegantly remove all ASCII characters outside the range 32 -126

BrettMRP — Tue, 01 Oct 2019 10:31:16 GMT

@Jordken Change the expression to read as follows:

regex_replace([_CurrentField_],"[^\xA,\xD,\x20-\x7e]+",'')

[_CurrentField_] being whatever your field name is if you are using the formula tool and not the multi field tool

Re: Elegantly remove all ASCII characters outside the range 32 -126

MarqueeCrew — Tue, 01 Oct 2019 11:13:29 GMT

[ -~]

That's the range as elegantly as possible.

Cheers,
Mark

Re: Elegantly remove all ASCII characters outside the range 32 -126

SideOfRanch — Thu, 23 Jun 2022 14:49:57 GMT

Is there a way to do this but return if the field had one of the untranslatable characters? I'm trying to identify them the fields and flag them with a 1 if they have a bad character.

Re: Elegantly remove all ASCII characters outside the range 32 -126

MarqueeCrew — Thu, 23 Jun 2022 15:02:55 GMT

MIN(Regex_Countmatches([Garbage Data],"^[ -~]"),1)

That might help.

Re: Elegantly remove all ASCII characters outside the range 32 -126

AGilbert — Fri, 20 Sep 2024 00:48:04 GMT

Adding another solution to the conversation.

Re: Elegantly remove all ASCII characters outside the range 32 -126

3I_ATLAS — Wed, 18 Dec 2024 05:28:39 GMT

Indeed elegant and something I will likely be using regularly to remove 'junk' and mystery hidden characters from data coming from different platforms. Thank you!
You deserve all the EXTRA COOL points Todd!