Parse Paragraph into Sentences and Ignore Abberviations with Periods

Question

Regex help (or Text Mining - Part of Speech?)

I want to parse each sentence in a paragraph into rows.

I run into a problem when a sentence begins (or any part of a sentence) contains an abbreviation with a period (e.g., U.S. Government).

How can account for abbreviations with periods when trying to parse each sentence of a paragraph?

ParseSentencesFromParagraph.yxzp

hellyars · Answer

@danilang

Thank you.

I don't know Python (yet).

I am going to try the second approach you suggest (regex).  More later....

I am surprised that the Text Mining Tools can't do it.

hellyars · Answer

@AndrewSu

Thank you.  This is only partially successful at parsing the real world data. I am looking at a few options to meld this with something I am trying.  More later.  Thanks again.

danilang · Answer

Hi @hellyars

Another excellent question from you!  As you've seen, the the answer is not simple and you don't want to spend time reinventing the wheel.

The best and simplest option for you would probably be to use a python tool and leverage the nltk library.  The authors of this library have done the grunt work to define all the edge cases for you.  There's an example here.

The second solution in that link contains an interesting python-only approach that uses a series of lists to define the edge cases.   The code uses a bunch of regex functions to replace the items in the lists with custom delimiters and then ends up with a series of sentences ending in <stop> (reminiscent of a telegram) so it should be convertible to standard Alteryx if you don't want to include a python tool.

Dan