Data Profiling - Pattern Analysis

[froseph]

Hi,

One feature which would be of significant value for Data Profiling is Pattern Analysis, with a view of pattern frequencies. This is available in many Data Profiling tools, including the free Talend Open Studio:

Is this something which can be easily produced in Alteryx, or is this a feature on the Data Profiling functionality roadmap?

Cheers!

[Emil_Kos]

Hi @froseph,

If you want this idea to get implemented in the Alteryx Designer you should post it in this place.

https://community.alteryx.com/t5/Alteryx-Designer-Ideas/idb-p/product-ideas

If there will be a lot of positive feedback it should get implemented sooner or later 🙂 

[smoskowitz]

Hi @froseph 

If you add a browse tool to the end of your data and then click it after a run, the data is profiled in the configuration section. Were you aware of this or am I mistaken as to your ask?

Thanks,

Seth

[froseph]

Thanks Seth - This shows frequency tables, but not frequency tables of common (and uncommon) patterns - Unless I am missing something?

[smoskowitz]

You could also try the Field Summary tool in the Data Investigation tab. That might give you further detail. There is also a Frequency Table tool, but you might need to download the extra tools. 

Seth

[smoskowitz]

This will give you a few extra tools in your data investigation tab that you might find of use.

Seth

[froseph]

Not quite what I'm looking for - in essence the tool should recognize common patterns in string fields, translate these into a kind of pseudo-regex and then produce a frequency table of those patterns as opposed to the values, a la:

Patterns - this is what we want:

versus Values - this is not what we want:

Note that screenshots are from Talend Open Studio - not Alteryx. 

[smoskowitz]

Another option is the summary tool. Just Groupby the Value use a Count Records tool to get a total amount of records. The append the Count Records to each record and do the math.

Based on what you are showing, it looks easy enough to create. I dropped in a quick mock up for you.

I Hope this helps.

Seth