The Anatomy of an Analytics Application

[GarySp]

How many of us honed our dissection skillsAnatomy - an analysis or minute examination. 

The science dealing with the structure.  Dissection of all or part in order to study structure.  Anatomy.  It's typically dealing with plants and/or animals.  But, the definition has been expanded to anything requiring minute examination.  

Before we examine minutely an analytics application, let's take a look as to WHY  analytics?  You probably already know that Analytics is more of a methodology of using mathematics and statistics to gain valuable knowledge from data (or turn Data into Information).  Analytics itself is an overused term.   There are 5 major categories of Analytics:

1- Statistics 

Statistics are what "most" see as analytics ... these are sums, averages, mins, maxes, standard deviations, etc.  Things I can get out of Excel.  Hindsight.  What happened.

2- Data Mining

Correlations, regressions, decision trees ... used to answer these types of questions:

• What's the propensity of <insert thing> happening?
• What's the likelihood that a person will do <insert thing> based on <insert factors>

3- Forecasting

future, time based predictions based on past events ... used to answer these types of questions:

• How many people do I have to have scheduled for <insert future event>?
• What the <insert thing> going to be on <insert future date> with a 95% confidence?

4- Text Mining

not just word counts, but predicting true sentiment.  If I said, "Paris Hilton likes the Paris Hilton", that's not 2 counts of Paris and 2 counts of Hilton.  It's the identification of a person, a place and a sentiment.

5- Optimization

armed with all the above, and with a series of "good" outcomes, what's the BEST course of action to take?

So WHY  Analtyics?  To gain Insight and Foresight.  To help you make good decisions based on past actions.  So you can be better than what you would be.  To take the guesswork, gut-feeling   out of decisions.

Great.  But HOW?

Glad you asked.  Every analytics application is broken into two parts.  Strategic and Operational.  Let's dissect each one.

Strategic

The model build process is an art form, predominantly performed by data scientists or statisticians - with an assist from your IT data wranglers.  The determination of which technique (or techniques) to use.  Getting the data shaped "just right".  Determining the data to use to model, validate, and train.  The iterative nature of the process.  Shape data, use some techniques, validate the results, see if you can reshape with different (maybe better) data, use a different technique, validate new results ... until an analytic model (think math) is deemed satisfactory to answer the question.  The strategic side of creating an analytic application can be very time-consuming, and, this activity operates on a SET  of data.  The output of this exercise is an analytical model - or math.

Operational

Not art, but science, predominantly performed by the IT information systems teams - with an assist from your data scientists.  It's how to take the strategic output - the model, and move it to production.  Make the model run over and over when called.  It's shaping the input data as defined by the model, executing the analytic, and moving the resulting output to the appropriate location.  Implementing the operational will take time as well, but once implemented, the model is readily callable, and this activity operates on a ROW  of data.  The output of this exercise is a productionized, operational model, giving results.

Finally, the Anatomy

The anatomy is a joining of data and processing - in steps.  Simply put, the anatomy of any analytics application follows this pattern:

• Operational Data flows through a Data Credibility process that identifies relevant data for the problem.
• This Relevant Data is then transformed into Information using one (or more) of the above techniques. (as an aside, we all know Data and Information are different)
• The Information is presented for decisioning.

Data                      ->          Process             ->            Data                 ->                   Process                             ->    Presentation

(Operational)        (Data Credibility)         (Relevant Data)         (Analytic Transformation)              (Decisioning)

Simple.  Pretty much EVERY analytics application follows this flow.

Now that you know the anatomy, go forth and dissect your problems!