As Alteryx becomes more focussed on the Enterprise - it is important that we build capabilities that support the needs of large-scale BI.
One of these critical needs is dealing with heterogeneous data from different systems that use different IDs for every critical entity / concept (e.g. client; product)
Here's the example:
Problem:
- In any large enterprise - there are several thousand different line-of business systems
- Each of these was probably built at a different time, and uses a different key for specific concepts - like Client & Product
- Most large enterprises that I've worked at do not have a pre-built way of transforming these codes so...
- This means that any downstream analytics finds it almost impossible to give single-view-of-customer or single-view-of-product.
Solution option A:
Reengineer all upstream systems. Not feasible
Solution option B:
Expect some reference-data team to fix this by building translations. More feasible but not fast
Remaining Solution Option:
Just as Kimball talked about - the only real way is to define a set of enterprise dimensions, which are the defined master-list of critical concepts that you need to slice-and-dice by (client; product; currency; shipping method; etc) in a way which is source-system agnostic
Then you need a method in the middle to transform incoming data to use these codes. This process is called "Conforming"
What would this look like in Alteryx?
Setup
- We would use the connect product to define a new dimension - say "Product".
- Give this a unique ID which is source-system independant; and then add on the attributes that are important for analytics (product type; category; manufacturer; etc)
- Then decide how to handle change (slowly changing dimension or SCD type 0,1,2, etc). Alteryx should take full responsibility for managing this SCD history; as do many of the competitors
- We then create a list of possible synonym types (within Connect). For example - a product may have a synonym ID from your supplier; from your ERP system; from your point of sale system. that's 3 different IDs for any product.
- We then load up the master data - this is painful but necessary
In Use:
- I read in data into alteryx via any input tool
- I bring in a "Conforming" tool off the toolbox (new tool which is needed)
- It asks me which column or columns I wish to conform
- For each - it asks me which synonym type to use
- It then adds a translated column for me to use which ties back to the enterprise dimension - and spits out the errors where the synonym is necessary.
Impact:
In BI in smaller contexts, or quick rapid-fire BI - you don't have to worry about this. But as soon as you go past a few hundred line-of-business systems and are trying to do enterprise reporting, you really have to take this serious. This is a HUGE part of every BI persons's role in a large enterprise - and it is painful; slow and not very rewarding. If we could create this idea of a simple-to-use and high-velocity conforming process - this would absolutely tear the doors off enterprise BI - and no-one else is doing this yet!
+ @AshleyK @BenG @NickJ @ARich @patrick_digan @JoshKushner @samN @Ari_Fuller @Arianna_Fuller
