This site uses different types of cookies, including analytics and functional cookies (its own and from other sites). To change your cookie settings or find out more, click here. If you continue browsing our website, you accept these cookies.
I'm somewhat new to Alteryx, and while my company has spent a lot of time looking at some very neat features, I wanted to get some kind of views / opinions on how well Alteryx could assist in improving some data processing / parsing.
Bullet points, for succinctness and clarity – hopefully:
·My company – in one way, shape or form, has presence in over 120 countries
·All of those countries are providing data (that can be read as XML) every second, of every minute, of every hour…you get the idea…
·We take the messages and load them in to a database for reporting purposes – so data accuracy and completeness is a challenge.
·The problem we have is that 120 markets are likely to be using a lot of different formats for constructing their messages.
·My company uses an in-built ‘mapping application’ that tries unravelling the messages (as best as programmatically possible) so that we can load the right data elements in to the right database columns.
·At this time, we use in-house developer dependent applications to assist us with the mapping problems / solutions.
My first question is: would Alteryx be a potential good for replacing the existing mapping solution?I suspect a not-insignificant amount of logic would need to be built in to any workflow developed.It’ll also be working all the time.Does anyone use Alteryx for this constant data processing purposes?
The reason I ask is that I want to see if we can achieve a number of things:
Can we remove the 100% dependency on development resources to help us see and troubleshoot mapping problems by using Alteryx?
Can my company shift the emphasis we currently have; telling 120 markets, you have to change your processes and procedures to map to a standard, to telling the 120 markets; tell us your standards and we’ll use Alteryx to program them.
I can appreciate your dilemma. We had a similar issue with our survey data platform, which also has an XML output. Now we were fortunate that the depth of the parsing is the same for most of surveys, however there are just over 100 different question types, which makes for a very comprehensive model. Alteryx can work for you, but you will have to focus on building some type of mapping mechanism, meaning a translation from your individual market to your standard. I was able to create a dynamic sql query based off of this concept which allowed us to import any survey with just the creation of the map. We store the map in the database and use joins and dynamic replace tools to connect it to the incoming data source. You can capture some of the efficiencies if you know that there are some aspects that can be standardized, it may even help you during the transition period to a standardized if your company still requires it.
Depending on your level of proficiency with Alteryx, you may even be able to parse your current applications to obtain your mapping. I was able to accomplish this with our PHP mapping, which greatly reduced our conversion times.
Another aspect that may help build a case for Alteryx is the clarity it does provide into the data throughout the entire process. It is easy to show even a business user where the problems are in the data. A common practice I have is to compare the initial input to what is going in the DB/reporting dataset. This also allows for convenient proof to show all of the data is making it through the entire process and if there are areas where the data is falling out legitimately it is accounted for down to the row level.
Please let me know if you need an example of the mapping or the dynamic aspects and I can provide them.