topic Re: Challenge #63: Data Preperation Using Pivot Tools in Weekly Challenges

Challenge #63: Data Preperation Using Pivot Tools

JoeM — Tue, 13 Sep 2022 19:10:39 GMT

The solution to last week's challenge can be found HERE.

If you have even been shy about getting to know the crosstab/pivot tools, this challenge will serve as an excellent guide to becoming better acquainted with the tools.

An organization receives an input file like this every month or so. Additional columns are added to the file each time, representing data for more weeks. They need a workflow that will pivot the data so that each week end date is included on a row for each DMA number in column 1. The workflow needs to allow for the fact that additional columns will be included in later months.

Re: Challenge #63: Data Preperation Using Pivot Tools

patrick_digan — Mon, 27 Mar 2017 16:01:52 GMT

@JoeM I look forward to someone else providing a much slicker solution. My methods are not fancy, but they do work. I made use of a generic macro I use, so I've included it in my yxzp file.

I first selected the relevant rows of data with a select records tool. I "Excel Transposed" the data which is a simple macro I created a while back to carefully transpose and then crosstab the data to match excel's paste special transpose option. I used a formula tool to get the dates formatted, a multi-row formula tool to fill in missing dates, and a formula tool to get the column heading for the total row. I then used a transpose crosstab combo to re-orient the data back.

Re: Challenge #63: Data Preperation Using Pivot Tools

MarqueeCrew — Mon, 27 Mar 2017 17:56:37 GMT

@patrick_digan,

Slick? I think not. I did take a different tact than you did. I don't use the select records tool often (and I still didn't), so thanks for the reminder. I ran into several bumps in the road:

The last column only has data for Inside Sales (Outside & Total Sales columns are not present)
The general unhappiness of spaces & special characters when using the crosstab tool.

My approach was more lengthy than yours was. I didn't run yours, but I did test my output against @JoeM's output where I got identical results.

When working with clients and prospects I often get Excel "Reports" that need to be converted to DATA before real analytics can be executed. This often is time consuming. This challenge was work for me. I beat it into submission.

Cheers,

Mark

Re: Challenge #63: Data Preperation Using Pivot Tools

alex — Mon, 27 Mar 2017 19:06:44 GMT

I didn't provide much documentation on this one, but it should be fairly easy to follow if you download the workflow. I tested it by adding a new column and amounts to make sure it would handle new columns. Worked as expected. @patrick_digan - I'll have to check out your macro when I have time.

Re: Challenge #63: Data Preperation Using Pivot Tools

alex — Mon, 27 Mar 2017 19:22:10 GMT

For extra credit I'll post my first attempt at this solution with no documentation. In this version I kept the input file that I added an additional column to. Still works and slightly faster than my previous posted solution. Why does cross tab tool add underscores to column headings? Thanks for the challenging exercise @JoeM.

Re: Challenge #63: Data Preperation Using Pivot Tools

Natasha — Mon, 27 Mar 2017 19:42:56 GMT

My solution is definitely not concise but I got to the required output. Looking forward to inspect other's neat solutions :)

Re: Challenge #63: Data Preperation Using Pivot Tools

NicoleJohnson — Tue, 28 Mar 2017 16:59:52 GMT

Solution attached. Most similar to one of Alex's solutions, I believe.

However, I think I also uncovered an issue with the output results as posted in the original workflow - in the case where there was not both an Inside & Outside sales column, therefore no Total Sales column for that week (in this example, week 2016-04-24), then the Total amount for that week was showing as zero in the posted output results, despite there being an amount for Inside sales. So I added a formula at the end to check to see if Inside POS + Outside POS = Total POS... if it didn't, then I replaced the Total POS value with Inside POS + Outside POS. Which is also a good check in case someone posted the wrong value in Total POS to begin with.

Hope this helps! :)

Re: Challenge #63: Data Preperation Using Pivot Tools

alex — Tue, 28 Mar 2017 15:14:39 GMT

@NicoleJohnsonNice auditing work! I checked my results against the answer, but did not check the answer itself.

Re: Challenge #63: Data Preperation Using Pivot Tools

pvara — Tue, 28 Mar 2017 16:09:09 GMT

Hi Nicole can you please repost your alteryx solution? The link points to some XML file. Thank you.

Re: Challenge #63: Data Preperation Using Pivot Tools

JoeM — Tue, 28 Mar 2017 16:32:03 GMT

@pvara, I had no issues downloading. Is your machine's default program from XML notepad or a similar program? Does the downloaded file show as .yxmd?

Re: Challenge #63: Data Preparation Using Pivot Tools

pvara — Wed, 12 Apr 2017 17:37:33 GMT

@JoeM the link does show https://community.alteryx.com/pvsmt99345/attachments/pvsmt99345/weeklychallenge/549/1/challenge_63_Results(NicoleJohnson).yxmd) However, the when clicking on it I see XML Code. I am using i.e 11. I will try it with another browser.

Thanks

Pete

Re: Challenge #63: Data Preperation Using Pivot Tools

pvara — Tue, 28 Mar 2017 16:57:39 GMT

Quick update the link works with Chrome, not with IE, or Safari...

Re: Challenge #63: Data Preperation Using Pivot Tools

NicoleJohnson — Tue, 28 Mar 2017 17:03:06 GMT

I added the file initially using IE11... just re-added it using Firefox, and the link appears to work correctly now opening from any browser... so I guess in the future, I will NOT be using IE11 to add files! Who knew?? :)

Re: Challenge #63: Data Preperation Using Pivot Tools

Garrett — Tue, 28 Mar 2017 17:26:36 GMT

Here's my take.

Re: Challenge #63: Data Preperation Using Pivot Tools

JoeM — Mon, 03 Apr 2017 03:05:02 GMT

The solution is posted. Lots of good and different ways to solve the problem this past week! Let me know what you think!

Re: Challenge #63: Data Preperation Using Pivot Tools

SeanAdams — Sat, 08 Apr 2017 08:49:57 GMT

very similar solution to @Natasha

Re: Challenge #63: Data Preperation Using Pivot Tools

markusyk — Tue, 11 Apr 2017 13:02:17 GMT

One more solution.

Re: Challenge #63: Data Preperation Using Pivot Tools

Coxta45 — Mon, 08 May 2017 17:12:00 GMT

Here's my best shot!

Challenge 63

Re: Challenge #63: Data Preperation Using Pivot Tools

estherb47 — Wed, 10 May 2017 13:19:03 GMT

Chose to go a filter route to remove the extra rows at the top instead of using select records or sample. In my mind, a more dynamic approach to changing data.

Otherwise, nothing very original in my solution

Re: Challenge #63: Data Preperation Using Pivot Tools

Joe_Mako — Fri, 12 May 2017 21:26:23 GMT

If we ignore the requirement of making this data driven, and allow for a manual workflow, then this can be done in as little as two tools:

- Select Records to remove all the headers
- Arrange manually configured to do all the reshaping

I then took on the challenge to make the Arrange tool data driven for this situation, no cross tab or transpose tools, and practiced writing XML to configure the tool in a macro.

Finally, I made a workflow similar to @patrick_digan