Maveryx Success Stories
Learn how Alteryx customers transform their organizations using data and analytics.STORIES WANTED
Showcase your achievements in the Maveryx Community by submitting a Success Story now!
SUBMISSION INSTRUCTIONSBarnes & Noble Drives Profitability Using Test and Learn Macros
Overview of Use Case
Barnes &Noble was founded originally 150 years ago by Mr. Barnes and Mr.Noble. It was bought by Leonard Riggio about 50 years ago and now it has 600 stores all across the country. Barnes & Noble uses Alteryx testing macro tools to create analysis of retail store tests and one-sided statistical tests; in unusual test timeframes. With the power of Alteryx AB testing and macros, Barnes & Noble can task a single analyst with managing dozens of tests simultaneously and provide business leaders rapid insights with a solid statistical foundation.
Before Alteryx, retail store testing was inaccurate, confusing, and time-consuming. We have a tremendous range of products that we sell at Barnes & Noble, such as gifts, music, toys and we still sell DVDs. There's a lot of things that we can get involved in testing to drive more profitability and drive repeat visits at Barnes & Noble. We can test if Bestsellers would do well in the back of the store or test where a café would go in the store. But to do some really good tests, we need really good data and results, and then we need to do this very quickly.
We wanted to customize the A/B Analysis and A/B Controls tools to Barnes & Noble, something that could be a lot more modular that we could just plug certain numbers in. We did not have enough data to meet a 95% statistical significance level using the A/B Analysis tool. We wanted to measure the results, not the method. The indexed values from Alteryx A/B Analysis tool were too difficult to show business partners. Last, we needed a method to measure a single store performance.
We have a group of stores where we do a change. Then we have a group of stores that we're comparing this change to, where we don't do anything. That measurement between them, that's the analysis that we need to provide to our business partners.
Test Analytic Success Factors
|
Strong Baseline |
Analytic Rigor |
Clear Conclusions |
|
What is a valid comparison?
|
What is the effect of the test?
|
What happens when it’s over?
|
Why build a macro?
- Enter EDW queries directly
- Satisfy requests from business partners for specific data
- Single-store results could be provided for deep dives into data Multiple macros covering numerous KPIs
- Quick selection of test dates and KPIs
- Built once, use again and again
- Use R and Python to prototype ideas and then bring that into Alteryx via Macros
Using Alteryx’s built-in R tool and expanding the library with additional packages, we can parameterize the analysis wireframe and make it easier to understand to non-programmers. We decided to use the power of Alteryx Macros to incorporate different R packages into our test and learn workflow as well. We used MarketMatching to determine the closest match from one unit to another group of unit, and CausalImpact to measure the effect of the test.
The Barnes & Noble Testing Suite
- Determine the goals and the KPIs of any proposed test with our business partners
- Test store selection with Store Operations and Merchandising
- Determine length of test and operational parameters
- Generate control stores using the BN Control Match Macro and send to store operations
- Measure test results with Group and Store Analysis Macros
- Present results using the BN Test Presentation Report
We have two test analysis Macros to replace the A/B Analysis tool that's built-in to Alteryx. The first one is our Group Analysis Macro, which assumes that we're going to measure the entire impact of the test on every single store in the test group. If we have five stores that are matched to five control stores each, equaling 25 control stores to this group, then it will calculate the results over that five test store group and the results over the 25 controls store group and make a comparison.
Then our other Macro, the Store Analysis Macro, it's another Batch Macro that iterates over each individual test store comparing only to the five matches. This way we can dig into any outliers or anything that's strange and that's going on between any control stores or any test stores as well. And then we send all the results as CSV files into PowerBI.
When we started using Alteryx we immediately jumped to hyperspace. It was probably five months from going from the Excel methods to building the workflow and then probably another two or three months building these Macros. We had results within just a couple of minutes rather than several hours in Excel and also we could connect directly to our data warehouse.
At Barnes & Noble, we have some people that know R, some people don't know Python, some people will just use Alteryx. And so we can use R and Python to prototype ideas and then bring that into Alteryx via Macros. And that means that we can share ideas within the team. Everyone on the team can adopt those ideas and be able to leverage them easily.
The entire PowerPoint presentation can be found here.
