Alteryx Success Stories
Learn how Alteryx customers transform their organizations using data and analytics.STORIES WANTED
Showcase your achievements in the Maveryx Community by submitting a Success Story now!
SUBMISSION INSTRUCTIONSBuilding the Safety Culture at Polaris
Over the last 60 years, Polaris has earned the position of the global power sports leader in the snowmobile, commercial, on-road and off-road vehicle space with a vision to fuel the passion of riders, workers and outdoors enthusiasts around the world by delivering innovative, high-quality vehicles, products, services and experiences that enrich their lives. With leadership comes the great responsibility of fulfilling our commitment to the guiding principle of ‘safety and ethics always’. “We are dedicated to leading the industry, not only in innovation and performance, but in safety and quality – because we will accept nothing less,” said Scott Wine, chairman and CEO, Polaris. Using Alteryx, Polaris automated the process of identifying potential safety concerns from technical cases data. This solution was able to flag 2% more safety issues in warranty claims and it resulted in better customer service, greater customer satisfaction and retention with the financial benefit of over $450,000 after one year of administering the process.
To support this direction, the company centralized its product safety organization in the Post-Sales Surveillance department and enhanced its analytics capabilities with a goal to rapidly identify, correct and learn from customer product safety and quality experiences. Warranty claims filed by Polaris dealers are one of the many data sources for such experiences. In addition to providing the company with technical details around the quality issues with our products, technical cases coming from the dealers can also be the first sign of any safety concerns with the vehicles. When properly identified as potential safety issues, they can be promptly addressed and resolved with our customers through investigation and understanding of the root cause. Not only does it allow us to be proactive in responding to the needs of individual customers and provide us with an opportunity to rectify the situation, an effective process of recognizing safety concerns early provides the opportunity to make vehicle safety and quality enhancements that provide a safe and fun experience for all riders.
Historically, the responsibility for identifying potential safety implications in the incoming warranty claims has rested solely with a technical services team working with the dealers on issue resolution. To improve the consistency of recognizing safety issues in warranty claim data, we have augmented the manual process with a set of Alteryx predictive models that help to identify additional warranty claims for the Post-Sales Surveillance team to review. The predictive models help identify potential safety instances faster, which reduces the net impact to our customers.
Warranty Claim Process Overview
For a simple warranty claim, a dealer will submit the claim, Polaris will review it, make a decision to authorize or deny it, and the claim process ends. For more complex cases, there may be a back and forth between the dealer and Polaris, which can take time depending on the dealer’s responsiveness and vehicle part availability. For this reason, the decision was made to evaluate and score warranty claim records every day that the claim is modified in the system to identify safety cases as soon as possible (see process diagram below).
Data Prep for Modeling
The R-based predictive tools rely upon categorical fields as inputs to the model. All fields that contain nulls are assigned a value in order for the model to run.
The inherent difficulty with identifying which cases should be reviewed for further product safety consideration is that it’s a true needle-in-the-haystack scenario. To overcome this, we over sample on the safety flag field (indicating a potential safety record) to create a data set with predictive power. Without over sampling, we would end up building a model that predicts that every warranty claim should not be reviewed, and it would nearly achieve 100% accuracy.
Developing Early Life and Late Life Models
Due to the varying degrees of record completeness when scoring a warranty claim that was just created versus scoring a warranty claim that was just completed, we developed two different types of predictive models. The Early Life model uses fields that a dealer adds as a warranty claim is created, and the Late Life model uses fields added throughout the process or at a point of claim completion. This gives us the ability to more accurately identify potential safety scenarios at different points in the claim lifecycle.
Iteration upon Variable Selection
While building out the models, we used the built-in variable importance plots to understand the relative importance each variable has in contributing to the overall predictive power. Dozens of iterations were run to determine which variables have significant predictive capability and to understand the best performing mix of variables. The mix of variables and the total number of variables included varies by model for each of the four models deployed. In the example below for one of the Early Life models, you can see that four variables are being used to predict whether a warranty claim is safety-related, and the case type code field is the main driver in the model.
Model Accuracy Testing
Testing the accuracy of the models goes along with feature selection and the variable importance plot. Not only do we want to understand which variables are predictive, but we also want to test the model against data the model hasn’t seen before.
In our scenario, we want to pick a probability threshold for each model that will minimize the number of false negatives. This is a deviation from choosing the model that is the “most” accurate, as we want to ensure we don’t miss safety-related claims at the expense of reviewing a higher number of warranty claims that aren’t safety-related, or false positives. In the example below, you can see that setting the probability threshold to 50% results in reviewing 55 warranty claims that aren’t safety-related, but the model has accurately caught all safety cases in the validation data set.
Launching Models in Production
Each business day, the workflow builds new models using historical records to continuously learn from a dynamic data set. All newly created and modified warranty claims are scored against each of the four models, and records that meet or exceed a designated probability threshold for the model are manually reviewed.
Alteryx takes the records that need to be reviewed and sends an email alert to the Polaris safety team to decide whether the warranty claim is truly safety-related. One of the safety reviewers can then click through the case links in the email below to begin reviewing the case in further detail. Furthermore, any records that are flagged for safety will become part of the data set used to re-create models in future workflow runs.
The new automated procedure of identifying potential safety concerns using Alteryx predictive tools was intended to add consistency to already robust product safety processes and training at Polaris. The capability to identify more safety cases in warranty data and promptly respond to customer safety concerns goes to the heart of the customer-centric Polaris culture. With the focus on deepening the relationships with our existing customers and continuously driving product improvements, the early identification of potential safety concerns allows us to take an even more proactive action to identify products suffering problems in the field and immediately address them, thus maintaining customer loyalty and protecting the Polaris brand.
Even taking the most conservative approach to estimating the realized benefits based on the first three months since the launch of the process, the new capability was able to flag 2% more safety issues in warranty claims. That directly translates into the improved customer service, greater customer satisfaction and retention with the financial benefit of over $450,000 in additional product revenue based on the customer lifetime value after one year of administering the process. Most importantly, we are able to continue serving our customers with the highest level of safety and ethics in fueling their passion for riding.
Alteryx’s intuitive interface, built-in predictive functionality and Community support have enabled individuals without formal data science training on the Polaris Post-Sales Surveillance team to stretch the boundaries of our comfort zone to develop previously unattainable advanced modeling functionality. By witnessing the power of advanced analytics in the Post-Sales Surveillance group, Polaris is starting to accelerate its journey towards becoming an analytical powerhouse across the organizational structure.
“Polaris is making safety and quality a competitive advantage. As the result of our comprehensive approach and significant investment, we are seeing the tangible results of our efforts, both in the quality of vehicles we are building today and in our approach to preventing, monitoring and responding to safety matters,” said Ken Pucel, executive vice president of Global Operations, Engineering and Lean, Polaris.
