This site uses different types of cookies, including analytics and functional cookies (its own and from other sites). To change your cookie settings or find out more, click here. If you continue browsing our website, you accept these cookies.
I'm new(ish) to Alteryx, so apologies if this is a simple question.
I run A/B/Multivariate tests for a company, so am often looking for tests to understand whether my treatment group has outperformed the control (baseline) group. The easy way to do this in Alteryx is to do a Test of Means. However, recently I realized that this tool uses the Welch's T-test which assumes normal distributions.
Often, my data is NOT normally distributed.
(For instance, if we send out a message to get people to register for a new product, Only 5-10% might register, and then 90% of those who register might buy 1 thing, versus 1% buys 10 things, etc. - i.e. very right-skewed. Now, I want to understand if my test increased revenue.)
SO I understand there are other tests/methods out there (e.g. Mann-Whitney U Test, Bootstrapping) that might be more fitting for non-normally distributed data, but I can't seem to find information about whether any such tests exist (or can be bundled together to exist) within Alteryx or in the gallery. Anyone with advice?
Currently, none of the standard tools in Alteryx are designed to run a Test of Means against non-normal data. As @CharlieS suggested, you could build this using the R Tool and the wilcox.test() function. Out of curiosity, have y'all tested the normality of your data to ensure that it is not normally distributed? (the Distribution Analysis tool can help confirm this).
Also, have you had any issues with the AB Testing tools? If so, can you briefly explain what the issue is, and we can try to help you work through them. :)
There is a tool package within Alteryx for A/B testing.
BUT . . . . to build on what Charlie said, you can use an OverSample tool to populate more of the rare situations that you are interested in. And then you can build out a difference in difference equation in Alteryx.
The R tool is definitely the right way to go about this and I agree with using bootstrapping to get it done.
Unfortunately I cannot get the R tool to run without an error using the 'boot' library in R. This core code running in RStudio DOES NOT run in the R tool. I've commented the Alteryx-specific code that I'm using in the R tool: