Macro Testing Tools

[SeanAdams]

So - given the importance of Macros - it would be valuable to have the ability within Alteryx to generate a test harness with test data that ships with the macro (this way you can maintain and enforce regression testing)

For example:

- Macro that takes in 2 numbers and adds them

- Alteryx would look at the Macro to determine the input types, output types (in this case - two integers; with an integer output)

- Based on this, it could walk you through creating a fairly robust test harness that allowed the user to specify a set of inputs, and prompt you to also include things like blanks; negatives; etc (boundary values; deliberately destructive values like % or ' signs in strings; etc)

[AdamR_AYX]

Not that it is exactly what you are asking for.  But if you are interested in testing macros I wrote a series of blog posts that might be of interest:

http://www.chaosreignswithin.com/2015/03/testing-alteryx-macros-part-1-question.html

http://www.chaosreignswithin.com/2015/03/testing-alteryx-macros-part-2-our-first.html

http://www.chaosreignswithin.com/2015/03/testing-alteryx-macros-part-3-comparing.html

http://www.chaosreignswithin.com/2015/04/testing-alteryx-macros-part-4-testing.html

[SeanAdams]

Thank you @AdamR_AYX - I'll go through these over the weekend.

(and by the way - thank you for all that you do at ChaosReignsWithin - the CREW macro and the articles here are top quality and they really do bridge a need.

We think a lot about testing in our world, especially how to reduce the effort to get fully automated testing up and running.    Complexity will continue to increase as will the pace of change, and the only way to keep ahead of this curve, is to ensure that everything we build is self-certifying so that when change does happen we can declaratively know if something has broken or if we remain safe in the face of this change (this problem is make more acute in heterogenous IT environments like large corporates where upstream changes can have unanticipated side-effects downstream)

So, one of the challenges we face (as a community of professionals who use technology in various forms to solve business problems - be it Alteryx or RPA tools or just plain Java code) is reducing the time from start to a fully loaded test harness - preferably without the agonizing & painful process of hand-crafting every test case and attribute yourself.    This is an exciting area - and it will continue to accelerate our pace of innovation but in a way which is safer at the core.   

There are promising developments on the horizon - things like fingerprinting an IT flow to see what attributes flow through it to back into test cases or to look for low-frequency events which signal higher value test cases (there's less value in testing for the behavior of yet another Black Lincoln Towncar on the highway between NYC and JFK airport than there is in testing for how the system reacts to a low-frequency event like a random pedestrian on the highway or a small motor-scooter); and with modern processing power we can easily use computers to build test sets like least-pairs, or to inspect the actual code/flow to see what would be useful boundary values (e.g. if you're doing string conversions, then useful test cases are blanks and nulls).

Thank you again for sending these over @AdamR_AYX and for the work that continues to be done on ChaosReignsWithin - it truly is a great asset to the community, and I hope that we can take some of your great thinking, and use it not only to do great testing, but to find ways to accelerate; automate; and simplify great testing so that this becomes the standard rather than the exception.

Have a good weekend

Sean

[Samanthaj_hughes]

I like the idea of being able to package it up with a few test cases to prove that I tested it properly. 🙂

[TheOC]

This would be a great addition! 
Especially within macros on the Gallery, it would massively increase the reliability of the tools listed by other users (and troubleshooting on the other side of the download).