For those of you who may have missed last weeks post, we're starting a weekly Q&A Thread, called the Thursday Thought. Every week we pose a question to the community and invite members to share their experience. Even if you don't have an answer/solution to the question, you are still welcome and encouraged to participate. Think ahead... What problems could you avoid by having these processes in place?
What processes, if any, have you put in place to make workflows more understandable, shareable, and traceable?
Community Engagement Programs Manager
Additionaly to fill all the Annotations on every tool, I tend to use Data Containers to group all specific tasks inside (can't wait to the 11 version, where I just select the Tools and Add a New Container automatically), use a color code for them, like: green for data input, red for business rules, orange for resulting datasets, white for interface controls, etc..., and add a Comment to explain what that particular piece of workflow does.
Get a good understanding of what you’re trying to accomplish, check the data that you’re given, be prepared to iterate, iterate, iterate and get a working module ready for testing. I’m not preaching, “Ready, Fire, Aim!”, but I am suggesting that perfect planning isn’t an agile philosophy.
Create simple documentation. Some of my workflows are a mess and others are picture perfect. It all depends on the need. The documentation effort should be less than 10% of the total effort. I like to organize workflows with CONTAINERS. They make it easy to ZOOM to the logical components in a workflow and keep things clutter free. Where specific use cases drive decisions in the workflow, I like to use COMMENTS to help jog my memory. Annotation TAGS help me to simplify the display of verbose formulas.
I’m a big fan of peer programming. Your work will improve if you walk someone else through your workflow. Use just enough documentation to help you talk through the workflow. Remember, if you win the lottery, your friends at work will inherit your workflows.
There are lots of unknowns. Help to think through some of these before you finalize your workflow. If you’re going to run the workflow on your desktop you might do things differently than if it is running on the server. If you’re giving the workflow to a friend, you might do things differently again. Options, Advanced Options, Workflow dependencies is a topic to consider. Hard-coding file locations and names is more fragile than relative or UNC (Universal Naming Conventions).
Alteryx is an enablement suite of tools. They work together to harmonize the thoughts of artisans and the power of the computer. Alteryx allows you to design and implement your workflows freely, yet it also includes help to conform to standards and best practices for thousands of clients’ shops. Hopefully, you get a chance to influence what “Best Practices” means and learn that those practices are living, breathing documents.
In addition to the techniques already mentioned in the two previous responses, I try to use some additional techniques shown in the pdf attachment.
For an additional thinking topic, I offer the following. For additional strategies for solving tough problems in Alteryx, you can see this article: https://3danim8.wordpress.com/2016/07/01/alteryx-strategies-for-solving-tough-problems/
Click here to see many more Alteryx articles
Great stuff @aguisande, @MarqueeCrew, @Ken_Black! I have to say that I appreciate @MarqueeCrew's recommendation that "the documentation effort should be less than 10% of the total effort." Do you think that if a user finds themself spending more than 10% of the total effort documenting processes, it could be sign that their workflow is lacking in efficiency?
Community Engagement Programs Manager
I don't think that spending more than 10% of the total effort in documenting means that the workflow is lacking in efficiency. We should consider some other stuff to determine this. For example, the target of the workflow (building a workflow for a business user, may need extra effort in documenting, since it may not be versed in technical stuff).
Collaboration is other thing that may affect your effort in documenting (or it should).
Multilanguage sometimes makes your documentation process take an extra effort. In my case, being a Spanish speaking native, sometimes I found myself replicating information that's already there, but in spanish, so the effort increses.
For clarification, the 10% estimate for documentation is part of an agile principle. If while you are creating your workflow, you naturally add comments and keep things organized then you probably won't even notice the time that the organization/documentation is adding to your day.
While I am not personally developing workflow after workflow for an organization, I am in the business of sending workflows to users to answer questions or demonstrate examples. For this discussion, I'll discuss the 'what' and the 'why'. As for the what, the places in which I can document a workflow include:
1) Comment Tool - This tool is great because I see it used in three different ways, 1 - text, 2 - color, 3 - text/color. For point 1, I see effective use of the comment tool by underlying a group of tools and using text to describe the process the sets of tools represents. The second point is interesting because some customers I see have a standard for how a colorized comment tool underlies a group of tools. I have seen organizations color code data cleansing, data prep, analysis, business rules, internal rules, etc. Using the coloration method, a user’s can open a workflow at 30,000ft to fix a data cleanliness issue and knows to go directly to the blue comments.
2) Annotations - This is possibly the second most detailed way to document individual expression building tools and the most detailed documentation for every other non-expression tool. These annotations provide an opportunity to address tools configuration settings and the decisions driving the configurations. Most of the time, I encourage the annotation to be an overview of the purpose of the tool rather than the specific configuration.
3) Containers - I appreciate a good container that can document a process, allows me to toggle a section of a workflow on and off, and minimize tools at a click of a button. However, I will say that sometimes I'll come across a workflow made completely of collapsed containers. When I come into an ecosystem made of completely collapsed containers, I feel like I am getting the message "you shouldn't be here". Document as needed, but my opinion is to use them in moderation.
4) Meta Info - I can imagine a case where a team has an Alteryx user that heads out on vacation, and it only takes an hour after the analysts’ departure for Murphy's Law strike. All the information the team has is that they know the network drive that the workflow is sitting in, and even the directory that the workflow is in. But in that directory, there are 30 cryptically named workflows. For a poorly documented organization, the choices are 1) have someone walk through all the workflows in their entirety to translate and discern if the workflow is the desired workflow or 2) email the developer and wait for a response. If you have a team, the workflow meta info is indispensable. The meta info can track author, company, copyright, workflow description and more. Entering meta info into a workflow is a very small effort for a very valuable gain for a team. A user can save their meta info settings and easily populate it just a click.
5) Expression Documentation - I've often said that expressions are one of the most difficult parts in working with Alteryx, but they are POWERFUL. Because of this, I think that documentation around expressions is always overlooked. In this discussions, let's suppose we are working in the formula tool. There are a few things to successfully documenting a formula tool.
6) Workflow Naming – In the discussion of the meta info for a workflow, did you notice my mention at cryptically named workflows? Often, I am a victim of my own lax nature of naming workflows. However, I always get back on my nomenclature after losing a workflow that I KNOW is on my machine, but just can’t find it.
7) Gallery Upload Descriptions – Version management 101 dictates that when you create a new version, it is wise to document changes. When uploading a workflow to a Gallery, a workflow description IS that version management. The next time you pull the workflow down for edits and save to the gallery again, note the changes in the description upload to have an elevated version management experience.
8) Gallery Job Names – Since the gallery tracks results from workflows and apps, naming the workflow prior to running in the Gallery is an easy way to find the results when coming back into the Gallery a day later. Not to worry, though – it’s easy to rename the results after they have been received.
Naming the run:
Seeing the workflow results:
9) Scheduling – When the file name and cadence of the schedule is not enough to determine what the workflow is doing – consider putting a description on the schedule. Also, consider what happens to all schedules being maintained by a Server admin on the controller. A description can hint at exactly what the job is and why it is so important.
So now that we have discussed the ways to document workflows, the ways in which I apply all of these documenting procedures vary situationally. When it comes to developing a workflow, I tend to document it based on user needs. A few questions that I ask myself before I begin building a workflow are:
1) Is this a one-time workflow or going to be a repeatable process?
Should I create this workflow in the mindset that this will be used one time or many times and need to stand the test of time? The former scenario would probably have me minimally document – just enough to let the user understand what is happening in the workflow and know the impact of a single run.
2) Is this workflow only going to be used by one person or many people?
Should I create the documentation that addresses a user in their own language to better address their exact needs, or should I, in a utilitarian fashion, generalize the language to be best understood by a group of people?
3) Are these users planning on editing/updating/maintaining this workflow?
If users are planning to spend much more time working on the workflow after I have handed it off, I try to document the ‘works’. Give the most information to make the workflow as understandable as possible.
4) Are these users advanced or beginners?
Often, I see there being two ways a workflow can be developed. The most ‘understandable’ way or the most ‘optimal’ way. What’s more important? Speed or comprehension? How much data is going to be pushed through this workflow? Instead of writing one expression made up of 10 functions in the formula tool, writing ten expressions with one function in a formula tool can be easier to comprehend. I know users who will not write compound expressions in a filter tool, but rather create multiple filter tools with one expression apiece. While adding tools to the workflow creates another step in the process, it lends itself to improved collaborative work. If your users are beginners, maybe you pose a question like “Based on your data volume, would you rather have a workflow that screams and completes in 30 seconds or a broken-down and simplified workflow that takes 45 seconds?” If the workflow is only 15 seconds longer than a fully-optimized runtime scenario, would it not be better to save 10 minutes from an analyst’s day when they try to update the workflow?
In the end, I believe ‘comprehension first’ is, more often than not, the ideal scenario. We at Alteryx talk about an improved life in ‘hours and not days’. I think with the right documentation, re-visiting an old workflow and decrypting its true configuration and behavior can be made in the very same fashion that the tool was founded on.
One thing I forgot to mention, is that sometimes, I include the "performance indicators", based on my computer (as reference). It helps me see very fast what to spect in terms of time consumption, volumes and provides a quick reference for the "The workflow is running slow" question.