Alteryx Obscura: Turning Workflows into Crafts

[Carlithian]

This article is part of the Alteryx Obscura Blog series. Alteryx Obscura is the Inspire session dedicated to all things non-analytics, highlighting what the software “can” do rather than what it’s “supposed” to do. 

Ahoy! Carlithian here. I’m assuming anyone reading this knows what Obscura is, and if you don’t, that means you’ve ignored the link above…go to that first, and check out the other blogs in the series! Make sure you come back here though—I’ll be waiting.

So you 100% get the point of Obscura? The whole pointlessness of it all. Sweet, let’s begin.

I like making pretty workflows, the kind that generally take longer to document than they do to build. I could go on and on about how important that is, but the character count here is already too long, so you are just going to have to agree with me. The thing is, I don’t think these works of art get the credit they deserve, and I’m tired of it.

Seriously, I went to my parents, and not a single printout of my flows has made it onto the fridge, so I needed to up my game and get that attention that I so justly deserved. The question was how, and Inspire 2024 had the answer in the form of a scarf.

This beautiful scarf, perfectly modeled by @phottovy above, had me set on converting my workflows into crochet. Now I just need to get hold of something to convert these flows into crochet patterns.

The solution is an Alteryx app, so if that sounds like something you’re interested in, let's look into how it all works.

Introducing the Workflow Weaver

So we start nice and easy and bring the Alteryx workflow we want to process into Designer, now if you’re thinking “Izee, what are you on about, a .yxmd file isn’t a supported file type in Designer,” don’t worry, I’m getting to it.

Using the Input Data tool and making sure you can see all file types, you can select your workflow and bring it in as a delimited text file with no delimiters and no field names.

And now we have a chunk of text. This is what a workflow is: it’s what you make with all of the pretty tools we pull onto the canvas and configure to work out magic. Everything is stored in the XML, ready for us to extract.

The XML is split into a few main sections, for this we mainly care about the tools themselves, and these are kept in the Nodes tags, with each tool having its own Node section with useful info like the Tool ID, Tool Type, Canvas Co-ordinates, and even the Configuration.

We can parse this data out so we can have a breakdown of what is in the workflow, and can even use this handy dandy Designer Tool List to convert them to nicer names.

Getting the Tool Patterns

Now that we know what is in the workflow, we need to convert it into something (a pattern) to help with our crafting. This is where the amazing work of the Crafteryx group comes in. A small group of Alteryx enthusiasts who bonded over their love of crafting came together to make many fun projects for the Alteryx Community, and I decided to steal this hard work (with the blessing of the creator, like every good thief) and build it into an Alteryx Database file.

To save me needing to have the image saved where the workflow can reach it, or even to pull it into the workflow every time, I opted to save the converted image directly to the yxdb, ready for future runs.

You might notice all the null rows. This is because Alteryx has too many tools, and neither @aiahwieder nor @PuffinPanic has had the time to create every single one of them. 

Now we join this database to the data we have parsed from our workflow, and you are ready to craft.

… This is where the build should have ended, the goal was reached, and I needed to start with the crochet part. But… I wanted to make it pretty. No point having a process to help you immortalise the art that is your Alteryx workflow if the output is ugly. So how to make things not ugly?

Source: GIPHY

Add a cover page with a cool image

This is easy enough with the base functionality of the Report Text tool and some good old-fashioned artistic talent, and we are good to go.

Color code the contents page

All the tools in the same pallet will use the same color of yarn, so why not color the contents to match. The Table tool will let you change its style in the Default Table Settings or by creating rules, but there are a lot of pallets, at least three that I’m aware of, but apparently there are more out there, and I’m not setting up 3+ table tools just to have this look pretty.

Obviously, it is time for a Batch Macro (technically, I made two, but since I’m lazy and also don’t want you to read the same thing again so we can just look at one of them and you’ll get the gist).

Yet again, we jump into the world of XML, this time instead of looking at the contents of a workflow, we are going to alter the settings of a tool during the workflow itself.

There’s a great blog out already about the amazing work done by @TheOC and kindly typed up by @mceleavey I recommend checking out.

Here we batch in data based on the color palette of the tools. This filters the data but also updates the Hex Code used in the settings of the table tool, giving us a much prettier output.

Look how pretty those colors are, let it distract you from the 137 Report Text tools you apparently need to crochet, if all the typing isn’t giving me enough hand pain, the hooking certainly will.

Getting the Workflow Pattern

At this point, I thought it would be pretty enough for a pattern, and that I could be done. But really, the pattern should show you the intended outcome. So we are still missing the image of the workflow itself, and we need to get that using Base A. No cheating here—sticking with Base A till it makes me cry.

Source: GIPHY

So we know what tools are needed, and we know their relative position from each other as we have the coordinates.

Now to get the third bit of info that we need from the XML. We need to know which tools connect to each other. All this info is held in the helpfully named Connection tags.

Between <Connections> and </Connections> we have all the connections, and then each individual connection is between <Connection> and </Connection>.

The individual connection will give you information on where the connection is, its origin, destination, and type, and where appropriate, it will also include a connection name.

A good reason to name your connections (ignore that I haven’t done that in the example below)

So now we have a connection, the tool it comes from and where it goes to, or more importantly, we have the coordinates of those tools. Basically, everything we need to make a polyline, and I'm hoping the Spatial nerds in the crowd can see where I'm going with this, as it’s time to become a cartographer and save that map as an image we can use in our output.

If all the tools can be placed on a map with the tool image as the custom point, and then layer in all the polylines we have our workflow. It sounded really straightforward, but I was very wrong….turns out to have a custom image, it has to be associated with a unique connection to the Report Map, which brings me back to my earlier complaint—Alteryx has A LOT OF TOOLS, and you should know I’m not about hard-coding all of that in, so another bit of XML wizardry is needed. It’s time to build a whole workflow.

Source: GIPHY

We create two new yxdb files: one with our tool data and the other with the polylines to represent the connections.

Now we need to write the full workflow XML, some of this is fully fixed, such as these Data Input tools, and the Render tool. We know what Tool ID they will have, their configuration, even where we want them to be on the canvas. So this is already written up and ready to go.

Other parts are created dynamically, and this is the annoying part. Every unique image that we want in the map needs to flow into the Report Map as its own data stream. So that means it needs to have its own Filter tool, with a specific connection running to and from it (and to keep things clean, we name that outgoing connection ready to be referenced in the Report Map tool. Then, the Report Map tool needs to be fully configured for all of these connections and layers.

Once the XML has been generated, it needs to be stacked together very carefully before saving it using those same settings as we use to get the workflow into Designer, and you get something like this mess of stacked filters.

We even have error handling, making use of that lovely missing tool block for anything that we don’t have an image for in our database. Have to plan for things failing 😀

Now we have a workflow to generate that image we need, but there is no way I’m running two processes, so this needs to run during my original workflow. Thankfully, the amazing @MarqueeCrew  and @AdamR made the perfect tool for this: the Log Runner. I’ve configured this one to look at the workflow that’s being generated and simply run it, that way the generated image can be called into the workflow and added to the PDF pattern.

Though the real MVP of this has to be the Control Container, you might have spotted them in these last few screenshots (if you haven’t…pay closer attention).

These make sure that everything runs in order, that the data sources are created before the sub workflow is, that the workflow isn’t run before its been Frankensteined together, and we aren’t trying to access an image that is yet to exist.

Now everything is ready to be stacked into a beautiful PDF, and we are ready to start crafting.

The specific workflow I decided to crochet took a disgusting amount of time, and I do not recommend it, but that’s probably because I build workflows faster than I weave them.

Conclusion

I guess this is where I explain the point of all of this, something insightful that makes all the work worthwhile. Honestly, I just wanted to have an outfit reveal on stage and to shout out the amazing group that is the Crafteryx Community.

So this is really more about the journey than the destination, but let's look at the main things I’ve been yabbering on about here.

1. Dissecting an existing workflow

You can read the underlying XML of any unencrypted workflow, and this could be used for useful things like finding that one workflow in the countless you’ve created that has a specific tool in it.

2. Dynamically updating a tool

Any tool can be made more dynamic (translation = better) with a little bit of tweaking on the fly.

3. Dynamically creating a workflow

Entire processes can be created if you need them to be. (Make sure you actually validate things work though, don’t just start a mass of workflows creating more workflows….)

4. Like Crafting? Join Crafteryx

Drop me a DM if you’d like to join our ever-growing community!