Community Spring Cleaning week is here! Join your fellow Maveryx in digging through your old posts and marking comments on them as solved. Learn more here!

Engine Works

Under the hood of Alteryx: tips, tricks and how-tos.
levsol
5 - Atom

Auto Documentation Tool

 

Say someone built an Alteryx workflow at your company. What does it do exactly? How does it work? Can you trace it step by step?  And just how did they configure these pieces? 

As data analysts and engineers, we take it for granted that we can readily convey our understanding of data flows to other analysts and engineers. As business drivers and decision makers, our business rules, processes, and decision criteria are clear to us and our teams. 

But as business and data teams become ever more integrated to adapt to accelerating business needs, our requirements and processes grow ever more complex.  

As a result, what once was an innocent question born of curiosity grows into an ever more cumbersome task, making you ask, What does this workflow do, exactly?

Enter the world of documentation. Being a data consultant, I frequently see the same theme play out. Users love building Alteryx solutions, but they often face business situations like these:

·        They need to be able to onboard new users and make sure they can support processes developed by existing and off-boarded users

·        They need to assure compliance and audit teams that everything they build is auditable and reproducible

·        They need engineering to work with them to access the data

·        They need to build confidence with engineering teams that they can build and manage their own automated processes

·        Alternatively, they should be able to sketch out tested and verified designs which they can easily hand to engineering teams (who will then manage implementations)

Whatever the specific situation, the users need to be able to quickly build documentation in parallel with developing and iterating on their workflows. 

When we developed an auto-documentation utility for Alteryx workflows originally, we had positive feedback, but there were a few consistent shortcomings inhibiting adoption:

·        It did not support workflows with containers

·        It did not provide an order of tools (the document would not be helpful in rebuilding a workflow)

·        It did not provide enough details about the configurations of various tools to reproduce the workflow

In addition, we saw a recurring need for sketching images of workflows for easy reassembly. Never to be the ones to shy away from a challenge, we took it upon ourselves to rebuild the documentation utility from the ground up to address these needs.

Enter the latest and greatest in workflow auto-documentation, the Auto Documenter version 2. Simply click Run to get a download link.

Check out a sample of the outputs: 


levsol_0-1585076402524.png

 

 

 

  • Read the full documentation here for installation and usage instructions.
    Check out our demo video
  • Find us online
  • Contact us directly for additional feature requests and support 
  • And don’t forget to check out our other public projects on our Gitlab

 

Comments
ppoorab1
7 - Meteor

This is a life saver, thanks a ton!!

Take a thousand bows.

JosephSerpis
17 - Castor
17 - Castor

This is awesome!

Accrual
7 - Meteor

This is awesome

levsol
5 - Atom

Hello Everyone.

 

By popular demand, we are publishing an alternate version which groups workflow sections by containers:

Please find the newest version on our public GitLab: Here 

 

The version Imager_2020_05_19_Container.zip uses container-based rather than join tool based sectioning. It was built using Designer version 2020.1, so older versions of designer may encounter some bugs on opening- you'll likely need to re-save the files contained in the zip with whatever version you are using.

indieching
5 - Atom

I get this error when running - is somebody able to shed some light? 

 

indieching_0-1592387093054.png

 

levsol
5 - Atom

@indieching 

The last line of the error message suggests pip installer returned a non-successful exit status, which means the two required python packages were not able to install.


The most common cause of this is lack of permission for an admin version of Alteryx Designer installation.  To resolve this:

  1. Right click Alteryx Designer icon and select run as administrator.
  2. Browse to the "Image Workflow.yxwz" file in the autodoc directories and open it.
  3. Use the wand icon in Alteryx Designer to run as analytic app.

Now the app should run with admin permissions and will be able to install these two libs in the Alteryx python environment. Once this runs successfully, you should be able to run the app normally (by double clicking the analytic app from the install directory).

 

 

indieching
5 - Atom

@levsol  

Thank you - running it as Administrator fixed the issue!

pbeeram
6 - Meteoroid

@levsol

 

I was able to auto generate the workflow document using this tool. I found the image annotations in the auto generated pdf are too cluttered for my workflow. Is there a way to disable or modify tool annotations in the auto documentation tool?

 

Thanks

 

pbeeram_0-1593986542417.png

 

Matthew_H_Hughes
5 - Atom

I am getting the same error as indieching above, but I am the admin and have Alteryx Admin install running elevated. Any help would be appreciated.

jtorana
5 - Atom

Hi All, 

 

I got this error while running it. Does anyone know how to get this resolved?

 

 

2020-10-30_12-35-25.png

jdbartosh
8 - Asteroid

Thank you @levsol for sharing!

 

I attempted to run but it never finishes running. I have changed the timeout value to -1. I also opened the imager.yxwz workflow and manually edited the inputs to run and the app seems to stall out after the formula or during the python tools. Any thoughts? 

 

I am using the non-admin version.  

HarleyBarnes
5 - Atom

@levsol - Have just stumbled upon this and this is super interesting.

I have built a similar tool which outputs the Alteryx workflow into a HTML file allowing for investigation that way.

It does quite a lot similar to your tool but decouples the way the annotating is done.

 

Carto Screenshot1.pngCarto Screenshot2.png

pbeeram
6 - Meteoroid

@HarleyBarnes Is it possible if you could share the Alteryx documentation tool that you have built (images you shared)?

 

Thanks!

HarleyBarnes
5 - Atom

@pbeeram - The tool is still in development as it currently only supports a handful of Alteryx tools.

lgregor
7 - Meteor

@levsol 

 

I followed the instructions you provided for an admin install but I'm still getting the same error. Is there anything else that I can do? I'm using an Amazon Workspace if that makes a difference. I'm getting the same error posted above.

mulderbm
6 - Meteoroid

Thanks for the great work on documenting Alteryx workflows, something that helps in analyzing flows abandoned by previous builders.

 

I have a question about something I miss in the Alteryx workflow files.

When a field in a select tool (any tool with an embedded select) is not changed it is not available in the Alteryx workflow file?

I could not locate it in the XML and also notice those are missing when I run the Auto Documentation tool itself.

 

The functionality to save a configuration of an embedded select is available in Alteryx but then I have to manually save them one by one or can they be accessed through the SDK? Looked at that but it was not clear to me except that you provide an XML config for your tools. The only way I could explain this is that the Alteryx Engine keeps tabs on all incoming fields and only saves changes to each attribute when it is needed (changed in a select tool) to keep workflow file sizes limited?

 

Could someone confirm this or point me in the direction of reading each embedded select config for documentation purposes?

 

Thanks,

Brian

DataDynamite
7 - Meteor

It would be nice if the Auto Documenter also captured some workflow Meta Info and Runtime event info

 

I converted this to an interactive macro and bypassed the image information. 

I have a process that is:

1 fed a directory of .yxzp files

2. moves and unzips these to another folder

3. performs a directory input of the resulting  .yxmc, .yxmd, .yxwz files

4. executes an iterative macro, that executes the python script

5. Writes the output to table

 

With that table I can query all of the workflows at once which can answer questions like which workflows use a particular tool, reference a particular table or macro, contain hard coded emails, etc...

 

Hi everyone :)

 

I have been using the tool and I really like the output, especially at the beginning. The workflow steps make no sense to me.

 

Guillermo_Huertas_0-1652107305955.png

this one is tiny and this one huge:

Guillermo_Huertas_1-1652107358417.png

 

 

 

Is there any way to change/understand the logic of how it creates the sections?

Guille

Alayna
8 - Asteroid

Long shot but could anyone see what may be the problem

Alayna_0-1666308029836.png

 

Sarafreire_
5 - Atom

Hello, I have this error, does anyone know what is going on?

 

JupyterCode(8): Traceback (most recent call last): File "C:\Users\RS559PS\OneDrive - EY\Documents\CoE\TenX\Flat Files\Teste GL\Alteryx Workflow\Alteryx Workflow\appdata\0946039d1c61898e6d8532760257e750\workbook.py", line 1174, in <module> buildReport(readFile, outpath) File "C:\Users\RS559PS\OneDrive - EY\Documents\CoE\TenX\Flat Files\Teste GL\Alteryx Workflow\Alteryx Workflow\appdata\0946039d1c61898e6d8532760257e750\workbook.py", line 1158, in buildReport Workflow(file).parseWorkflow(mapPath).drawWorkflow(thisPath).writeObjects(thisPath).drawSubflows(thisPath) File "C:\Users\RS559PS\OneDrive - EY\Documents\CoE\TenX\Flat Files\Teste GL\Alteryx Workflow\Alteryx Workflow\appdata\0946039d1c61898e6d8532760257e750\workbook.py", line 989, in drawWorkflow self.img.paste(self.imLib[n.nodeType],(int(n.positionX),int(n.positionY)),mask=self.imLib[n.nodeType]) File "C:\Program Files\Alteryx\bin\Miniconda3\envs\DesignerBaseTools_venv\lib\site-packages\PIL\Image.py", line 1582, in paste self.im.paste(im, box, mask.im) ValueError: bad transparency mask

LukasSchoemig
Alteryx
Alteryx

Hi Levsol,

is it possible to update the links? Some of them are not reachable anymore.

Cheers

Lukas

marcus_keyrus
5 - Atom

Hi Levsol, 

 

I'd love to give this a go, but the links to the tool are broken. Is it possible to update them please?

 

Thanks,

 

Marcus

AndrewSu
Alteryx
Alteryx

@levsol , echoing the last 2 comments above.  Could we please get the updated links?

tbezold
7 - Meteor

@AndrewSu @marcus_keyrus @LukasSchoemig 

Yes, the links need to be updated by @levsol.

Try this GitLab repo:  https://gitlab.com/keyrus-us/public/alteryx_auto_documentor

However, this appears to be the deprecated original version.

AndrewSu
Alteryx
Alteryx

@tbezold @marcus_keyrus @LukasSchoemig , i found this which i think is the most recent version, but it still seems to be around 2 years old.  I just tested it with 2022.1.1 Designer and it worked fine.  

 

assets:https://gitlab.com/keyrus-us/public/alteryx_auto_doc_revamp

 

video tutorial: https://zoom.us/rec/play/1L-JA19wt7LNmBfKspGMCsdkNw9_8khjDRW_VBdH8pd2uh3zovj2RgdGpb5-_j6oaX39OguCs89...

 

PhaniV
8 - Asteroid

Is this link working for anyone to install, I tried and it returns with empty search or broken gitlab site. Any insight/idea on where to find the download is much appreciated.

AndrewSu
Alteryx
Alteryx
PhilippBrinckmann
Alteryx Alumni (Retired)

Hi everyone, 

I have 2 questions regarding the auto documenter tool:

- if the workflow runs longer than 59 seconds the auto documentation does not work, i.e. documentation stops after 59 seconds. any ideas on how to solve this?

- does the auto documenter also work with CREW Macros, intelligence suite etc.?

Best,

Philipp

IMSL
7 - Meteor

@levsol 

Hi,

Thank you for the very nice solution that you offered here. I have two questions:

- Is there any way to change/understand the logic of how it creates the sections?

- Is it possible to add the tool Name to the output?

IMSL_0-1685081900647.png

 

Best regards,

Iman

esoden
8 - Asteroid

We had clients struggling with documentation. They needed something that supported all tools and was accepted by IT and Audit. They also needed something that kept up with all of Alteryx changes over time. 

 

We built a tool that is sold by us and you can also buy it from Alteryx. Workflow Admin Manager (WAM) is the result! 

 

Tons of information and that ability to submit a Workflow online and get a detailed document online here. 

https://capitalizeconsulting.com/workflow-admin-manager/

 

It does cost money but it's affordable, supported, and goes far beyond just documentation. 

amtraker1777
7 - Meteor

We were working directly with an Alteryx Data Partner on developing an Auto Documentation Tool for us that tries to replicate what an ACL log file would look like. The tool requires the use of control containers to force linear processing since that is how the tool determines how to output the order in top-down fashion. We are very close to getting it to work, but have hit a road block when it comes to getting information to output in the correct order. That Data Partner was willing to do the work at no cost up to a certain point, but now they are requiring billable hours. I wish Alteryx would support the development of a tool like this out of the box instead of leaving up to the community to create their own given the complexity. As I can tell there are several efforts happening simultaneously, but none appear to be both free and up-to-date.

DataDynamite
7 - Meteor

Forgot, to mention in my above post that we also have a follow up job that executes after the workflow parse routine parses all of our alteryx workflows, apps and marcros.  This follow job executes a recursive table valued function in SQL Server, basically this job creates a table of ordered inputs and outputs.  With that table we are able to determine what workflow inputs lead to lead a workflows outputs (ignoring what's in between).   The workflow parse is relatively fast, the recursive table valued function is the slow part of the process.   We only parse and order what's new or has changed since the prior Sunday.

nsbaum
5 - Atom

Running the image workflow and getting this error: 

JupyterCode (8): Traceback (most recent call last):
File "C:\Users\NICHOL~1.BAU\AppData\Local\Temp\Engine_13984_817387f41ab243d6a70a9d66d31f8d5b_\46b697aaa1cbb3829775e70f89e35adc\workbook.py", line 1174, in <module>
buildReport(readFile, outpath)
File "C:\Users\NICHOL~1.BAU\AppData\Local\Temp\Engine_13984_817387f41ab243d6a70a9d66d31f8d5b_\46b697aaa1cbb3829775e70f89e35adc\workbook.py", line 1158, in buildReport
Workflow(file).parseWorkflow(mapPath).drawWorkflow(thisPath).writeObjects(thisPath).drawSubflows(thisPath)
File "C:\Users\NICHOL~1.BAU\AppData\Local\Temp\Engine_13984_817387f41ab243d6a70a9d66d31f8d5b_\46b697aaa1cbb3829775e70f89e35adc\workbook.py", line 274, in parseWorkflow
self.collectNodes(self.doc["AlteryxDocument"]["Nodes"],self.nodes)
File "C:\Users\NICHOL~1.BAU\AppData\Local\Temp\Engine_13984_817387f41ab243d6a70a9d66d31f8d5b_\46b697aaa1cbb3829775e70f89e35adc\workbook.py", line 298, in collectNodes
n = Node(node,isChild=level, parentContainer=parentContainer)
File "C:\Users\NICHOL~1.BAU\AppData\Local\Temp\Engine_13984_817387f41ab243d6a70a9d66d31f8d5b_\46b697aaa1cbb3829775e70f89e35adc\workbook.py", line 91, in __init__
self.text += self.configuration['Caption']
TypeError: can only concatenate str (not "list") to str

 

 

Works fine for most workflows, only thing I can think of is that these ones have Macros i installed from the web in them (other workflows have my own custom macros and the imager works fine)

nsbaum
5 - Atom

Update on the above. Don't know how I managed to fix it but I did. Error is caused by an "&" included in the caption of a tool container. Removing the & removes the error. Don't know the reason behind it. would love someone smarter to explain it. 

Akansha029
7 - Meteor

JupyterCode (8): Traceback (most recent call last):
File "C:\Users\HA179TE\AppData\Local\Temp\Engine_34504_2df2305f11ec4c33b44466c5596204b0_\5d429d0bb6f01d14c9266f4841110867\workbook.py", line 1174, in <module>
buildReport(readFile, outpath)
File "C:\Users\HA179TE\AppData\Local\Temp\Engine_34504_2df2305f11ec4c33b44466c5596204b0_\5d429d0bb6f01d14c9266f4841110867\workbook.py", line 1158, in buildReport
Workflow(file).parseWorkflow(mapPath).drawWorkflow(thisPath).writeObjects(thisPath).drawSubflows(thisPath)
File "C:\Users\HA179TE\AppData\Local\Temp\Engine_34504_2df2305f11ec4c33b44466c5596204b0_\5d429d0bb6f01d14c9266f4841110867\workbook.py", line 1004, in drawWorkflow
self.d.text([n.positionX, n.positionY+nodeHeight],
File "C:\program files\alteryx\bin\Miniconda3\envs\DesignerBaseTools_venv\lib\site-packages\PIL\ImageDraw.py", line 468, in text
return self.multiline_text(
File "C:\program files\alteryx\bin\Miniconda3\envs\DesignerBaseTools_venv\lib\site-packages\PIL\ImageDraw.py", line 600, in multiline_text
line_width = self.textlength(
File "C:\program files\alteryx\bin\Miniconda3\envs\DesignerBaseTools_venv\lib\site-packages\PIL\ImageDraw.py", line 735, in textlength
return font.getlength(text, mode, direction, features, language)
File "C:\program files\alteryx\bin\Miniconda3\envs\DesignerBaseTools_venv\lib\site-packages\PIL\ImageFont.py", line 198, in getlength
width, height = self.font.getsize(text)
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 4-5: ordinal not in range(256)

 

 

 

getting this error can somebody please help

nsbaum
5 - Atom

Hard to tell exactly. Do you have a copy of the workflow you are trying to document? 

I don't know exactly, but my guess is theres a character in one of your tools that isn't a regular latin-1 character. I think maybe its a select, append, join, or join multiple tool. if you can't share your workflow, then I'd just go through all your tools to make sure they dont have special characters in them

Akansha029
7 - Meteor

Output tools are creating the issue. Other than that it is able to document the workflow