Advent of Code is back! Unwrap daily challenges to sharpen your Alteryx skills and earn badges along the way! Learn more now.

Engine Works

Under the hood of Alteryx: tips, tricks and how-tos.
KatieH
Alteryx Alumni (Retired)

(Exciting news! Version 2 of the Auto Documenter tool can be found in an updated blog post. Check out the new post by @levsol, with links to the tool, documentation, demo and contact info!)

 

Do you ever have those moments in life where one particular topic or idea seems to come up in multiple conversations with different people, seemingly unrelated and coincidental, but the idea is so specific and pops up enough in a short period of time that you think the universe just must be trying to make you pay attention? I love those moments. They make me ponder the interconnectedness of all things, people and ideas. They make me think “something interesting is going on here.”

 

I had such a moment this year during US Inspire in Anaheim. An idea was pitched in a couple of different ways, by a couple of different people, with a few different primary values being pointed out.

 

What if there was a way to automatically generate a catalog of all the summarizations and formulas that are being used in a massive workflow, so that I can quickly audit what transformations are happening?

How can we successfully communicate to others what a workflow entails, without having to take screenshots and manually write out a description of the workflow’s actions? Moreover, what do we do when that level of auditing doesn’t suffice for regulatory bodies?

What if we had an easier way to visualize the transformation path a particular data field went through from a workflow’s input all the way down to its output?

The challenges around workflow documentation come up in the solutions center; it is the targeted topic of conversation from an excited customer; it is pitched to me from another Alteryx employee over an after-work glass of wine as an “I heard this really great idea from a customer today” conversation. Ok, universe… I’m paying attention.

 

Well, on this idea, the Alteryx universe has really kicked it up a notch. Not only have you (our beloved customers) given us some great food for thought around the importance of being able to audit your workflows and document the data transformations, but we have a clever group of customers cooking up something really special. Understanding that we in the Alteryx Development Center (here in lovely Broomfield, Colorado) can’t always move as fast as we want to on all the great product ideas we hear from you, they have decided that this sounds like a problem that the Alteryx product itself can help to solve. This crafty crew of users has been collaborating to bring the Alteryx community a crowd-sourced project for building a solution in Alteryx with Alteryx. I am excited to share with you all today, project AAD (Automated Alteryx Documentor). This customer-led initiative starts here, with a beta version of this customer-created Alteryx app.

 

Huge kudos and credits go out to all of those involved to make this a reality: Raz Nistor and Kevin Maroney with Keyrus; Bryan Adel, Zach Brookler, Jean Schiele and Richard Balter with Express Scripts; Northern NJ User Group for their ideas and support! You are all rock stars!

 

Here’s how it works

 

An Alteryx Analytic App has been created to let you point to the workflow you would like to document.

 

 Click Run as Analytic App buttonClick Run as Analytic App button

 Choose your workflowChoose your workflow

Select the YXMD file you’d like to document and click Finish (hint: you may need to change File Types to search for all files). At this point, the magic starts. The workflow uses an XML reader macro to organize and parse the workflow XML into distinct data sections for each supported tool. These data points are then fed into Alteryx Reporting tools to piece together sections of your output report. Take a sip of coffee (I bet it’ll be too fast for you to have time to get a refill) and voilà! A PDF report is generated that begins telling you the story of the workflow.


Guts of the XML readerGuts of the XML reader

Guts of the appGuts of the app


Sample report page 1 (click to enlarge)Sample report page 1 (click to enlarge)

Sample report page 2 (click to enlarge)Sample report page 2 (click to enlarge)

Sample report page 3 (click to enlarge)Sample report page 3 (click to enlarge)

Sample report page 4 (click to enlarge)Sample report page 4 (click to enlarge)

Sample report page 5 (click to enlarge)Sample report page 5 (click to enlarge)


Pretty awesome, don’t you agree?! So far, the list of supported tools includes Input, Output, Formula, Select, Summarize, Sort, Join, Union, Regex & Filter. I’m hoping by now we really have your brain thinking on all the other cool functionality that could be useful here.

 

There’s so much more that could be done!

 

This is where the Auto Documentation team is looking for your help! One of the most fun parts of the project in my opinion is that customers are driving this initiative and users are coming together to make it happen. They’re looking for your help by crowdsourcing this project and asking you to bring your ideas and Alteryx workflow development skills to table. Here’s how you can get involved.

 

Next steps

 

This initial workflow will be hosted on GitLab. Improvements and updates will be incorporated into the primary version, and the plan is to release the Automated Alteryx Documentor to the Public Gallery once the app is in a great working state! From there, Alteryx users all over the world can take advantage of an automated way to include documentation as part of their workflow.

 

Need some ideas for how to contribute?

 

Here are some improvements the team has already brainstormed and would love to see added in:

 

  • How can we get around having to create an XML Parse branch for each newly supported tool? Is there a more efficient parsing mechanism that can be built in?
  • Similarly, in order to generate the report document, currently a separate set of Reporting tools must be used for each newly added tool. How can this be built more efficiently?
  • Support more tools! Where to start, you ask? Maybe you could be a snowflake and add in the Unique tool!

 

I hope you’ve enjoyed reading through this fun example of customer ingenuity; I know I certainly have enjoyed seeing it come to fruition! The excitement and creativity I’ve witnessed from those involved made me want to help, and I hope it does the same for you. There’s no better way to solve a problem than to open it up to a crew of Alteryx users and let the brilliance begin.

Katie Haralson
Sr. Manager, Product Management

I joined the Alteryx team in 2015 and have been having a great time with the Product Management team ever since. I currently lead a team of Product Managers who are focused on creating the vision for our Core Designer and Visualytics strategies. It is a pleasure to work on the Alteryx platform with such smart and motivated individuals, and I love hearing all the ways our products are making a difference in your work.

I joined the Alteryx team in 2015 and have been having a great time with the Product Management team ever since. I currently lead a team of Product Managers who are focused on creating the vision for our Core Designer and Visualytics strategies. It is a pleasure to work on the Alteryx platform with such smart and motivated individuals, and I love hearing all the ways our products are making a difference in your work.

Comments
larrysanchez
7 - Meteor

This is very cool.  I would love to see something like this for analyzing the workflows of all the users at our company.

WayneWooldridge
Alteryx Alumni (Retired)

This is a thing of beauty!!!

Atabarezz
13 - Pulsar

Great way of automating tasks... Documention is so need and so much neglected...

Joe_Lipski
13 - Pulsar
13 - Pulsar

This looks awesome and as it's all in Alteryx should be very easy to build upon!

Thableaus
17 - Castor
17 - Castor

This seems to be a handful initiative and I'm so excited to use it.

 

However, in my first try, some errors were thrown in the xml_reader macro part.

The XML Parse Tool isn't able to check tools inside Tool Containers, it just checks the container itself, and this messes up the whole workflow.

For example, the EngineSettings field is not even created as an output of the XML Parse Tool, because it's present only in the Child Nodes of the Tool Container. So, in this step, maybe and additional XML Parse Tool should be created, to parse the Child Node within it.

 

Cheers,

HildebrandoS
Alteryx
Alteryx

This is awesome!!!

DanL
Alteryx
Alteryx

Great work Katie, Keyrus, and ESI team!!

marlyn12
5 - Atom

Thanks for this. I tried running this app.  But the below error message popped up. 

 

Capture.PNG

CharlesColeman
5 - Atom

This is fantastic.  I had a need for this exact tool two days ago. 

 

You're all awesome.  Thank you.

prasadpvrs
7 - Meteor

Can I know how to get this done please

mkav9893
8 - Asteroid

Tried running the app and received the following error:

 

image.png

kjochim
5 - Atom

I got an error as well :( Super excited for this tool though!

Alteryx AutoDoc Error.png

 

Loperal
7 - Meteor

This is great!!  i just test it! 

 

It works but only one remark...so the workflow should not have containers (somebody already mentioned)... which is a pity since you use containers to organize and "try" to document in some way.

 

awesome!!

Atabarezz
13 - Pulsar

+1 looking forward for containers to be added to auto documentation

DamienMinter
8 - Asteroid
Hi @KatieH, I have been working on something similar and was just taking your guys through this at Inspire Sydney. I just saw you talk and wondered if you had a few minutes to talk about this as my organisation (large multinational bank) is very keen to get a more finalized version of this into production. Thanks Damien
Atabarezz
13 - Pulsar

+1 very important capability for auditors and risk managers at a bank or insurance company...

Landi
6 - Meteoroid

I’m trying to run the latest “Auto Documentation” workflow  downloaded from GitLab , however I’m getting the following error:

 

"There were Errors"

(164): The Entry point is invalid.

workflows_stats (263): The entry point is invalid.

xml_reader (163): The entry point is invalid

 

 

any suggestions?  possible the location of macros?

 

cheers

papalow
8 - Asteroid

@KatieH 

This is an interesting idea.  Documenting workflows is an opportunity and a challenge.   Tools to make this easier would be welcome.  Getting this to work with containers....would be great.

marinamaller
6 - Meteoroid

Is this tool available? Where can I find it? It's exactly what I've been praying for!

wildflower
8 - Asteroid

@KatieH Fantastic app! I get an error when running workflows on version 2019.1. I ran the app on a 2018.3 workflow version and it worked beautifully. Will there be an update to the autodocumenter for 2019+ versions?

KatieH
Alteryx Alumni (Retired)

Hi all! @Thableaus @marlyn12 @prasadpvrs @mkav9893 @kjochim @Loperal @Landi @papalow @marinamaller @wildflower 

I'm excited to see the responses here and excitement around the idea. I notice a couple of questions for me as well as suggestions and wanted to make send you all a quick note for clarification. Just to make sure everyone understands the intent here, this project has been and is a purely customer led initiative and not a feature that Alteryx as an organization is supporting. The hopes for socializing it here was to get a wider group involved in the idea and the contributions! I too love this idea and wanted to help socialize the customer project and to understand the feedback, as this (documenting workflows) is a topic I am excited to be gathering feedback around!

 

To try and give some suggestions for the specific questions or issues coming up, here are some thoughts:

 

  • Request for addition of containers into the documentation process - cool idea! has anyone tried adding this to the app themselves? The creators of this original version of Auto Documentation were hoping for ideas like this and that others could contribute as well. Maybe someone could try adding them into the tool and sharing out a new version if it is successful.
  • Errors when opening - I'm sorry to hear that some are having issues when running the app. Has anyone else here run into similar issues and found a way to resolve that could provide some support? This is not something I am able to help with myself, as the tool not created by or owned by Alteryx and I am not sure what to advise.
  • "PortfolioComposerRender" error - this one looks related to one or more of the Reporting tools that are embedded as part of the tool processing. A "grouping field" that is referenced is a function that you can do in the reporting tools where you batch out your report into multiple separate reports based on a grouped field (think of a scenario where you want the same report sent out to every department in your organization, with only that department's relevant information present. you could accomplish this with reporting tools by grouping your reporting data fields). I am guessing the app is expecting more than one report to spit out based on a specified group, but only one is present and it is erroring. I would suggest digging into the reporting tools of the app and seeing if making tweeks to the grouping field or functionality will resolve.
  • Question about 19.1 support - Since this isn't an Alteryx created tool and isn't supported functionality in any release, unfortunately I'm not sure what to suggest here. Maybe the error alludes to the underlying problem? You could go into tools in the app and try finding which specific tool is erroring, to tweak those settings to see if you can get it to work.

 

I love hearing all the feedback around the idea! Feel free to reach out to the team on GitLab to submit new versions of the app if anyone has built any improvements in, or to reach out in general around their project!

 

Happy Alteryx-ing!

Katie

HelmutDahl
5 - Atom

Great Job @KatieH !

Automatic Documentation is a great tool and a start in the right direction when Alteryx Connect is not yet needed.

You can write the output to a database with Alteryx basic knowledge (which I would recommend from Best Practice).

For external service providers who have to document a workflow (e.g. in PDF format) this is a good step and a simple documentation process.
I see potential for improvement in the clarity of the PDF structure. From my point of view, the tool number documentation is irritating for complex workflows. There you could think about clarity. (Image copy of the workflow).

Good start. Still room to grow.

Keep it goiing and thank you @KatieH 

MichaelSu
Alteryx Alumni (Retired)

@marlyn12  & @Landi -

 

The Entry Point is Invalid message you are receiving is because there are macros packaged inside of macros that the workflow cannot find. You need to make sure that you have all of the macros on GitLab installed in your Macros repository

 

@KatieH , is there any way that you can upload the .yxzp to GitLab? I am having some trouble myself understanding how the the macros should be configured, specifically within the XML_Reader Macro.

 

Thanks,

Mike

KylePeterson
7 - Meteor

Hello Everyone,

 

I can tell that this is an amazing tool, if you can get it to work. I know when I download all aspects of this package the Macro's aren't being configured so it's not as easy as a "plug-and-play" scenario.

 

The issue I'm running into is determining where each macro needs to be configured within other macros - mostly having issues within the AutoDocumentor.yxwz and xml_reader.yxmc (and subsequent nesting).  If anyone has been able to really make this hum along, could you maybe post a "how-to" or a brief direction of where each macro really needs to configured?  

 

Thanks!

alim
5 - Atom

Hi,

 

Similar to KylePeterson, MichaelSu, shqiptar, marlyn12 and Thableaus

 

When I try to use this tool I get following message:

 

"There were Errors"
(164): The Entry point is invalid.
workflows_stats (263): The entry point is invalid.
xml_reader (163): The entry point is invalid

 

My Alteryx version 2019.1.6.58192.

 

NeilR and KatieH  Do you have any ideas on how to fix this?

 

Also when I open the macro to see the workflow itself I get following message:

 

This workflow references macros (Macros\xml_reader.yxmc, Macros\clean_up_data_flow.yxmc, Macros\workflows_stats.yxmc) that are missing or inaccessible. Saving this workflow will permanently disable these tools.

 

Does it mean that I have to find and download these macros separately? Where should I find those macros? Am I missing something here?

wildflower
8 - Asteroid

@alim - Are the Inputs and Macros folders in the same directory as the workflow (AutoDocumentor.yxwz) on your machine? https://gitlab.com/keyrus-us/public/alteryx_auto_documentor/tree/dev_kev/AutoDocumentor

alim
5 - Atom

wildflower, thank you very much for your suggestion. It fixed the problem I was having.

 

kunanh2
5 - Atom

I am also getting the same error.

 

"There were Errors"
(164): The Entry point is invalid.
workflows_stats (263): The entry point is invalid.
xml_reader (163): The entry point is invalid

 

 

Could you please help me how did you fixed your problem

Thank you

 

kwilkins
7 - Meteor

I LOVE this tool! 

 

i'm hopeful that in the near future the bugs will be worked out. I'm more than willing to help test any changes. 

 

Thank you so much!

ChristinaFPE
7 - Meteor

@kjochim I was also getting the "PortfolioComposerRender" error and figured out it has to do with the Render tool in the Auto Documentor workflow. You need to uncheck the box that says "Group Data Into Separate Reports" in the Render (4) tool before running the analytic app.

anotherusername
8 - Asteroid

Fantastic concept!  Look forward to trying it out!

 

Data Lineage specifically asked separately about here so linked to this:

https://community.alteryx.com/t5/Alteryx-Designer-Discussions/Extract-Lineage-and-Metadata/m-p/29943...

 

A question - do Alteryx staff still monitor the forums?  It seems like that used to be a thing, but has fallen outside of the norm in the last couple of years.  Would be great to bring that back into Alteryx staff norms so that unsolved threads can be brought to a close (even if the answer to the question of whether something is:  "no, not easily - post it as an idea").

KatieH
Alteryx Alumni (Retired)

Hey everyone,

Exciting news! Version 2 of the Auto Documenter tool can be found in an updated blog post. Check out the new post by @levsol, with links to the tool, documentation, a demo and contact information! 

rohit782192
11 - Bolide

can we get this in Alteryx Market place.