Engine Works
Under the hood of Alteryx: tips, tricks and how-tos.
By Lachlan Nichols (@Lachlan_Nichols) and Philipp Maier
Alteryx has revolutionized data analytics and automation. However, effectively managing the Alteryx platform is crucial for optimizing its potential. To better serve your needs, hear your feedback, and also share best practices we see across the industry, Alteryx held the first Governance Q&A session in the Maveryx community on Nov. 1st, 2023.
Check out the recording (passcode: s4%Yk#jD), where Lachlan Nichols and Philipp Maier discuss best practices and answer questions submitted during the session to help Alteryx customers better govern their analytic workflows on the server. And, because we did not get to answer all questions during the sessions, we answered additional user questions we felt were interesting for a broader audience below as well. For clarity, we lightly edited the questions and grouped them by topic.
We would like to thank all participants for submitting questions and hope that such events can further enrich your Alteryx experience!
Governance Best Practices and Customer-Managed Telemetry
Q: We want to be able to scan all of our Production Server XMLs for auditing purposes and to better understand data lineage.
As covered in the session, the new Enterprise Utility on Customer-Managed Telemetry shows how to implement that functionality. This utility is available for free on the Alteryx marketplace.
One callout: our large customers typically require some customization. Depending on the industry, e.g., different retention requirements on records will need to be followed, or different privacy laws will apply. That is why, for large enterprises, Alteryx recommends a session with Professional Services to walk through setting up telemetry, decide on data storage, and help customize reporting. If interested, reach out to servicesops@alteryx.com with a description of your situation so we can determine how to best help get started.
Q: I've installed the Customer-Managed Telemetry Enterprise Utility, but it doesn't explain how to start the telemetry logging. It's instructing me to point to my directory of log files, but I'm not sure how to set up that directory.
This community post explains how to set up logging.
Q: How is customer-managed telemetry supposed to be handled at an enterprise level with multiple regions?
In general, Customer-Managed Telemetry logs can be saved to a network path or log parser within a compliant region, then anonymized or aggregated, and finally combined with similarly processed logs from other regions. Reporting can then be built off the combined logs.
Q: How can one identify new workflows landed in an Alteryx sandbox environment? How does one kick off automated risk checks or set up alerts?
As covered in the session, this is possible by running a pre-defined workflow to monitor the server using the server APIs. Then, when a new workflow is detected, you can kick off alerts or parse the workflow for automated risk checks in ways similar to how the telemetry log files are parsed. The free Enterprise Utility on the Alteryx Marketplace can help outline how to process the XML.
Q: What are other governance best practices for Alteryx, especially in the areas of SOX controls and tracking and documenting version control (change management)?
This question was answered in the live session; in addition, we recommend the blog post on 360 degree Governance with Alteryx. For SOX-specific discussions, please also check out our blog post on the SOX Playbook.
Q: How can I best get started on the journey to better governance?
Individual needs may vary, but we usually suggest thinking about key risks the team or customer faces and then building a risk framework around that (many clients will already have existing risk frameworks). Big steps forward are introducing customer-managed telemetry and designing a process for promoting workflows to production. That said, we firmly believe that the key components of good governance are education and training. Additional activities related to building a community of practice, setting up review sessions, and a certification drive are highly recommended. Also, a well-designed best-practice guide for building workflows helps avoid repeating mistakes and getting new users up to speed fast. The sample work plan below provides an indication of how such activities may be planned, and if clients need help, Alteryx Professional Services is available to help with implementation.
User and Workflow Management
Q: What's a practical use of the User Groups feature?
User groups can help assign permissions efficiently. One example is users creating data connections and sharing them with the whole team, and then instead of adding team members individually, user groups allow adding the entire set of users. In practice, substantial benefits arise in combination with Windows authentication because Active Directory user groups usually reflect existing privileges across a broader set of applications.
Q: Is there any way to create some sort of hierarchy of collections with different authorizations at different levels?
This is currently not a feature within collections. Please feel free to submit the suggestion to the VOC program!
Q: Can MongoDB be used to:
a) Catalog workflows in an 'inventory' for both UAT and PRD environments?
b) If so, can it also capture and report information relating to the workflow, for instance: Author, Company and/or Legal Entity, Business Process ID and Internal control IDs, etc.?
c) Are there 'user-defined' fields that the MongoDB can capture in addition to those mentioned?
Yes, MongoDB can catalog workflows in an inventory and can capture information like author, company, etc, if the user fills out this information in the "Meta Info." Adding user-defined fields in the Mongo DB is not something we would recommend. Depending on the exact objective, one solution could be to add the required information directly in a container in a workflow template (see also next question).
Q: Does Alteryx have the option to leverage recommended workflow templates with comments boxes that include headers like Author, Date, Purpose, etc.?
Yes! This is one of the cool new features introduced in Alteryx 23.2. And, for older Alteryx versions, check out this community post.
Q: Is there any documentation on being prepared for when studios will be deprecated on what the asset management will look like with a focus on collections?
This page provides the most recent information on this issue.
Server Usage Report and Workflow Documentation
Q: What type of server usage reporting do you have to help support admins? The Server Usage Report is deprecated as of 2022.3; are there other options to help admins understand the usage on their Server environment? Things for job analysis, user management, asset management, etc.
Over the coming weeks, we will publish new ways to help better manage server usage on the new Alteryx Marketplace. Stay tuned!
Q: Migrating workflows to the server or between servers can be challenging, as data connections can differ (for example, a file path could be mapped differently to UAT and production servers, or schema names could change). Any suggestions?
There are (at least) 2 ways to accomplish this.
- The first option relies on parsing the XML workflow file. Via server APIs, it is possible to download Alteryx workflows and then leverage the XML parsing tools in Alteryx to identify the relevant connections and replace them (the new Enterprise Utility on Customer Managed Telemetry provides an example of how to parse the XML file).
- Alternatively, various options are provided by our great partner ecosystem. For instance, tools like the Workflow Admin Manager by Capitalize or ServerPAHL by Aimpoint may offer the functionality required.
Q: Where can we find the workflow summary tool? Does the workflow summary tool ever look at any data being passed through?
The workflow summary tool is available as a free download on the Alteryx community. To generate the workflow summary, the tool analyzes the workflow XML, not the data. This post details the type of information sent to OpenAI.
Data Connections, Data Lineage, and Data Security
Q: We need data owners to approve who can or can not access their data. How are you looking to handle the complexity of data source access management?
The latest release (23.2) expands on DCM functionality, including admin monitoring and additional APIs. Gallery Curators/Designer teams are able to share DCM connections, and once shared, those users will not be able to update the connection or share it further. It may be worth checking if these enhancements solve the issues encountered.
Q: We need to support row-level security data sources. Any ideas or suggestions?
Check out this post in the Alteryx community for some ideas.
Q: We are not able to see any data lineage with Alteryx, and it is a core reason we are looking to shift to dbt vs. Alteryx. Any suggestions?
This is an important issue and comes up often in discussions with clients. As discussed in more detail during the session, there are several elements to this.
- First, our Customer-Managed Telemetry feature provides full transparency into data sources and destinations, enabling users to build out full data lineage views. The new Enterprise Utility includes Tableau and PowerBI dashboards showing data sources and destinations across all workflows, and users can adopt these to build out ways to visualize lineage at the workflow level.
- Second, Alteryx integrates with various specialized data lineage tools like Manta, Solidatus, or Collibra. Connectors to these tools are available directly from the vendor, from partners, or on their respective marketplaces.
- Third, a more general point is the typical user profile of a dbt user is quite different from the typical Alteryx user: where Alteryx aims to democratize analytics by giving a low-code platform to business users, dbt requires more advanced skills, and the user case is much smaller (typically IT Professionals). Hence, the typical usage pattern is different.
Special thanks for helpful comments to Philip Burrow (@PhilipB) on an earlier draft.
