360 Governance with Alteryx

[pmaier1971]

Overview

“How should we govern our Alteryx platform?” is a question we get asked very frequently. This post introduces a comprehensive yet flexible reference framework for governing Alteryx. A hallmark of this framework is that it maintains flexibility to enable rapid prototyping and quick analysis while outlining how additional controls can be applied to routinely executed, high-risk processes. This post also contains a “cheat sheet” for discussions with Audit and Compliance partners, providing a multi-layered approach to apply adequate, automated controls for the Alteryx platform and to generate audit trails at every key step of a user’s analytics journey in Alteryx.

Note: This post mostly focuses on workflows created in Alteryx Designer and run on desktop and server; while core principles will apply to our Alteryx cloud offerings as well, implementation may be different. We will provide more cloud-specific details in future posts.

Designing a modern, flexible governance framework

Defining “analytic governance”

Analytics are now a keystone of most companies, with few decisions not informed by analytics, pulling together indicators, metrics or forecasts constructed from multiple data sources. In a self-service world, tools like Excel, Alteryx, Tableau or Python scripts are developed, used and managed by business users. Moving ownership of these tools to end users provides added flexibility and agility, and since the work of business users is typically closely aligned to delivering on the Enterprise’s overall goal and strategy, providing easy access to powerful tools for end users can deliver sizable benefits. In many firms, analytics prepared by business users are integral to daily management and decision-making.

That said, the proliferation of end-user computing tools can also create risks. These tools are not subject to the same monitoring as traditional IT applications, and management may lack visibility into how integral EUCs are within the company. Spreadsheets, in particular, can house

confidential data, use macros to perform complex operations, or are infrequently checked and reviewed for errors.

Alteryx is not only a more powerful tool than most spreadsheet applications, but it can also provide much richer capabilities to govern analytics and ensure that work is well-controlled and less risky. In this post, we outline these capabilities. We define “analytic governance” (or governance of the Alteryx platform) as provisions and processes to ensure that an Enterprise manages analytics well, reducing risk and having adequate insights and oversight, notably over business-critical processes.  And, as shown below, the right setup provides desirable capabilities not just for overseeing Alteryx workflows but that are useful for better managing the company more generally.

Discussions with clients have revealed that Alteryx is used to solve a wide range of use cases. Governance requirements can not only vary by sector and company maturity, but even within a client, we see wide disparity. Figure 1 shows, from left to right, how increasing analytic complexity and risk may impact the optimal Alteryx setup. For relatively simple workflows with low risk (i.e. no use of confidential data, not a repeatable process and absent material financial or reputational risk, etc.), very limited controls are likely needed. That said, the same client (possibly even the same user) may rely on Alteryx for financial reporting, requiring a much more tightly controlled setup (see also Transforming SOX Testing Leveraging Alteryx: A Playbook).

 Figure 1: Diverse use cases necessitate a flexible approach to analytic governance

Never be surprised: Leveraging Risk-Tiering

Figure 2 gives examples of different types of risk we observe across our client base, as well as ways to mitigate each risk.

• Many of them are fairly common (such as the risk of a workflow error resulting in material financial or reputational damage);
• Others may be more likely to be overlooked (like resiliency risk, e.g. the inability to execute a workflow due to a server outage or simply staff turnover)

Note that many of these risks are not specific to the Alteryx platform; for instance, resiliency risk is arguably a concern for all end-user computing tools for which documentation is lacking. Also, not all risks are applicable to each industry. But this template may be a useful starting point to think about the types of risk your company may be most exposed to.

To understand where your risks lie, we suggest as a starting point to think about defining classifications (like Low/Medium/High) and measurements for key risk types. For instance, “material financial or reputational risks” could be defined as financial damage exceeding >$1M.  Leveraging such classifications, workflows can be scored and triaged into different risk categories (this process is also known as risk-tiering). Management and oversight of risk can then better be tailored: Simply put, the process for developing low-risk workflows should not be as complex as developing business-critical workflows, and risk-tiering, supported by automated checks, gives confidence and reduces negative surprises for platform owners.

Figure 2: Perform risk-tiering of workflows by scoring across different types of risk

Foundational principles: How we think about analytic governance

Our core governance principles are outlined in Figure 3. We take a holistic view, even including elements such as training users to make sure they understand why processes need to be followed, providing best practices to write documentation (to ensure that workflows can easily be transferred to other team members), avoiding key person risk through cross-training, and more.

Let’s run quickly through the main components: when developing a framework for good governance, we always recommend starting with the process, the risks to manage and the goals various governance features need to accomplish rather than with specific tools or functions. This ensures Good Governance by Design for the analytics journey. For example, an arms-length review process and managing servers through APIs create Segregation of Duties, avoiding that authors of workflows can promote their own work to production without anyone else having the ability to test the logic.

From a user’s perspective, a governance framework needs to be simple and transparent and give flexibility for rapid development. A flexible approach leverages the KISS principle (“Keep It Simple, Stupid”) to ensure that all users can develop workflows (#AnalyticsForAll) while directing focus to the subset of workflows that may put the Enterprise at risk. At the same time, platform owners or compliance partners look for ways to ensure that users follow Enterprise policies and do not put undue stress on the server (e.g. through malformed database queries). The best practice here is a multi-layered approach (Defense In Depth), not just a single control.

Figure 3: Core Principles Underlying our Governance Framework

Two core capabilities of Alteryx help us operationalize this framework in an automated fashion:

• Alteryx Server can be entirely managed through APIs, allowing a “hands-off” approach to managing the platform (relying on automation and creating segregations of duties).
• Alteryx workflows are stored in an XML format, which can be parsed. This enables automated checks without disrupting the user experience.

A flexible, modern framework for Analytic Governance

Let’s see how this all comes together. In what follows, we trace key steps in the lifecycle of a typical workflow. Figure 4 focuses on workflow development and testing:

Figure 4: Workflow Development and Testing

• At the very top, going from left to right, the chevrons show the main stages of developing analytics.
• The user starts by building a workflow on the desktop (lower blue box on the left). Once the development is complete, a first choice point is whether the workflow is tied to a repeatable process: If it is a repeatable process or passes a certain risk threshold, we recommend putting it on a pilot server; if it is not, it can simply be executed on the desktop.
• Assuming that the workflow is to be run on a server, the user has to pass a “checkpoint” by uploading it to the server. We recommend an arm-length review through a qualified team member to test the logic, check for documentation, and test that the workflow can be run by other colleagues as well (i.e. no embedded credentials).
• In addition, an Alteryx workflow running on a schedule can monitor the server for new uploads. Once a new workflow is detected, automated checks can be kicked off based on parsing the XML file in a very similar fashion to how customer-managed telemetry parses workflows (see Customer Managed Telemetry: Walking through our new Enterprise Starter Kit). This allows testing for specific tools, or access to confidential data, both of which may suggest that this workflow could be classified as high-risk.
• Importantly, the results of the review process and the automated checks can also be stored in a database, as indicated by the light grey box at the bottom. Maintaining a record of these tests can provide insights for managers or platform owners on the types of risk they are running.

Once the testing phase is completed, the workflow should be moved into a production environment. The key steps are outlined in Figure 5.

Figure 5: Moving workflows into Production

• As the user signals the desire to promote the workflow to a production environment, it passes another checkpoint. This provides an opportunity, if so desired, to request a manager sign-off for high-risk workflows.
• Optionally, moving the workflow to production also allows tagging workflows to be executed on specific nodes, for instance, on a high-resiliency server or if local data processing rules need to be followed.
• Lastly, the actual migration to a production environment can be accomplished hands-off via an Alteryx workflow contained in our latest “Enterprise Utilities” Starter Kit, available for download on the Alteryx Marketplace (search for "Enterprise Starter Kit" and select "Customer-Managed Telemetry"). This creates segregation of duties and keeps developers from promoting their own workflows to production. If deemed appropriate, a process can be put in place to attest to the inventory of production workflows, e.g. on an annual basis to ensure that workflows are still relevant and have owners, even if the original author may have moved on.

Figure 6 puts it all together. This is our recommended governance framework, which can be used as a starting point to walk compliance partners or auditors through Alteryx governance. Laid out this way, it is easy to introduce preventative or detective controls, notably for business-critical workflows with a material impact on the health of the company.

• An example of a preventative control is a review process, as it prevents poorly constructed workflows from being promoted
• Customer-Managed Telemetry is an example of a detective control, allowing detection of unwanted behavior of users.

Examples of several preventative and detective controls are shown in Figure 6, as well as opportunities to write out audit logs. Figure 7 provides details on all controls shown, with the numbers corresponding to the dots in Figure 6. In addition, at the bottom of Figure 7, we also provide additional suggestions and ideas, such as leveraging Auto-Documenter or the WAM tool to take the sting out of writing documentation.

Figure 6: 360 Governance with Alteryx

 Figure 7: Preventative and detective controls

Disclaimer: This view can help frame governance discussions with Audit and Compliance partners, but the exact process and controls will depend on the industry, type of analytics, and data, and needs to reflect local rules and regulations.

The main insight from Figure 6 is that at all major stages of workflow development and execution, it is possible to put preventative controls and/or detective controls in place, as well as write out audit trails. This provides full transparency into what users are executing on the platform, and because customer-managed telemetry even works on the desktop, it even allows users to gauge future risks before workflows are migrated onto the server.

We hope that this reference framework is helpful. Based on the preceding discussion, note also that controls and checks can easily be tailored to client-specific or industry-specific needs, reflecting existing or future rules and regulations.

Changing the Reference Point: Capabilities You Always Wish You Had

This governance framework provides benefits extending beyond typical risk management considerations. Here are a few additional benefits users and platform owners receive from a well-governed Alteryx platform:

• Automated risk tiering (i.e. classification of workflows according to risk levels) reduces the “administrative burden” for low-risk analytics, while bringing adequate controls to where they are needed (that is, workflows that carry high risk). Risk categories can easily be customized, streamlining oversight for low-risk workflows throughout the entire workflow lifecycle. For example, once an inventory of high-risk analytics has been built, one could apply an annual revalidation process only for these workflows, as opposed to every workflow indiscriminately.
• Many of the checks and controls can be automated and run in the background without requiring manual user intervention. Smart design of e.g. checks for using confidential data also allows building an inventory of all workflows using specific data sources. Were data connections to change, such an inventory allows pro-active impact assessments, enabling notification of all users actively leveraging the data.
• To make the review process robust, one could mandate that the reviewer be Alteryx certified. This provides not only additional assurance that the reviewer is familiar with the platform, it also encourages team members to get certified.
• Good documentation is a prerequisite for reducing key person risk and being able to transfer work and workflows across team members. One automated check that can be built into the promotion process is whether the workflow contains custom annotations (as opposed to the default annotations).

Wrapping Up

Discussing governance is a surefire way to put half of the audience to sleep, while good governance is a prerequisite for the other half of the audience to sleep soundly at night. We only scratched the surface with this post - hoping to strike the right balance between being clear and simple yet detailed enough to reassure platform owners and compliance partners that Alteryx does not only not increase risk but that its governance capabilities far surpass those of many other end-user computing tools. Alteryx’s features to automate governance and its superior documentation capabilities should make control freaks rejoice!

Hopefully, this blueprint will help reframe the governance discussion at your organization. We realize that this is a complex topic, and we welcome any feedback.

For those interested in implementing this, we should note that while many of the necessary resources are provided in our latest Enterprise Starter Kit for no additional charge (incl. workflows to automate migration between environments, workflows to parse data from customer-managed telemetry and more), but every customer is different, and customizations will likely be required. If help is needed, please contact Alteryx, and we will be happy to connect you with a partner or our Alteryx Professional Services organization to help jumpstart the discussion or implementation.