As your analytics work grows - you find yourself using the power of Alteryx to create shared macros. These act as an accelerator for a team because one team member can us a reusable solution created by another team member. For example - many teams need to get data out of JIRA (or some other system) so you create a connector that everyone can use.
That's going well - and now you have 20 teams all publishing canvasses to your server (possibly 100s of canvasses running in production) which make use of your JIRA connector - all good so-far!
BUT THEN - you discover an issue with the JIRA connector and you need to fix it and publish a new version!
It's at this point that you realise that the canvasses on your server which use your JIRA connector are NOT pointing to it, but they have made a copy and included this inside their canvas. So when you fix the problem with the JIRA connector - no-one gets the fix!
This is because every application uploaded to the server is a yxzp file, which zips up a COPY of all the shared macros and uses this in an isolated way.
So - in order to get the new JIRA connector (with the defect repaired) used instead of the old one you now need to:
- Download EVERY canvas on your server
- Unpack them all to expose the sub-macros being used
- Inspect them to see if they are actually an instance of the JIRA Macro
- Make a list of the owner and application IDs
- reach out by e-mail or phone to every one of these folk to ask them to republish their Alteryx workflow with your new version of the JIRA connector.
Please can we revisit this - we really do need the power of shared macros - and we also need the ability to fix and manage these like a product over time. This will have an impact on the engine (hence copying @AdamR )
- When you build a canvas using a shared macro - it doesn't store the macro itself, but rather a reference to the version on the server - unless you explicitly decide to break the connection and take a copy.
- When you check this canvas into the server - your application / yxzp does NOT include a copy of the shared macro - instead it has a reference link
- this means that Alteryx Server can now track which canvasses use this shared macro very simply
- When I fix this shared macro - I can then do an in-place update; or if the interface is not the same (i.e. different inputs or outputs) then this has to be a new version and the users will stay pointing to version 1.
This is how shared assets are managed in a micro-service world, which is the way that all of our architecture is going - and it seems important that we build this thinking into the Alteryx infrastructure too.
Given the need for administrators to be able to perform analysis and monitoring on server performance; user usage etc - it is necessary to provide full documentation for both the API and the database underlying the server so that admins can use this to good effect.
Although very limited documentation is available on the server API (https://gallery.alteryx.com/api-docs) what we're looking for is a much more fully formed and navigable experience like some of the examples below.
This will make building helper processes substantially easier; as well as allow admins to fully manage their environment.
This idea covers 2 things:
- Disabling certain tools for users
- Pushing out other tools
- If we want to disable a particular tool for everyone or for specific users - then we need to go to every single deployed machine and adjust settings or delete some DLLs.
- What we would need is to be able to take a user / group of users - and de-select certain tools or apply limits
- In our corporate environment - every designer needs to check in with the server before fully starting up. If it cannot connect to the server, it needs to terminate gracefully
- As the designer logs in - the first negotiated conversation is "What tools and capabilities and standard default settings should I have"
- If a useful tool is brought into our environment (like the JIRA connector here: https://gallery.alteryx.com/#!app/JIRA-Connector/58d87c2feffc2a0dd0b5ed8f or the CREW macros) - we want to be able to push these out to all the users
- Again - every designer checks in with the server first
- Then negotiates tools
- then downloads any missing tools or updates new versions of existing tools.
This kind of central server capability is essential in any enterprise deployment of several hundred seats.
So - one of the biggest challenges that we have with the MongoDB used by Alteryx Server is that we continually have issues with locking (where our admins have to go in and undo locks)
Additionally - the current implementation of MongoDB connectivity does not support full Kerberos authentication which means that we're on a non-compliant install (which in a large enterprise is an uncomfortable place).
Given that a very large amount of what the server does is transactional - it would make sense to have an option to use a large-scale SQL server instead of using Mongo. For large enterprise customers, there must be flexibility to allow the databases that they have large supported instances of (my strong preference would be MS SQL 2016).
MS SQL natively supports XML so all the canvasses can be stored in native format. Additionally, MS SQL allows very fast query across XML, and given the clustering and reporting capabilities in MS SQL, this would dramatically increase our ability to self-manage our infra.
Given that Alteryx is looking more and more at large Enterprise customers - a move to a large-scale clustered SQL env as the back-end would be a very positive move.
NOTE: as we consider DB options for a SQL backend - please consider your large-scale enterprise customers. For example - MS SQL or Oracle or DB2 are all much more prevalent in enterprises than databases like Postgres - so it's important to focus on the enterprise support for the DB that you choose.
When we create a shared macro on the Gallery, the desire is that we are able to:
- publish this down to users simply and seamlessly
- Allow them to use these assets in their canvasses
- Allow them to simply update to the latest version of the asset on an existing canvas.
Unfortunately - right now the only way to distribute shared macros is to create a shared folder; and the only way to manage version upgrades of a shared macro is to manually find every single usage and manually upgrade.
It would be helpful to have a central Logging tool for large & complex environments.
This would be useful to allow teams to have a central way of logging data transformation errors by error-type; severity; alteryx Canvas; etc.
Right now, there's a message tool, but this doesn't provide a way to create well structured central logging across a team; especially once deployed to a server environment.
We currently have the ability to store connections on the server which protects the credentials - however this capability does not exist for APIs for Sharepoint sites etc.
Please could you extend this to cover authentication for ALL connectors?
Please Enable OAuth 2.0/OpenID Support for Alteryx Server & Connect. Currently, it supports only AD , SAML .
Current SAML has limitations, Unable to import Security groups from LDAP/AD if SAML is enabled.
When restoring an Alteryx Gallery instance to a second box for test & dev it's highly likely that you don't want all your workflows scheduled from your production instance to run in your secondary instance.
However there doesn't currently seem to be a kill switch that you can implement up front to stop your scheduled workflows from running. The only way to disable scheduled workflows in your test gallery is to manually delete them all, which is annoying when you have hundreds.
It would be great to have a config flag to disable scheduled workflows before the service is started.
I was reading a post on the Community (http://community.alteryx.com/t5/Publishing-Gallery/Macro-sharing-Best-practice/m-p/38330) which reminded me of an idea that I had.
It would be really nice if a Gallery location could be used as a "Macro Search Path" so that macros don't need to be downloaded from the Gallery and saved locally to be used in a workflow.
So in addition to going to Options>User Settings>Edit User Settings>Macros and adding a local/network path, you could add your internal gallery information...
Use case: I need to schedule a workflow to run every 2 hours each day, with the first run at 7am and the last run at 7pm
Current solution: I need to create seven daily schedules for each hour that i need the workflow to run
Idea: Add an intraday "Start Time" and "Stop Time", or "Between" to the "Minutes / Hours" schedule type, so i can create one schedule that meets the use case requirements
Example schedule with new parameters that meets use case requirements:
Start Time = 7:00 AM
Stop Time = 7:00 PM
Repeat Every = 2 hours
Start Time could default to 12:00 AM
Stop Time could default to 11:59 PM
As large enterprises continually strengthen security around their system and data assets, we're seeing adoption of products like CyberArk's Enterprise Password Vault (https://www.cyberark.com/products/privileged-account-security-solution/enterprise-password-vault/ )
The system is essentially a central repository that secures and automatically rotates passwords for privileged accounts- things like a functional account you would use to run workflows against a certain database or set of systems.
It would be great if Alteryx could build both Server (Run As Account) and Designer (for individual database connections) integrations with a tool like that.
Following up on this post (https://community.alteryx.com/t5/Alteryx-Server-Discussions/Server-logs-do-not-include-download-tool... ), inclusion of the activity of the download tool in the Server logs would be very useful in monitoring this high-risk capability within an enterprise tech environment.
We spent almost a full day on Friday internally, and also about 3 or 4 hours of time for 3 Alteryx Server engineers (thank you all - MattH; ZachH; WayeW) and Ben Burkholder trying to get our gallery server to start.
Error message was "
Failed to construct Gallery daemon process, process exited code <9002>
This was all that was provided in the server log files; the gallery log files; and the Windows Event Viewer.
In the end, it turned out that the issue was very simple - the MongoDB had open locks in place that needed to be deleted; and then the service came up immediately.
Several man-days of our effort; as well as the precious time of the Alteryx Engineers who gave up their Friday afternoon to help could have all been avoided if the error-message on the server failure said "Login failure on server due to open locks - please clean out locks on db.locks collection" with a link to the solution or a lookup code that be researched on the Alteryx support site.
Please could we go through the server service and eliminate all the exit codes and replace with human-friendly error messages - that way we could allow this kind of issue to be self-diagnosed, and allow the Alteryx Engineers to spend their time building the next generation of Alteryx Server rather than diagnosing a table lock.
Alteryx Server does a pretty good job of tracking all the information you need in order to find out who ran a workflow at a given time, even if that workflow has been deleted from the Gallery.
However, if I need to identify what that workflow actually did on a specific date, and that version is no longer available on the Gallery, it gets really hard to track down.
There are some posts throughout the community on leveraging the MongoDB to try and reconstruct this, but in some cases the workflow is "Chunked" and becomes impossible for an end user to reconstruct.
I spoke to support about this recently and they suggested that by using the Scheduler, I could schedule the historical instance of the workflow and then "Really quickly" grab it when it was reconstructed from the temporary staging folder used by our Alteryx Server. This has a few concerns, among them, finding a way to run this workflow so that the files can be created but it has no impact on tables, for instance if the given workflow drops a table as part of its execution. Additionally, the only way to schedule with a different set of permissions in a 1-Worker environment is to change the default "Run Workflows As" to a different user, who also needs permission to access all files on the Alteryx Server. This can also impact any other Scheduled or Gallery job that is executed while this recovery is underway, effectively causing downtime to do file recovery.
These restrictions are specific enough as to be impractical for most organizations using Alteryx Gallery/Server.
I think this could be solved by adding to the Scheduler an option along the lines of "Download a copy of the workflow", available to administrators. Ideally this copy could include some important metadata, like when that version of the workflow was uploaded, who uploaded it, and its ID/Version Number from the Gallery.
This would go a long way towards making it easier to Audit and respond to requests for historical information about Alteryx Workflows on the Server, and since the Alteryx Engine can already recreate these workflows, I think that a basic version of this feature could simply save out that temporary file, rather than executing it.
Alteryx has its own version control built in - however in large Enterprise environments there is a requirement to be able to use the version control platform that already exists.
In other words - when you install the server - you are asked:
Do you want to use Alteryx for version control, or your own SVN; GIT or a 4th custom option based on API integration (i.e. build this yourself based on a defined API).
This is very important to ensure that enterprise assets are maintained under the regimen of control that our Infosec & Auditors need, and the investment in making sure that Version Control contains all the right discipline; governance; etc - is all done in the enterprise Version Control platform.
Can we please extend Alteryx to have this flexibility so that we can cater to large Enterprise customers, and also anyone else who has a strict governance & change control requirement with existing infrastructure already in place?
If this is built as an independant layer and separated out from the core server slightly - it will be possibly to flexibly add other VC systems later (or provide companies with the ability to build their own).
As part of enterprise alteryx server deployments, would it be possible to remove alteryx server controller tokens and use asymmetric certificates (X509) instead?
Wondering if there are any plans to change the RC4 encryption on alteryx server to something stronger (eg: AES, Blowfish etc)
Would be curious to see if anyone else in financial services has this challenge from a security perspective.
Be able to subscribe to a concrete App in the Gallery so you can receive an email or notification when an App is updated. Several times I have been using an App that suddenly stops working properly because there's a new version available, but actually there's no way to be aware of this unless you check the date of the last version updated.
It will be ideal if you could subscribe to the app to receive new updates notifications or if Alteryx Designer could give automatically notifications of updates in the Gallery of Apps you have installed.