This site uses different types of cookies, including analytics and functional cookies (its own and from other sites). To change your cookie settings or find out more, click here. If you continue browsing our website, you accept these cookies.
The ask is to provide some sort of "real time" status of the Alteryx server environment ... INCLUDING the ability for workflows to successfully communicate with other data sources like NAS. SQL, Oracle, Teradata, Excel.. etc etc.
We would like to identify potential environment issues before the users start complaining
The thought was to to monitor the number and types of errors being generated in the output of the AS_results collection. We should be able to quickly ID issues that are occurring
The relevant portion of this collection is only the last X hours where X may be 1 or 6 or 24 or whatever but definitely not the the entire collection
In a larger corporate environment the As_results collection can be very large ... ours takes 20 min or more to read in but as I mentioned only a small portion of it is relevant to this ask.
I would like to set a criteria on the mongo db input but nothing is exposed until the service blob is parsed and that means the entire As_results collection is processed
Does anyone have any thoughts on how to accomplish this?
I'm not sure if this is helpful in your case, but let me somewhat briefly describe what I did for my situation.
Every morning, I have to check the errors for one specific scheduled workflow that runs overnight. I need to see if one specific error exists. Like you, it's not efficient to read in the entire results collection. I first grab the Queue ID from that specific workflow that ran. I then created a macro that feeds that Queue ID into the mongo input tool, using the Manual mode:
You may have already known you can do this, and it may or may not help your specific case, but figured I'd throw it out there.
HI … I am so sorry for the late response.. I missed your reply...
Since I would be looking in AS_results for a set of workflows that meet a certain criteria ..such "completed in the last hour" .. or something like that .. I would have to build my As_results query criteria to be dynamic.. based on results from the AS_Queue query.. I will have to think on how to do that...
I have a module that first reads in from the Queue, looks for the module I need, then feeds that Queue ID to the criteria in the screenshot. Make it a batch macro using the control parameter tool to update that Queue ID in the Mongo results input.
I use a similar approach to incrementally load Queue/Result data on a 15-minute interval into a SQL environment for further analysis. Basically I query the max(id) that already exists in my SQL tables, then dynamically uses these values in MongoDB queries to only get records where ids > max(ids).
It's important to note that I only store reporting-relevant data in the SQL environment (to mitigate data volume), and then I can do all kinds of different reporting over this mini-repository without hammering the persistence layer of the application itself.
It's basically a trimmed down version of the AlteryxServerUsageReport with incremental loading capability.