Challenge #476: Extract Operational Metrics from Your Workflow Logs

[AnaMuniz]

Hi Community members,

A solution to last week’s challenge can be found here.

This challenge was submitted by Yoshiro Fujimori (@yoshiro_fujimori). Thank you, Yoshiro, for this great challenge!

Many data analysts run multiple Alteryx workflows daily. Monitoring when each workflow runs, how long it takes, and whether it encounters any issues is crucial for troubleshooting and audit readiness. Sound familiar? Then this challenge is for you!

For this week’s challenge, you are provided with a series of Alteryx log files. Configure the Directory tool provided in the workflow start file to gather these files. Also, note that to read files from a directory, you need to use a Dynamic Input tool.

Your task is to analyze these logs, extract key operational metrics, and from these metrics, determine which workflow ran the longest.

Task 1: For each log file, extract:

• Workflow name
• Start time
• Duration (in seconds)
• Number of warnings
• Number of errors
• Number of conversion errors

Task 2: Across all log files, compute summary statistics:

• Total number of jobs (log files)
• Total combined run time (in seconds)
• Total number of warnings
• Total number of errors
• Total number of conversion errors

Sort by the Seconds field in descending order so you can detect which workflow ran the longest.

Hint 1: Logs may vary in structure slightly; pay close attention to how time and errors are logged.

Hint 2: When joining the log files and the extracted data files, be aware of field names. You need to create new fields for this.

Need a refresher? Review the following lessons in Academy to gear up:

Sorting Data

Multi-Row Formula

Summarizing Data

Good luck!

The Academy Team

Download Start File

 

Download Alteryx Log Files

Download Solution File

[Kenda]

Spoiler

It looks like the log files and workflow names may be a bit mixed up in the given answer, but I think what I put together is good based on the input given

[Pilsner]

Tough challenge this week. Here's my solution:

Spoiler

Macro:

 

[martinson]

Very tough one

[Towers]

definitely a harder one for me but really fun to solve!

[alexnajm]

Done! Not sure the solution is correct in terms of lining up the correct file name to the data, but the data for the correct log file matches

Spoiler

[alineruizcampos]

Spoiler

[olga_strubbe]

Thank you, Yoshiro @yoshiro_fujimori. 

I struggle with RegEx in the Formula tool, but instead of writing multiple RegEx for operation metrics I leveraged @Kenda  expert knowledge to be able to write them all in one Formula tool and learned a bunch. 

Thank you, @Kenda 

Spoiler

[Bennu]

Nothing a few REGEX can't fix...😎

Can also use the hack of wildchars in the input tool to read all log files without the need for the directory tool which in this case is convenient because it enables you to bring the filename along with their lines of text.  I usually pick the flat files data type but CSV with \n separators also works. 

The more nagging part is parsing and then adding up mili-seconds and seconds but not too bad.

Spoiler

[balajilolla2]

Spoiler

Solution Attached