Community Spring Cleaning week is here! Join your fellow Maveryx in digging through your old posts and marking comments on them as solved. Learn more here!

Alteryx Designer Desktop Discussions

Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.
SOLVED

Find Min/Max within date range

MrSmartyPants
6 - Meteoroid

I am looking for the most efficient way to accomplish the following ...

 

I have two tables.  One that stores time ranges indicating production downtime, and one that stores incremental units produced as recorded at various timestamps.

 

Downtime:

Start TimeEnd TimeDuration (Mins)
5/26/2019 8:17:445/26/2019 8:18:441
5/26/2019 9:13:045/26/2019 9:36:2423.3
5/26/2019 9:46:445/26/2019 9:50:444
5/26/2019 9:52:045/26/2019 9:52:590.9
5/26/2019 11:13:355/26/2019 11:16:252.8

 

Production:

timevalue
5/26/2019 8:17:351104
5/26/2019 8:17:431196
5/26/2019 8:17:511288
5/26/2019 8:17:571380
5/26/2019 8:18:031472
5/26/2019 8:18:111564
5/26/2019 8:18:171656
5/26/2019 8:18:231748
5/26/2019 8:18:311840
5/26/2019 8:18:371932
5/26/2019 8:18:432024
5/26/2019 8:18:512116

 

For each record in the Downtime table, I want to capture the min and max values of the records in the Production table where the Production timestamp falls within the Start Time and End Time. This is so I can determine the number of units produced during the downtime period (Max Units - Min Units)

 

Output:

Start TimeEnd TimeDuration (Mins)Min UnitsMax UnitsUnits Produced
5/26/2019 8:17:445/26/2019 8:18:44112882024736

 

For reference, the Downtime table would typically have about 100 entries per day and the Production table would have 40-50,000 records per day.  I can see this workflow being run for date ranges up to a full year, so there could be a massive amount of records to process.

 

I am also open to other suggestions on how to achieve the same results.

 

Thanks in advance

4 REPLIES 4
Kenda
16 - Nebula
16 - Nebula

Hi @MrSmartyPants 

 

The first idea that comes to mind to solve this would be to first use an Append Fields tool then a Filter to keep only the records where the production time falls within each of the start/end times. From there, you just have to Summarize and use a Formula to calculate the difference. With so many records in the production table, though, the Append Fields tool may take a while. Try this out and let us know how it works for you.

 

TIP: Make sure you change the drop down at the bottom of the Append Fields tool to "Allow All Appends."

 

 

Capture.PNG

ponraj
13 - Pulsar

I would solve this problem using iteration macro as there are some limitation with appending data using append tool.   You just need update max of number of iteration field of interface designer with count of records there in production table for macro everytime when you run this workflowResults.PNG 

 

Find Min or Max within date range.PNGmacro screenshot.PNG

estherb47
15 - Aurora
15 - Aurora

Hi @MrSmartyPants 

I find that the Generate Rows tool can be less frustrating than Append, especially when you're appending so many records. Perhaps it's faster too (I haven't tested)

You can use Generate Rows to create entries for all of the times between the ranges in your Downtime table. Then, join to the Production table, Summarize for Max and Min, and Formula to calculate the difference.

image.png


Let me know if this helps!

 

Cheers,

Esther

MrSmartyPants
6 - Meteoroid

I tested all three suggested solutions with a larger data set.  I used a month of data. 

 

The append fields solution worked and took a little over a minute to run and required upwards of 20GB of memory while running. 

 

The iterative macro solution might have worked, but after 10 minutes and 25,000 iterations (out of 147,000 production records) it was pretty clear that it wasn't going to be the most efficient.

 

The Generate Rows solution provided the same results as the Append Fields solution but was able to crunch all the data in less than 10 seconds with no noticeable memory spike!!

 

Thanks for your help everyone!

Labels