Alteryx designer Discussions

Find answers, ask questions, and share expertise about Alteryx Designer.

Date Filter vs. Filter

Highlighted
Asteroid

Hi everyone,

 

I have a quick question. This morning I tested filtering with the same date range using Date Filter vs. Filter tool.

With the same data source, Date Filter takes 11.2 seconds vs. Filter tool takes 90 seconds.

So Date Filter runs faster than Filter Tool. Do you feel the same way?

Highlighted
Alteryx Certified Partner
Alteryx Certified Partner

@marlline,

 

I had the exact opposite experience.  I generated 10,000,000 input records starting with today's date and added between 1 and 50 days to each record.  I then setup a date filter for 2017-10-12 and 2017-10-19.  I executed the workflow 3 times with 23.8 seconds as the middle time {23.7, 23.8, 23.8}.  I then swapped the date filter (deleted it) for 2 filters.  The first with a filter of less than 2017-10-19 and the second with a filter of >= 2017-10-12 .  Now the time is reduced to 8.0 seconds.  A third approach used a single filter and the result was 9.5 seconds.

 

The 2nd approach should be the fastest for 2 reasons:  First, the biggest bang for the filtering was applied as a single comparison and secondly because my starting date shouldn't have been the low date (it made things very easy on the data).

 

I would expect the date filter to be slower than the filter because it has more CODE to run through.  That filter is a beast.  It is a core tool that is very efficient.  The Date filter tool is a MACRO.  it contains 3 select tools, 2 formula tools, a Join and a filter. 

 

I can't explain why your times were so different, but I'll stick with the FILTER solution for now.

 

Cheers,

Mark

Alteryx ACE & Top Community Contributor

Chaos reigns within. Repent, reflect and reboot. Order shall return.
Labels